Scrunch home

Yes, Scrunch's Agent Traffic feature shows exactly which webpages are most visited by AI bots, how often bots visit, which AI platforms are sending the traffic, and what type of request each bot is making.

Additional context: Agent Traffic distinguishes between three bot types: training (collecting data to build AI models), indexer (building search indexes), and retrieval (fetching content for live user queries).

Example

For example, imagine a Scrunch user at a SaaS company connects their CDN or web service provider to Scrunch's Agent Traffic feature. In the Agent Traffic tab, they can immediately see which pages ChatGPT, Gemini, Perplexity, and other AI models visit most often.

They notice their pricing page gets heavy traffic from some models but not others. Meanwhile, key product pages see the highest level of retrieval traffic.

Armed with this, they prioritize auditing the pricing page for issues that may be preventing AI consumption and optimizing the product pages to ensure AI answers include important positioning information.

They can also see total bot volume versus human traffic side by side, helping them understand how large their AI audience actually is relative to human visitors.

Follow-up question: How should I use AI bot traffic data to inform my content strategy?

Scrunch recommends using the following workflow:

• Identify which URLs are most frequently accessed by retrieval bots.
• Compare those pages to tracked prompts to confirm the right questions are being monitored and to spot gaps where high-traffic pages don’t have corresponding prompts.
• Prioritize optimization or net new content creation based on high-crawl pages since they’re the best proxy for what AI is using to answer real user questions.