Agent Experience
Monitoring & Insights
AI visibility, insights, and action—any scale.
Help your clients navigate the AI-first customer journey.
Scrunch recommends addressing four technical issues that cause AI assistants to skip site content: blocked AI bot access, JavaScript-dependent content, slow page loads, and pages exceeding the AI model's token limit.
Additional context: Most AI crawlers cannot execute JavaScript. They may deprioritize pages that load slowly. They also stop reading content once the page exceeds their token limit.
For example, a Scrunch user initiating our site auditing feature may identify the following technical issues:
Access controls: Missing or misconfigured robots.txt files can block AI user agents entirely. Bot-blocking rules in firewalls and WAFs have the same effect, preventing crawlers from reaching any page on the site.
Content delivery: JavaScript-dependent content is effectively invisible to most AI crawlers, which cannot execute scripts or wait for dynamic rendering. Slow page loads compound this problem by causing bots to abandon the page before reading it.
Content quality: Token-heavy pages get cut off before the AI finishes reading. Unclear or undescriptive metadata makes it harder for AI to understand what a page covers, reducing the chance it is cited.
Scrunch recommends prioritizing technical fixes first. No content optimization will help if AI cannot reach the page. Access and delivery blockers should be resolved before addressing content quality.
Follow this order:
Once technical blockers are resolved, shift focus to content improvements. Target the specific topics where the brand is underperforming in AI search results.
Scrunch supports your company at every stage of growth — from first tracking AI citations to scaling a full AI search presence across ChatGPT, Perplexity, and Google AI Overviews.
