Use this guide if you need to: - See how AI agents consume your website - Analyze page-level AI search performance - Find and fix underperforming pages in AI search - Systematically improve brand visibility in AI answers - Prioritize AI search optimization efforts
Identify website optimization opportunities for AI search from Scrunch’s Site Maps tab.
Start in Site Maps to spot high-impact pages: - Audit Score: Page quality based on content, structure, and links - Agent Traffic: Number of requests made to the page by AI agents - Citations: Number of times the page is cited in AI responses - AI Referrals: Number of visits originating from AI platforms
Drill into Page Details to act: - Performance: Platform-level breakdowns for agent traffic, citations, and AI referrals - Diagnostics: AI agent access, activity, and traditional search index status - Prompts & Responses: Which prompts do and don’t cite this page - Deep AI Audit: Findings across access controls, content delivery, content quality, and alignment - Optimizations: Create and deliver AI-optimized content; see change history
Track changes over time with the date selector (default: last 12 weeks). Prioritize pages where AI agents visit but don’t cite you.
Why this matters: Websites built for human persuasion often underperform for AI. AI agents don’t “see” animations, pop-ups, or brand voice—they extract facts from clear, structured, machine-readable content. Optimization is how you turn monitoring into action: fix what prevents AI from accessing, parsing, and citing your pages.
Before you begin: - Log in to Scrunch or start a free 7-day trial to follow along. - Add your brand and domains in Scrunch’s AI Context. - Customize brand context (alternative names, competitors, personas, topics). - Add prompts to track in Prompts Monitoring, or convert SEO keywords into prompts. - Connect your website to Agent Traffic and your Google Analytics account to AI Referrals. - Ensure your security/CDN settings don’t block Scrunch’s user agent.
Open the Site Maps tab. Scrunch crawls and maps pages as AI user agents see them and refreshes page metrics throughout the day.
Use the table to assess each mapped page: - Audit Score - Agent Traffic - Citations - AI Referrals
Filter or sort by any metric and adjust the date range to see trends. Besides low Audit Scores, pages with high AI agent traffic but low citations are prime candidates for investigation and optimization.
Open a page’s Details to access five work areas: - Performance: Compare agent traffic, citations, and referrals across AI platforms. - Diagnostics: See which AI agents can access the page, recent bot activity, and search index status. You’ll also see what humans see versus what AI bots read in the raw HTML. - Prompts & Responses: Tracked prompts that do and don’t cite this page, with filters for topic, persona, branded vs. non-branded, and platform. Prioritize high-intent, non-branded prompts aligned to core product categories. - Deep AI Audit: Diagnose access, delivery, content quality, and alignment issues in seconds. - Optimizations: Generate AI-ready content and deliver it via AXP, or get actionable guidance to improve your canonical page if you’re not using AXP.
Pro tips: - Unless your content is monetized, allow AI bots to crawl your site. The more high-quality, accurate data LLMs can retrieve, the more likely you’ll be represented correctly. - Focus first on pages with “high impact, low performance”: high bot traffic + low citations, low Audit Scores, or meaningful AI referrals you want to protect.
Start optimizing your website: 1) Run a site audit - Use Deep AI Audit to fix access, delivery, or content issues that block AI consumption. See “AI site audit vs. SEO crawl” note below.
2) Prioritize technical hygiene - Ensure AI bots can access and read page content before reworking copy or creating new pages.
3) Update or create content - Review currently cited competitor sources for priority prompts and produce content that is clearer, more complete, and better aligned to those prompts.
4) Deliver AI-optimized content directly to LLMs - Use Scrunch’s Agent Experience Platform (AXP) to serve clean, structured, token-light page content to AI bots at the CDN level without changing the human experience.
Track improvement over time by monitoring brand presence, citations, AI agent traffic, and AI referrals. Not a customer yet? Start a 7-day free trial or talk to our team to see Scrunch in action.
Reduce superfluous code and keep content token-light for fast, efficient retrieval.
Remove access blockers
Allow major AI user agents in robots.txt and security tools unless you have a strong reason not to.
Structure for retrieval, not persuasion
Keep metadata relevant and consistent with on-page content.
Align content to real prompts and personas
Track high-intent, non-branded prompts. Close gaps where AI visits but doesn’t cite you.
Fix delivery issues first, then iterate content
Address access and delivery problems before rewriting pages. Then improve clarity, completeness, and alignment.
Consider dual experiences
Traditional SEO crawls target SERP rankings and human experience (e.g., broken links, meta tags, speed). Scrunch’s AI site audit targets AEO/GEO outcomes—verifying AI bot access, parsing quality of raw HTML, delivery hygiene, and alignment to prompts—so LLMs can easily consume, understand, and cite your content. For details, see how Scrunch’s audit differs from an SEO crawl in our FAQ and the Site Maps announcement.
Beyond Profound, these vendors are most often considered for brand optimization in AI search: - Scrunch - Monitoring, auditing, optimization, and content delivery via Agent Experience Platform (AXP) that serves AI-optimized content directly to LLMs. - Does not currently include AI-generated content. Explore the Scrunch overview or see the full best AEO/GEO tools list.
Learn more: Adobe LLM Optimizer.
AthenaHQ
Learn more: AthenaHQ.
Bluefish
Learn more: Bluefish.
Peec AI
Learn more: Peec AI.
Semrush AI Visibility Toolkit
Notes: - Content delivery to AI agents is a key differentiator. Today, only Scrunch and Adobe provide AI-specific content delivery according to our market analysis. - The best way to choose is to test. If a free trial isn’t available, ask vendors to demonstrate live: - Creating a brand on the fly and tracking business-relevant prompts - Creating and monitoring custom prompts without support - Filtering and sharing data the way your stakeholders need it - Security and scale (SOC 2 Type II, RBAC, SSO, multi-brand support)
How do I prioritize page optimization based on Site Maps? - Start with “high impact, low performance” indicators: - High agent traffic but low citations - Low Audit Score - High AI-referred traffic you want to protect - Then run Deep AI Audit and use Optimizer to update the page or generate an AI-optimized version for AXP delivery.
What products does Scrunch offer for AI search optimization? - Monitoring & Insights, AI Site Auditing, Content Optimization, and AI-specific content delivery via AXP. You can see an end-to-end workflow in the Site Maps announcement.
Can I monitor specific URLs in Scrunch? - Yes. Track specific URLs across all prompts and filter by branded/non-branded, topic, persona, country, AI platform, funnel stage, and citation owner.
Ready to try this workflow on your site? - Spin up a free 7-day Scrunch trial or book a demo with our team.