Scrunch home

Technical issues are typically anything a web developer would handle (i.e., code). Content issues are anything a non-coder could fix (i.e., creative).

Both are crucial to AI search performance.

We recommend tackling technical issues first for one simple reason: It doesn’t matter how good your content is if LLMs can’t access and use it.

Before you start prioritizing what needs to be updated and when, start here: Make sure your website is not blocking AI traffic. You’d be surprised just how often this happens.

Once you’ve triple-checked that your robots.txt is in good working order, you’ll want to chunk out your technical optimization efforts based on webpage traffic volume (it’s much better than picking URLs out of a hat).

Log into your AI search product to see which pages are crawled most frequently by AI agents—these are the ones that are likely being mentioned (or trying to be mentioned) most in prompt responses.

Another option is reviewing citations in your AI search product to see which URLs are getting the most love from LLMs.

For example, in Scrunch you do this by going to Citations → Add filter → Citation owner → Brand.

From here you can click into your domain to see a list of the most popular pages with LLMs in descending order, including each page’s Influence Score (i.e., the percentage of responses that have cited the source multiplied by the unique number of prompts).

🔌 Shameless plug: Scrunch’s Site Audits feature does this for you automatically. You can drop in a URL and we’ll tell you if there are any issues related to access controls, content delivery, or content quality that need attention.

1. Access controls

The big question: Is your website configured to allow AI platforms to access webpages?

If an LLM can’t index or retrieve content from your site, for all practical purposes, your site is invisible.

The most common issue is related to robots.txt files. Make sure your robots.txt is telling AI user agents that they can access the right pages. You can dig more into our recommendations here.

Keep in mind: If you’re missing a robots.txt file completely, that can also create ambiguity about how AI platforms should interact with your site.

It’s also worth checking that you’re not using any tools or configurations that may unintentionally be blocking access.

Consider access controls an always-on priority. It’s very common for web dev teams to test something on a new staging site (something that they don’t want crawled just yet) and push pages live without going back and unblocking them.

2. Content delivery

The big question: Is your content delivered with the necessary technical hygiene and speed?

Put another way: Is there meaningful content on your page that can be delivered without JavaScript and does the page load in under five seconds?

LLMs read raw code, not what happens after it runs. Lots of companies use JavaScript to dynamically display content after a page loads, but LLMs can’t execute it to see what actually appears on the screen.

If content on your site loads in stages—or is hidden behind dropdown menus—AI will only get a snapshot of what’s really there.

Pages with lots of superfluous code and content eat up AI tokens. The ultimate goal is to create a token-light page that’s as easy as possible for LLMs to consume.

🔌 Another shameless plug: Scrunch makes this easy with the Agent Experience Platform (AXP), which generates a parallel, AI-friendly version of your website for AI traffic.

Meanwhile, crawlers are known to deprioritize or skip slow-loading pages. That makes quick content delivery key to visibility.

3. Content quality

The big question: Is your content complete and in a format optimized for AI platforms?

This is where code starts to bleed into the creative.

For instance, pages need to be short enough to be completely read by LLMs. Page titles and descriptions need to be relevant to the content of the page. And if you have a non-JavaScript version of the page, it needs to substantially be the same as the JavaScript version.

We’ll dig deeper into content optimization below. Just remember: LLMs want answers, not clickbait. Clear, relevant, and accessible content wins the day.

Making technical updates

Many technical updates are relatively quick fixes.

Web devs can update robots.txt files. Content teams can update the lengths and titles of pages.

And any solid AI search product should proactively alert you to issues and make recommendations.

🔌 Another shameless plug: Scrunch takes the guesswork out of technical and content updates with our Content Optimizer feature. We’ll analyze any page you’re auditing to understand its intent, let you choose a specific persona to target for optimization, suggest which prompts to optimize the page for, and recommend which other pages on your website you may want to pull content from.

Once you review and approve (or edit), we’ll generate an optimized version of the page in markdown that you can manually add to your CMS or deliver directly to LLMs via our Agent Experience Platform (AXP).

How do I prioritize content updates?

When you think of content updates, you probably think of creating net new content. But that’s not always the most impactful place to start.

That goes double if you’re planning to fill in content gaps with low-quality AI-generated content (aka slop). Churning out AI-authored content might give you some short-term visibility gains, but over time, AI platforms are likely to reward high-quality, human-written (or at least human-edited) content.

Research shows that AI-generated content leads to model collapse. Garbage in equals garbage out.

The companies behind the models aren’t big fans of that. Meaning they’ll likely reward original, expert-written content in the long run.

We recommend starting with the content already on your site—and using citations as your guide. Citations are a map to the sources LLMs already know and trust.

Also keep in mind that the pages most frequently crawled by AI agents and the ones getting the most organic SEO traffic (there’s a good chance that they’re the same pages) give you a solid starting point for beefing up existing content versus starting from a blank page.

Identifying and enhancing pages that AI platforms already see as somewhat authoritative is going to yield faster results than creating net new pages.

1. 3rd-party citations

There are two kinds of third-party citations to go after: ones that don’t feature competitors and ones that do.

The former are an untapped way to show up in AI search results.

If you want to get your brand mentioned, you can reach out to third-party sources to get a placement (this is easier to do if the source in question is a listicle or if you can add productive insights to the content).

The latter—third-party citations that mention competitors—work the same way.

If a third-party source is willing to mention a competitor, there’s a decent likelihood that they’ll do the same for you. Use the same approach as above to get listed.

Another option is to create your own content to better address the prompt in question. This tactic is definitely more time-intensive, but, if successful, you’ll get a new citation as well as a mention.

If the page currently being cited is substance-light or there’s an obvious discrepancy between the title of the page and the actual content, it may be possible to beat it for the citation fairly easily.

2. Competitor citations

There’s an almost-zero chance that a competitor will mention and backlink your brand if they’re a cited source (at least not in any positive way).

In cases like these, creating your own content is likely the best bet. It’s simply a matter of determining how well your competitor answers the prompt and weighing whether devoting the time and resources is worth it for the prompt in question.

In other words: You don’t necessarily need to sweat prompts that are only tangentially related to your brand or seem like they may not be getting a ton of volume.

3. Brand citations

Congrats! You’re already a cited source. Now it’s worth evaluating the content and finding ways to optimize it further to hang on to your citation.

Balancing volume vs. intent

When building your citation strategy, you’ll want to focus on the citation sources that will have the biggest impact.

Sources like Reddit, Quora, and Wikipedia probably come to mind. They’re often highlighted as among the “most-cited sources” for LLMs.

There’s definitely volume there at the top of the funnel (think high-level educational prompts like, “What is a CRM?”). But just because a prompt might see a lot of volume doesn’t mean it will lead to conversions.

In fact, we’ve found that the further down the funnel you go, the more likely LLMs are to cite more niche websites.

With that in mind, we recommend striking a balance between volume and purchase intent (i.e., how likely someone is to use a prompt vs. whether that prompt indicates likelihood to buy).

These types of prompts usually land at the evaluation or comparison stages of the funnel (get a quick refresher in the Prompt Tracking Framework section in chapter 2). Think prompts like:

• “What are the key features of Microsoft Dynamics 365 for enterprise companies?”
• “What do customers say about Zoho implementation?”
• “Compare Salesforce vs. HubSpot for a mid-sized B2B SaaS company - which has better support options?"
• "I'm deciding between Monday.com and Copper CRM for a Google Workspace-based team. Which integrates better?"

🔌 Another shameless plug: Scrunch makes it easy to filter prompts by funnel stage, along with persona, region, and custom tags.

Once you’ve got a handle on which later-funnel-stage prompts make sense for your brand, you can prioritize citations based on which sources are most often cited for the prompts in question.

Get cited by the source or beat them for the citation. Wash, rinse, repeat.

How do I scale my AI citation strategy?

If you’re feeling overwhelmed at the prospect of increasing citations by hand, there are purpose-built tools to help. We won’t name them all, but here are a couple we work with to give you an idea:

Noble

Noble automates outreach, negotiation, and payment to help get you mentioned in publications and forums that are being cited by LLMs. This helps take a lot of the grunt work out of “citation outbound."

Stacker

Stacker automates the process of getting native, non-sponsored placements in various publications. Data shows that content picked up by multiple news outlets can increase AI citation rate by up to 325%.

How do I make content updates?

There are two ways to update content for AI search: optimize existing content or create net new content.

1. Optimize existing content

The most straightforward way to do this is to analyze the content already being cited for the prompts you care about.

Investigate how it differs from your existing content in terms of:

• Length and depth
• Answering a specific question
• Recency for time-sensitive questions
• Proprietary information versus secondary sources
• Expert opinion and verified data versus speculation
• Unique facts or statistics that aren’t common knowledge
• Information that directly supports or contradicts related prompts

In general, you can improve page performance by:

• Structuring content to make it easy for LLMs to read and prioritize (header tags, markers in the text, etc.)
• Optimizing schema and metadata to make it easy for LLMs to understand the meaning and context of content
• Adding FAQs to the content to address the prompt(s) that match the page intent
• Updating the page with a fresher point of view or new data, links, and resources

Pro tip: Ask AI. You can tell your favorite LLM which prompts you’re trying to show up for, run your content through it, and ask for augmentation suggestions.

2. Create net new content

After you’ve picked the proverbial low-hanging fruit of existing content, you can turn to net new content creation.

We recommend starting with greenfield prompts (aka prompts where your competitors don’t show up). These are open opportunities.

Next, move to prompts where competitors are present but your brand is not. It’s smart to prioritize based on substance-light content (i.e., lower-quality competitor content that will be easier to beat).

🔌 Another shameless plug: Scrunch makes it easy to find both. You can filter prompts by the presence (or lack thereof) of your brand and your competitors’ brands.

And that’s all she wrote for chapter 3. In chapter 4, we’ll cover how to deliver content directly to LLMs to maximize optimization efforts.