Powered by KwikChexHow ChatGPT, Claude, Perplexity and Google AI find their sources
By About This Business editorial team · Published · Updated
Short answer
ChatGPT, Claude and Perplexity each run their own search crawler and index, and fetch pages live when a user asks. Google's AI Overviews and AI Mode draw on Google Search's normal index. All of them retrieve candidate pages, choose a few readable, relevant sources and combine them into one answer. Being crawlable and clear is the precondition. Nothing guarantees a citation.
Key takeaways
- Each assistant has its own crawler and index, and also fetches pages live for some questions.
- Google's AI features use Google Search's index, with no extra requirements.
- Ranking well in one system doesn't guarantee being picked by another.
- Clear, specific, current passages are what assistants find easiest to use.
What are the basic steps every AI assistant follows?
Interpret the question, retrieve candidate pages, choose a few sources, and write a combined answer. The details differ by assistant, but the loop is the same. This article looks at where each assistant gets its candidate pages. For the bigger picture, see How AI assistants decide what to say about your business.
Where does ChatGPT get its sources?
From OpenAI's own search crawler, OAI-SearchBot, from live fetches when a user asks, and from outside search providers. OpenAI documents separate crawlers for separate jobs: OAI-SearchBot surfaces websites in ChatGPT's search features, ChatGPT-User fetches pages for specific user requests, and GPTBot collects training data. OpenAI states that sites opting out of OAI-SearchBot won't be shown in ChatGPT search answers, though they can still appear as navigational links.
Where does Claude get its sources?
From Anthropic's search crawler, Claude-SearchBot, and from live fetches by Claude-User when a user's question needs a page. ClaudeBot is separate and is used for model training. Anthropic says disabling Claude-SearchBot may reduce your visibility and accuracy in Claude's search results, and disabling Claude-User stops Claude retrieving your pages for user questions.
Where does Perplexity get its sources?
From its own index, built by PerplexityBot, plus live fetches by Perplexity-User. Perplexity says PerplexityBot surfaces and links websites in its search results and isn't used to train foundation models. It recommends allowing PerplexityBot if you want to appear.
Where do Google's AI Overviews and AI Mode get their sources?
From Google Search's normal index, the same one used for regular results. Google says there are no additional requirements to appear in AI Overviews or AI Mode: a page must be indexed and eligible to show with a snippet. Ordinary SEO foundations apply, such as crawlable pages, internal links, important content in text and structured data that matches the page.
How do the assistants compare?
| Own crawler/index | Live fetch for users | Extra requirements | |
|---|---|---|---|
| ChatGPT | OAI-SearchBot | ChatGPT-User | Allow OAI-SearchBot |
| Claude | Claude-SearchBot | Claude-User | Allow Claude-SearchBot |
| Perplexity | PerplexityBot | Perplexity-User | Allow PerplexityBot |
| Google AI features | Google Search index | — | None beyond normal Search eligibility |
What does this mean for a business owner?
Make sure every assistant can reach you, then make your facts the clearest and most consistent version available.
- Allow the search crawlers. See AI crawlers and robots.txt.
- Put key facts in plain HTML text, not only in images, PDFs or scripts.
- Write answer-first passages that make sense on their own.
- Keep facts consistent everywhere. See When AI gets your business wrong.
- Date your facts so assistants can tell current from old. See Why every business fact needs a source and an "as of" date.
Part of How AI assistants decide what to say about your business.
FAQ
Does ChatGPT use Google's results?
OpenAI documents its own search crawler, OAI-SearchBot, for ChatGPT search, and ChatGPT search also works with outside search providers. Don't assume a high Google ranking means ChatGPT will use your page, or the reverse.
Why do different assistants give different answers about my business?
They use different indexes, run different queries and choose different sources. If your facts are inconsistent across the web, each assistant may land on a different version.
Do assistants prefer recent content?
For many questions, current information matters, and dated content makes it clear what's current. Visible dates help both readers and machines judge freshness.