Google-Extended: control Gemini without leaving Search
Google-Extended is a robots.txt token, not a crawler. It controls Gemini training and grounding; blocking it does not affect Google Search.
Updated
Google-Extended is a robots.txt product token, not a separate crawler. Google describes it as a standalone token publishers can use to manage whether content Google crawls from their sites may be used for training future Gemini models and for grounding. Google says it does not affect a site’s inclusion in Google Search and is not a ranking signal.
| Owner | |
|---|---|
| Purpose | Training opt-out token |
| robots.txt token | Google-Extended |
| Follows robots.txt | It is a robots.txt token |
| Published IP ranges | developers.google.com/static/crawling/ipranges/common-crawlers.json |
| If you block it | Content isn’t used for Gemini training or grounding. Google Search, including AI Overviews, is unaffected. |
There is no Google-Extended user-agent string. Requests come from Google’s regular crawlers, so the IP list above is the one for Google’s common crawlers.
What does Google-Extended control?
Whether content Google crawls from your site may be used to train future Gemini models and to ground Gemini’s answers (giving the model fresh information from your pages when it responds). It does not change how Googlebot crawls, what it indexes, or how your pages rank.
What does blocking Google-Extended not do?
- It does not remove your pages from Google Search.
- It does not remove you from AI Overviews or AI Mode, which are Search features. Google’s documentation points to snippet controls (
nosnippet,max-snippet,data-nosnippet) andnoindexfor those. - It does not stop Googlebot from visiting.
How do I block or allow it?
User-agent: Google-Extended
Disallow: /User-agent: Google-Extended
Allow: /Never use User-agent: Googlebot with Disallow: / to opt out of AI. That removes you from Google Search as well.
How do I verify Google’s crawlers?
Since Google-Extended has no traffic of its own, verify the underlying Google crawler: do a reverse DNS lookup on the IP, check the hostname ends in googlebot.com, google.com or googleusercontent.com, then do a forward lookup to confirm it resolves to the same IP. Or match the IP against Google’s published JSON lists.
Should you block Google-Extended?
It costs you nothing in Google Search, so it is the lowest-risk AI opt-out there is. The trade-off is Gemini itself: if you want the Gemini app to draw on your pages when it answers, keep it allowed. Check your current setting with the AI crawler checker.
Sources
Common questions
Does blocking Google-Extended remove me from AI Overviews?
No. AI Overviews and AI Mode are part of Google Search, and Google says Google-Extended does not affect inclusion in Google Search. To limit what Search shows, including in AI features, use nosnippet, data-nosnippet, max-snippet or noindex.
Will I see Google-Extended in my server logs?
No. Google says Google-Extended doesn't have a separate HTTP user-agent string. The crawling is done by Google's existing crawlers, such as Googlebot; Google-Extended only tells Google how it may use what was crawled.
Does Google-Extended affect my rankings?
Google says it is not used as a ranking signal in Google Search.