Crawler policy
Google-Extended and AI Overviews: Search Console controls Search presentation
Use Search Console to include or exclude your site from Google’s generative Search features. Google-Extended governs specified training and grounding uses; changing it leaves Search inclusion and rankings unchanged.

Use the Search generative AI control in Search Console to manage the specified AI Overviews features. Google dates its worldwide rollout to August 31, 2026. I would inspect the property’s effective setting first: Google-Extended governs specified training and grounding uses, leaving Search inclusion and ranking unchanged.
Ordinary Search, generative Search presentation, and model-training preferences can each have a different answer. Put the choice in the control that owns it and record the effective property setting.
Method & limits
Official Google documentation was checked October 21, 2026. Google’s help page states that the Search generative AI control rolled out worldwide on August 31, 2026. No authenticated Search Console property or setting was inspected or changed. The property hierarchy and publisher decisions below are worked examples.
Which control matches your goal?
Google’s current control inventory links the Search Console setting alongside robots and page directives. Its crawler reference describes Google-Extended as a product-use token without a separate request User-Agent. Google’s sharing controls and crawler reference support the map below.
Think about the consequence for a reader. A publisher declining generative Search presentation may still want someone searching for its title to find the ordinary result. A Googlebot disallow would change the crawl path supporting Search much more broadly.
| Goal | Relevant control | Scope or consequence |
|---|---|---|
| Manage appearance in specified generative Search features | Search Console: Search generative AI | AI Overviews, AI Mode and generative AI features in Discover |
| Manage specified Gemini training and grounding uses | Google-Extended in robots.txt | Product-use preference; does not control Search inclusion or ranking |
| Manage automatic crawling for Google Search | Googlebot in robots.txt | Search crawling, including its AI features |
| Remove a page from Google Search results | noindex, visible to the crawler | Broader than opting out of generative Search |
| Limit page previews or selected text | nosnippet, max-snippet or data-nosnippet | Page/text controls with effects beyond a Search Console opt-out |
Where is the Search Console setting?
Google places it under Settings → Search generative AI. Its documented choices are inclusion, exclusion, or inheritance from a parent property. Exclusion concerns the site’s links and content in the listed generative features; the setting does not control training or other Search ranking and inclusion signals. Setting and scope.
For a publication that wants ordinary search results but declines the specified AI features, I would open the intended property, inspect its effective choice, and record whether it is inherited. The owner can then make a targeted change with the affected domain or path visible before saving.
Check the property selector before saving. An old hostname or narrow blog prefix can produce a valid setting for the wrong scope. Keep the property identifier, saved choice, and check time together.
Why can a child property disagree with the domain?
Imagine example.com permits inclusion while https://example.com/reporting/ has a manually configured exclusion. A further property at https://example.com/reporting/press/ can have its own explicit choice. This is a worked hierarchy: the nearest configured parent matters unless the child has overridden inheritance. Google’s inheritance examples.
A domain-level change can leave an earlier manual child override in place, so inventory relevant properties before the change and read their effective choices afterward, tracing each section to its nearest configured parent unless the child supplies its own choice. The result should show the policy applied to the intended content.
Keep the property, parent, effective choice, and manual-override state in a small table. It preserves the scope a tightly cropped screenshot can lose.
What does Google-Extended still do?
Google-Extended covers the future Gemini training and grounding uses specified in Google’s crawler reference. The reference says ordinary Google agents perform the requests, so there is no Google-Extended network visit to count. Google-Extended definition.
The example below expresses a site-wide preference for those documented uses. Merge it with the existing file and test the combined policy. It leaves the Search Console decision to the property setting and leaves Googlebot’s crawl policy to its own matching rules.
Additional robots.txt group. This is not a replacement for the site’s existing file or a Search Console setting.
User-agent: Google-Extended
Disallow: /When are page and text directives the better fit?
A publisher might want one passage excluded from snippets while the rest of a page remains available, or want an entire page removed from Search. Those are editorial choices at a finer or broader level than the generative Search setting. Google documents data-nosnippet for selected text, nosnippet for page previews and noindex for Search exclusion. Googlebot must be able to retrieve the directive. Page and text directive reference.
For example, an editor could wrap a sensitive public quotation in a supported data-nosnippet element after deciding that a public quotation is appropriate at all. The attribute governs Google’s presentation of that text. Confidential material needs server-side authorization; preview controls govern the public response.
I would use the broader provider map to confirm the distribution choice before applying it. A targeted AI Overviews choice should preserve the ordinary Search result the publisher still wants.
How do you verify the change?
Keep the saved Search Console choice with its property and inheritance state. For robots or page-directive changes, retain the publicly delivered file or HTML and use URL Inspection where appropriate to check what Google received. The raw HTML and rendered DOM guide helps when the directive visible in a browser differs from the response you captured.
Google describes a propagation period for the Search Console change and warns that cached content can take longer. Record when the setting changed before inspecting subsequent results. One query or one screenshot is too narrow to judge a whole site’s outcome.
I would accept the configuration when the intended property shows the right effective choice and the public files express the owner’s other decisions. Measure traffic afterward, but keep the publishing choice precise enough to stand without a traffic promise.
Sources
- Google Search Console Help — Search generative AI controlGoogleVerified Oct 21, 2026
Official page explicitly states worldwide rollout as of August 31, 2026. No publication or modification date is displayed; rollout date is an event date, not an invented publication date. Setting, scope, timing and inheritance checked.
- Google Search Central — Control what you share with Google SearchGoogle · Updated October 21, 2026Verified Oct 21, 2026
- Google Crawling Infrastructure — List of common crawlers / Google-ExtendedGoogle · Updated October 15, 2026Verified Oct 21, 2026
- Google Search Central — Robots meta, data-nosnippet and X-Robots-TagGoogle · Updated March 24, 2026Verified Oct 21, 2026
- Google Search Central — AI features and your websiteGoogle · Updated December 10, 2025Verified Oct 21, 2026
Older page checked for crawl/snippet context. It does not list the later Search Console control; newer control-specific help and current sharing inventory govern the control inventory in this article.