Excluded by ‘noindex’ tag: what it means and how to fix

Excluded by ‘noindex’ tag means Google crawled the page, found a noindex rule in a meta tag or HTTP header, and obeyed it by leaving the page out of search. Google is doing exactly what the page asked. Whether that is a problem depends only on whether you meant it.

What does it mean?

Google’s help page says that when Google tried to index the page, it found a ‘noindex’ directive and therefore did not index it. The rule can be either of these:

<meta name="robots" content="noindex">

X-Robots-Tag: noindex

Learn more in the glossary entry on noindex.

Is it a problem?

For checkout, account, thank-you, internal search, staging and thin archive pages, noindex is the right call and this list is just confirming it. It is a serious problem when it covers pages that should rank. A noindex that slips into a shared template can remove a whole section, or a whole site, from Google within weeks, so a sudden rise in this count deserves an immediate look.

What causes an unwanted noindex, most likely first?

1. A CMS visibility setting, such as WordPress’s “Discourage search engines from indexing this site”, left on after launch.
2. SEO plugin settings that noindex a whole content type, category or taxonomy, or a single page by mistake.
3. Staging configuration deployed to production, including an X-Robots-Tag header set at the server or CDN.
4. JavaScript that adds or changes the robots meta tag after the page loads.
5. Template logic that noindexes pages under a condition, such as low stock or few words.

How do I fix it?

1. 1. Split the list into intended and accidental

   Export the examples. Login, cart, thank-you, internal search and tag pages are usually noindexed on purpose. Product, service, article and landing pages in the list are almost always accidents.

2. 2. Find where the noindex comes from

   Check the page source for a robots or googlebot meta tag containing noindex, and the HTTP response headers for X-Robots-Tag: noindex. Also check the rendered HTML in URL Inspection's Test live URL, because JavaScript can add a noindex that is not in the source.

3. 3. Remove it at the source

   Turn off the setting that adds it: the CMS 'discourage search engines' or 'hide from search' option, the SEO plugin's per-page or per-type setting, a template condition, or a server or CDN rule that sets X-Robots-Tag.

4. 4. Make sure Google can recrawl the page

   The page must not be blocked by robots.txt, must return 200, and should be in your sitemap and linked from indexed pages.

5. 5. Request indexing and validate

   Request indexing for the key pages in URL Inspection, then click Validate fix on the reason in the Page indexing report.

6. 6. For pages that should stay hidden, tidy up

   Keep the noindex, but remove those URLs from the XML sitemap so you are not asking Google to index pages you are also telling it not to index.

How do I check that the fix worked?

Run Test live URL in URL Inspection: under Indexing allowed? it should say Yes. After Google recrawls, the inspection shows “URL is on Google”. In the report, Validate fix moves from Started to Looking good and Passed as the examples are reindexed.

How long does it take?

Requested pages are often recrawled within days. For pages Google visits rarely, Google’s documentation warns it can take much longer for it to see a change, which is why requesting indexing for the important pages helps. Traffic usually returns once pages are reindexed, though rankings can take a little time to settle.

The free SEO checker shows whether a URL carries a noindex in its HTML or headers.