— 'Listed or noindex, both ways' — every route resolves to exactly one of the two states.
— A robots disallow does not remove a URL from an index; the noindex does.
— The sitemap table and the crawler list come from one source, so they cannot drift.
The ambiguity
The audit walked every built route and asked two questions: is it in the sitemap, and does it tell crawlers to stay away? A hundred routes answered neither. Some were admin surfaces with no robots metadata; some were public pages missing from the list; all of them were ambiguity, and ambiguity is how stale pages end up in search results.
The two rules
Rule one: every indexable route is listed, and every unlisted route is noindex — checked both ways, so the sitemap and the metadata cannot disagree. Rule two: a robots disallow is polite, the noindex is binding — the admin console carries the binding one.
The gate that runs both rules fails on a route that is neither listed nor noindex, and the failure names the route. Ambiguity became a build error.
Practise the lesson
Theory sticks when you ship it. These original Motif assets put this guide's lesson to work — open one and copy it into your own page.