Record the exact URL, evidence source and observation date before deciding what to change. A page opening in your browser, a successful Google live test and a Google index status answer different questions. Treating them as one result can lead to fixing the wrong problem.
For an owner reviewing a missing service page, the useful first step is a short evidence record. Keep access, crawling, indexing and search appearance separate. If you cannot obtain a particular observation, mark it unknown rather than inferring it from another test.
Ask Four Separate Questions
| Question | Useful evidence | What it does not establish |
|---|---|---|
| Can the current page be fetched? | Browser/HTTP observation, final URL and response; Google live-test evidence when available | That Googlebot previously crawled it or indexed it |
| Has Google crawled this version? | Search Console crawl date, fetched version and relevant verified server evidence | That the version is indexed or ranks for a query |
| What is Google's recorded index state? | URL Inspection's indexed information and selected canonical | The result of a new live test, fixed rank or a qualified enquiry |
| Does it appear for this search? | Exact dated query/result observation with context | Complete index coverage or a stable position for every searcher |
Google's URL Inspection documentation distinguishes recorded index information from a live test. Read the actual view and timestamp. The historical crawl and the current page may differ.
Start With the Correct Public Address
Copy the URL from the page you intend customers to use. Include its protocol and path. Record any redirect and the final address. A login page, preview site or old address may answer a different question from the current public service page.
Open it normally and record what you see. An HTTP status is useful but incomplete: a 200 response could show the wrong content or an access challenge. An error might be temporary. Retain the observation and ask the website owner to inspect the specific issue.
Do not remove login protection or security controls simply to make a test turn green. The owner needs to distinguish a page intended for public search from private material and check legitimate crawler access within that scope.
Read the Indexed Information and Live Test Separately
In the authorised Search Console property, inspect the exact URL. Record the indexed-view status, last crawl, relevant fetch/indexing information and Google's selected canonical where shown. If a field is unavailable, leave it missing.
If a current diagnostic check is needed, record the live test as a separate observation. Google's documentation explains that the live test cannot check every indexing condition or determine all recorded index states. A successful live test is not proof that the URL is currently indexed or appearing for a particular search.
Record which crawler or tool produced the test. A user-triggered diagnostic fetch is not the same event as a routine Googlebot visit.
The evidence worksheet has separate rows for these views. Add the version or change date so a reader can see whether they concern the same page state.
Distinguish a Crawl Rule From an Index Rule
A robots.txt rule concerns crawler access. Google explains in its robots.txt introduction that a disallowed URL can still appear in search if discovered elsewhere, even though its blocked content cannot be crawled normally. “Blocked by robots.txt” is therefore not interchangeable with “absent from the index”.
A noindex instruction concerns indexing. Google's noindex guidance says Google must be able to access the page to read that instruction. Blocking the fetch can prevent the instruction being seen.
Ask the website owner which rule is present, where it is supplied and whether it matches the intended public/private scope. A meta tag and an HTTP response header can provide indexing instructions. Do not prescribe a blanket robots or firewall change from one screenshot.
A simulated Googlebot user agent is not evidence of a real Googlebot visit. If server evidence is used, the technical owner must verify the crawler identity using Google's request-verification guidance, rather than trusting the name in a request alone.
Keep the Canonical and Reason Visible
If Google records a different canonical, note both addresses and inspect their relationship before concluding the intended content has no search presence. Google's canonicalisation guidance explains how similar URLs can be consolidated and how Google selects a representative URL.
For a status such as crawled but currently not indexed, record the status without inventing the reason. Google's page indexing guide explains the recorded states. That status is not by itself evidence of a robots block, penalty or a required page deletion. Review the fetched version, page purpose and other relevant evidence with the responsible specialist.
A Worked Mixed-Evidence Example
The example records are invented. They are not observations about AWE or a real customer site.
A service page opens in a browser today. An older indexed-view observation says the URL was not indexed. The owner has changed the page since the recorded crawl, and a new live test succeeds.
The defensible conclusion is that the observations differ in time and scope. The current test supports current diagnostic access; it does not replace the older indexed-view record with a claimed new index result.
The worked example record retains both observations and leaves the next index observation unknown. Its action is to review the actual version and agreed follow-up, not to announce that the page is now ranking.
Agree the Next Check and Owner
Write the narrow action: investigate a fetch error, confirm the intended rule, review canonical handling or inspect the relevant recorded version. Name an owner and review date. Keep any implementation and retest evidence separate from the original observation.
A request for indexing, where appropriate and authorised, is a request rather than a publication or inclusion guarantee. Avoid repeated requests as a substitute for understanding the evidence.
Bring the URL and observation record when you talk to Matt. A specific access, crawl or index question is easier to investigate than a general instruction to “fix the SEO”.