Can a crawler read Aedrova's public website?
By Aedrova team · Measured and published .
What we measured
We audited the fourteen public URLs in Aedrova's sitemap before adding this report, the FAQ and About page. This is an audit of our own website, not a customer study, independent benchmark or measurement of AI recommendations.
Method
For each sitemap URL, an HTTP client fetched the page, followed redirects, recorded status and canonical tags, checked robots meta and response headers, and evaluated Googlebot access against robots.txt. We also checked the four apex/www HTTPS robots and sitemap endpoints against generated source. The sample is a point-in-time snapshot; it does not measure all locations or cold-start conditions.
Observed results
- All fourteen sampled pages returned HTTP 200 with matching self-canonical URLs.
- All fourteen allowed Googlebot crawling and supplied index/follow metadata without a noindex response header.
- Both public hosts returned the tested robots and sitemap files with HTTP 200 after an exact-path edge fallback was enabled.
- Search Console subsequently reported sitemap Success with fourteen discovered pages.
What the results do not establish
Successful fetching is not proof of indexing, ranking, citation in AI answers, uptime guarantees or product-market fit. Google's individual indexing requests and robots-warning validation were still processing at the time of this report. Google's decision to index or serve a page remains external.
Reproduce the public checks
Read our XML sitemap and robots.txt. Fetch each listed page, inspect its status, canonical link, robots meta and X-Robots-Tag header, then compare the public URLs with the sitemap. Do not probe private endpoints or submit personal data. Later results may differ as the site changes.
Sources and interpretation
Google's AI features guidance explains that ordinary search requirements apply and inclusion is not guaranteed. Google's robots guidance explains why crawler rules are not access controls. Our raw observations are first-party measurements; Google documentation supplies interpretation, not endorsement of Aedrova.
Questions or corrections: [email protected]. Continue with the product guide or FAQ.