What it does
- Reads your robots.txt rules and Sitemap directives.
- Detects your XML sitemap and counts the URLs it lists.
- Resolves sitemap indexes to their child sitemaps and real page count.
How it works
- Enter your domain.
- We fetch robots.txt and sitemap.xml and parse both.
- Review crawl rules and sitemap coverage, and fix any gaps.
Why use a robots.txt & sitemap checker?
Before Google can rank your pages, it has to crawl them. A single wrong Disallow rule can deindex your site, and a missing sitemap slows discovery.
Best practices
- Reference your sitemap in robots.txt with a Sitemap: directive.
- Submit your sitemap in Google Search Console.
- Audit Disallow rules so you don't block important pages.