Robots Sitemap Validator
Check local robots.txt and sitemap files for common crawl-blocking mistakes.
Overview
Robots Sitemap Validator guides an agent through a local check of robots.txt and sitemap.xml. Its script reports findings with a rule, severity and location, and it flags referenced sitemap files that are missing from the local set. This is useful in a pre-deployment review when a change to crawl directives needs to be checked against the files that will ship.
Getting started
Provide the local directory, files or standard input to the documented Python scanner. Review its findings against the intended crawl rules.
Things to know
The scanner reads local inputs. It cannot report what the live server serves or whether Google has indexed a URL.
Skill instructions
Sources and documentation
Related MCP servers and skills
- GST Validator — hosted on Cloudflare Workers (HTTP + npm stdio)MCP server
- Fixzi AI ValidatorMCP server
- Web Validator MCP by DigestSEOMCP server
- MCP Protocol ValidatorMCP server
- UI Visual ValidatorAgent skill
- Docs Transloadit RobotsAgent skill