io.github.jmshinhwa/robots-txt-ai-crawler-audit
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
Connection details
Sources and provenance
- Official MCP RegistryObserved
Grouping basis: packages. Exact endpoints or package sets can be grouped automatically; other relationships require evidence review. This does not verify publisher affiliation, connection health, or equivalent capabilities.