Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
Partial audit · 2/5 dimensions · Audit v1-github · today · imported from glama ai
Įrankių informacija netrukus. Šis serveris turi 0 galimų įrankių.
Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
Yes, robotstxt-ai-mcp is completely free to use with no usage limits on the free tier.
robotstxt-ai-mcp is listed under the AI & Machine Learning category in the AgentForge MCP registry.
robotstxt-ai-mcp has a current uptime of 99.9% with an average response time of 0ms.
To connect robotstxt-ai-mcp, click the "Connect Agent" button on this page to get the configuration snippet. Add it to your MCP client (Claude Desktop, Cursor, or any MCP-compatible tool). Your AI agent will then have access to all of robotstxt-ai-mcp's tools via the Model Context Protocol.