Methodology v1.0

AI tools

One page per crawler. Pick from the menu โ€” no typed URLs.

Full AI Scanner

All bots + schema + llms.txt ยท one score

GPTBot

Check if GPTBot / ChatGPT-User is blocked in robots.txt. Free, instant.

PerplexityBot

Free PerplexityBot robots.txt access check.

ClaudeBot

Check ClaudeBot / Anthropic crawler access.

Google-Extended

Is Google-Extended blocked (Gemini / AI Overviews related crawling)?

Applebot-Extended

Check Applebot-Extended robots.txt access.

llms.txt Generator

Generate a root llms.txt