ClaudeBot
Anthropic's web crawler, which collects public web content that may contribute to training its Claude models. Anthropic describes separate agents for search and for fetches made on a user's behalf.
Last updated by Vouch
Example: a robots.txt rule for ClaudeBot controls that crawler only. Rules for other Anthropic agents are separate, so check Anthropic's crawler documentation before assuming one rule covers all three.
See the AI crawler readiness checklist and the free AI crawler checker. Related terms: GPTBot, AI crawler.
More terms
See every term in the glossary, try the free GEO tools, compare AI visibility tools, or read the Vouch methodology and pricing.