The names matter. OpenAI documents OAI-SearchBot for ChatGPT Search and GPTBot for potential model-training use. Anthropic separates Claude-SearchBot, Claude-User, and the training-oriented ClaudeBot. Perplexity separates its search crawler from user-requested fetches. One robots.txt decision does not control every product or use.
For a business that wants search visibility, review the search-specific agents and ordinary search crawlers separately from training crawlers. A publisher may allow search indexing while opting out of model training. Web application firewalls can also block a legitimate crawler even when robots.txt allows it.
Crawler access only makes a page eligible to be retrieved. It does not guarantee that the page will be indexed, cited, or used in a particular answer.
For OpenAI, allowing OAI-SearchBot removes that robots.txt restriction, while disallowing GPTBot signals exclusion from potential training. Host, CDN, and firewall controls can still prevent access, and access does not guarantee inclusion.
Go deeper: How to Get Your Business Cited by AI Search (2026).
This is part of the AI visibility work.