OpenAI revises ChatGPT crawler documentation with significant policy changes
OpenAI modified technical specifications for ChatGPT-User crawler, removing robots.txt compliance language and clarifying OAI-SearchBot usage no longer includes training data collection.