OpenAI announced on 30 October 2026 that it will honour a shared robots.txt signal for websites to express preferences about the use of their content in generative AI training. For publishers and other site operators, the change is intended to replace separate instructions for each participating provider with one common signal.
OpenAI says the signal draws on work by the Internet Engineering Task Force’s AI Preferences working group, which is developing standards for communicating those choices. Until now, operators using robots.txt for this purpose have had to identify individual crawlers and maintain rules for each provider.
OpenAI aims to implement the signal in the coming months. It says the shared option will sit alongside its existing robots.txt crawler controls, which will continue to apply.
The company is joining other working-group participants in announcing the signal and is encouraging other AI providers to honour it.
