Why we're supporting a common signal that lets website operators communicate their choices to multiple AI providers at once
Share Today, we're announcing that OpenAI will honor a common signal (opens in a new window) in robots.txt, the file that lets websites express their preferences about the use of their content for foundational generative AI training. This voluntary effort makes it easier for publishers and website operators to communicate the same preference to multiple participating providers at once
Previously, website operators would identify individual crawlers and maintain separate rules for each and every provider. Having a shared signal across multiple providers reduces that work for operators
