You should not miss this news: OpenAI has announced support for a shared signal in robots.txt that lets website owners express that their content should not be used to train foundational generative AI models.
Instead of maintaining separate rules for every company and crawler, the proposed signal is intended to provide one preference that multiple participating providers can read and respect.
The development is important for publishers, but it is not a complete solution. Participation is voluntary, and OpenAI says implementation will begin over the coming months.