What Happened
Cloudflare has announced plans to block crawlers that collect data for training artificial intelligence starting September 15, 2026. This will apply to all pages featuring advertisements. If a website forbids data collection for AI training, not only bots like Googlebot but also other search crawlers, including Applebot and BingBot, will be completely blocked, even in areas where their use is permitted.
Why This Matters
This move could significantly impact Google and other major search engines. Separating bot functions into those meant for search and those for AI data collection may become a prerequisite for continued cooperation with Cloudflare. Otherwise, tech giants may face serious restrictions on accessing the information needed to enhance their algorithms and services.
Context
The situation unfolds against a backdrop of increasing concerns about data privacy and the use of information for AI training. Cloudflare, as a leading provider of protection and optimization for internet traffic, is attempting to establish stricter rules for data handling. This could be a response to rising demands from society and regulators for transparency in data usage.
What This Means
If Google and other companies cannot adapt to the new requirements, it could lead to significant changes in their business models. They may need to create separate architectures for their bots, which would require additional resources and time. Ultimately, this could also affect users who rely on high-quality search results and access to information, potentially facing a decline in service quality due to these changes.
