Cloudflare will block mixed-use web crawlers (used for search, AI agents, and training) from ad-hosting pages by default starting September 15, 2026, pushing AI companies to pay publishers for content. The policy affects new and free-tier Cloudflare customers and aims to protect publishers’ intellectual property. This could limit AI providers’ access to web content for training and services.
Analyzed by FNXT Experts