Jun 24, 2026 15:00:00

Cloudflare has launched a feature that allows creators to easily control whether or not to allow AI to scrape their content.

AI crawlers , which collect data from the internet for AI to use for learning and answering questions, are useful for AI developers, but some creators are concerned about the unauthorized use of their content by AI. In this context, Cloudflare , which speeds up and optimizes the delivery of content such as websites and apps, has announced a partnership with the newsletter distribution service beehiiv to provide a feature that allows creators to easily control AI crawlers.

Cloudflare, Inc. - Cloudflare and beehiiv Introduce AI Crawl Controls to Help Independent Publishers Navigate the AI Era
https://cloudflare.net/news/news-details/2026/Cloudflare-and-beehiiv-Introduce-AI-Crawl-Controls-to-Help-Independent-Publishers-Navigate-the-AI-Era/default.aspx

Beehiiv adds Cloudflare AI Crawl Control so writers can block or allow bots - SiliconANGLE
https://siliconangle.com/2026/06/23/beehiiv-adds-cloudflare-ai-crawl-control-writers-can-block-allow-bots/

In recent years, AI systems are said to account for a large portion of web traffic, and this proportion is steadily increasing. Some creators want to stop using AI-generated content, but this requires a lot of work for individuals, such as editing robots.txt files and creating firewall rules.

Cloudflare partnered with beehiiv to integrate advanced AI crawler control features directly into beehiiv's newsletter distribution dashboard. This allows creators to easily utilize Cloudflare's advanced AI crawler control system from their dashboard and configure how the AI model uses their content.

Creators can choose between 'maximum discoverability,' which allows AI search engines and AI agents to freely crawl their content and reach a wider audience, or 'content protection,' which blocks AI scraping and prepares them for future monetization and licensing opportunities.

Cloudflare stated, 'As AI models evolve and offer new ways to search and discover, independent creators are seeking flexible ways to understand and manage how their content is accessed. This integration simplifies the process by giving beehiiv users two clear choices to manage their digital footprint.'

The following features will become available to beehiiv users as a result of this partnership.

• Personalized analytics features
Creators can use a dashboard powered by the Cloudflare API to see things like 'which AI crawlers are trying to access their content,' 'which AI crawlers are being blocked,' and 'what kind of referral traffic AI crawlers are bringing to their newsletters.'

・Switch permissions with one click
Depending on the business goals of creators and publishers, you can block or allow specific AI models.

• Forward-looking authority management
Whenever a new AI crawler appears on the web, the controlled targets are automatically updated, allowing creators to always manage the latest permissions without having to update their code.

Cloudflare CEO Matthew Prince said, 'Cloudflare is committed to protecting and supporting content creators, from individual bloggers to the world's largest publishers. Our commitment remains the same as the internet evolves: to ensure creators have the tools they need to thrive. This partnership with beehiiv is a natural next step in that mission. Whether optimizing for greater discoverability or protecting work for future opportunities, it provides newsletter operators with the transparency and control they need to navigate the AI era on their own.'

Cloudflare's AI crawler control feature will be available to all beehiiv users as a beta version. It will be rolled out gradually starting June 23, 2026, and will be available from the platform's standard dashboard settings.