Cloudflare launches way to charge AI bots for crawling sites
Cloudflare announced on Tuesday a new tool that can allow publishers to charge robots AI to scratch websites.
This could prove to be a major change in the operation of the Internet so far in the age of AI – the models have, by default, scratched the Internet with abandonment and without authorization, often to the chagrin of the content owners. Cloudflare is an important infrastructure provider, which means that large internet stations will have access to this tool.
The “Pay Per Crawl” functionality was part of the Cloudflare announcement on Tuesday that the company is now the leading internet infrastructure provider to “block AI robots accessing the content without authorization or compensation, by default”. The tool, which is currently in a beta version, allows the owners of sites to charge costs each time a bot AI wishes to “crawl” its website for information.
Mashable lighting speed
The idea behind this Cloudflare push is to put power in the hands of people who make the content that AI uses to train. (Complete disclosure: Ziff Davis, who owns Mashable, was one of the many publishers cited in the Cloudflare press release which supported an approach based on the authorization of IA bots.)
Wrote Cloudflare in his press release:
“For decades, the Internet has worked on a simple exchange: the content of the search index for search engines and direct users to original websites, traffic generation and advertising revenues for websites of all sizes. This cycle rewards creators who produce quality content with money and following content, while helping users to discover relevant information. know that someone looks at its content. “
We will see how it evolves, but the tool could be useful for people who create original content on the Internet.
Subjects
Artificial intelligence
