Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.| The Cloudflare Blog
As AI threatens traditional publishing business models, Cloudflare proposes a solution that could ensure content creators get paid when AI crawlers ingest their work—and potentially revolutionize how we pay for online content.| TidBITS
Pay per crawl is a new feature to allow content creators to charge AI crawlers for access to their content.| The Cloudflare Blog