Cloudflare has unleashed a devious new trap for data-hungry AI bots that ignore website permissions – the “AI Labyrinth.”
The AI Labyrinth attempts to actively sabotage AI bots by serving realistic-looking pages filled with irrelevant information and hidden links that lead deeper into a rabbit hole of AI-generated nonsense.
“When we detect unauthorized crawling, rather than blocking the request, we will link to a series of AI-generated pages that are convincing enough to entice a crawler to traverse them,” Cloudflare revealed.
“But while real looking, this content is not actually the content of the site we’re protecting.”
Here’s how the system works:
- It generates convincing fake pages with scientifically accurate but irrelevant content
- Hidden invisible links within these pages lead to more fake content, creating endless loops
- All trap content remains completely invisible to human visitors
- Bot interactions with these fake pages help improve detection systems
- Content is pre-generated rather than created on demand for better performance
- Crawlers waste their resources rather than wasting Cloudfares’ resources
Such tools are needed because bot internet traffic is growing alarmingly.
Volgens Imperva’s 2024 Threat Research report, bots generated 49.6% of web traffic last year, with malicious bots accounting for a whopping 32% of the total.
AI crawlers bombard Cloudfare’s network with more than 50 billion requests daily – nearly 1% of all web traffic they handle – wasting their resources in the process.
These numbers lend credibility to what many dismissed as the “dead internet theory” - an internet conspiracy claim that most online content and interaction is artificially generated.
Cloudflare is attempting to support its customers in the cat-and-mouse game between website owners and AI companies.
The trap remains completely invisible to human visitors, so they shouldn’t be able to accidentally stumble into the maze.
As Cloudfare explains: “No real human would go four links deep into a maze of AI-generated nonsense. Any visitor that does is very likely to be a bot, so this gives us a brand-new tool to identify and fingerprint bad bots, which we add to our list of known bad actors.”
