Online bot traffic will exceed human traffic by 2027, Cloudflare CEO says

According to Matthew Prince, CEO of Cloudflare, bots are taking over the internet. In one interview at the SXSW conference in Austin this week, he said that at the rate artificial intelligence is growing, AI bot traffic will exceed the amount of human traffic online by 2027.

Prince explained that web use of bots has increased alongside the growth of generative AI technology, as bots are able to visit many more sites to get answers to users’ chatbot questions.

“If a human were doing a task (let’s say you were shopping for a digital camera) and you could go to five websites. Your agent or the bot that does that will often go to a thousand times the number of sites a real human would visit,” Prince said. “So it could go to 5,000 sites. And that’s real traffic, and that’s a real burden, that everyone has to deal with and that everyone has to take into account.”

Before the generative AI era, the Internet was only about 20% bot traffic, with the largest being Google’s web crawler, according to Prince, whose infrastructure and security company is used by a fifth of all websites. But apart from some other reputable crawlers, the only other bots were those used by scammers and bad actors.

“With the rise of generative AI and the insatiable need for data, we are seeing an increase where we suspect that by 2027, the amount of bot traffic online will exceed the amount of human traffic online,” Prince said.

The director also noted that this change in the web would require the development of new technologies, such as sandboxes for AI agents that can be started immediately and then shut down when their task is completed. These could come into play when consumers ask AI agents to perform certain tasks on their behalf, such as planning a vacation.

“What we’re trying to think about is how do we build that underlying infrastructure where you can – as easily as you open a new tab in your browser – actually come up with new code, which can then run and serve the agents that are there,” Prince said.

He imagines that there will soon come a time when millions of these agent “sandboxes” will be created every second.

Naturally, the use of the Internet by bots on this scale would require a physical infrastructure in the form of data centers and servers. Prince pointed out that internet traffic increased so quickly during the corona crisis, especially among video streamers such as YouTube, Disney and Netflix, that some parts of the internet almost collapsed under the pressure.

“This [growth] is more gradual, but unlike COVID, where it peaked over the course of two weeks and then kind of plateaued at the new high, we’re seeing internet traffic grow and grow and grow, and we don’t see anything that’s going to slow it down or stop it,” Prince added.

All these overload concerns are great marketing for Cloudflare, a company whose services focus on helping websites stay highly available, load quickly, and stay safe from attacks. Offerings include a content delivery network, a range of security and DDoS protections and an “Always Online” technology which serves cached versions of websites when the main server fails or goes offline. It also provides companies with tools to do that block the AI ​​bot traffic they don’t want.

Still, Cloudflare’s scale gives it the advantage of looking at the continued evolution of the internet and the rapidly emerging challenges facing the generative AI era.

“What people don’t appreciate about AI is that it’s a platform shift,” Prince said, recalling the internet’s previous platform shifts, such as moving from desktop to mobile. “AI is another platform shift… the way you consume information is completely different.”