The agentic web is the re-architecture of internet infrastructure and web protocols around autonomous AI agents rather than human users. As of May 2026, bots already account for roughly 31% of global HTTP traffic over the prior six months, with AI crawlers, search, and assistants making up about a quarter of all bot requests, according to data cited by TechCrunch. Cloud providers including AWS, Cloudflare, and Microsoft Azure are rebuilding their backends for agent traffic bursts, while Google has proposed WebMCP, an open web standard that begins its experimental origin trial in Chrome 149.
This is a structural shift, not a feature update. For three decades the web was built for one consumer: a human with a browser, a mouse, and the patience to read a page, fill a form, and click a button. That assumption is now breaking. AI agents do not read pages the way people do, they do not click the way people do, and they generate traffic patterns that the existing internet was never designed to absorb. In response, the companies that own the plumbing of the internet are quietly re-engineering it from the ground up.
This is our state-of-the-agentic-web explainer for 2026. We lay out the two layers being rebuilt right now: the infrastructure layer (the cloud backends, databases, and compute fabric handling agent traffic) and the protocol layer (the open standards like MCP, A2A, and WebMCP that let agents and websites actually talk to each other). Then we project where this goes next. Every fact below is attributed to its source. Where something is a forecast rather than a measurement, we say so explicitly.
What Is the Agentic Web? A Plain-English Definition
The agentic web is the version of the internet optimized for software agents that browse, reason, and act on a user's behalf, instead of for humans clicking through pages manually. An AI agent is a program built on a large language model that can take goals ("book me a multi-city trip under a budget"), break them into steps, call tools and APIs, and execute the steps with minimal supervision. When millions of these agents operate at once, they create a different kind of demand on the internet than a billion human browsers ever did.
Three properties make agent traffic fundamentally different from human traffic. First, agents are bursty: a single user request can fan out into hundreds of parallel sub-requests in milliseconds, then go quiet. Second, agents are persistent: a long-running agent may hold a session, memory, and context open for hours or days rather than a few minutes. Third, agents are machine-precise: they want structured, callable functions and clean data, not HTML laid out for human eyeballs. Each of those properties stresses a different part of the stack, and each is driving a specific wave of rebuilding.
The scale is no longer hypothetical. According to data cited by TechCrunch in its May 28, 2026 report "The internet is being rebuilt for machines," bots accounted for roughly 31% of all HTTP traffic over the last six months, and AI crawlers, search engines, and assistants made up approximately a quarter of all bot requests. The same report cites a forecast that non-human traffic will exceed human traffic "sometime in the first half of 2027." We want to be precise here: that crossover is a prediction attributed to industry sources, not a measured fact. But even today's measured numbers are enough to force a rebuild.
Why "Best for" Matters: Who the Agentic Web Is Being Built For
Best for: developers building agentic applications, platform and infrastructure engineers, e-commerce and travel companies whose sites agents will increasingly visit, and SEO or growth teams whose 2024 playbooks are quietly expiring. If you publish anything on the web, or you build the tools that serve it, the agentic web changes your assumptions about who your "visitor" actually is.
The Three Forces Driving the Rebuild
If you remember one framework from this explainer, make it this one. Every infrastructure and protocol change described below is a response to the three ways agent traffic differs from human traffic. Burstiness drives the demand for elastic, scale-to-zero compute, because a single agent request can explode into hundreds of parallel calls and then vanish. Persistence drives the demand for long-lived environments and memory, because agents hold sessions and context open far longer than a human ever would. Machine-precision drives the demand for structured tools and clean data, because agents want callable functions, not HTML laid out for human eyes. Hold those three forces in mind, and the seemingly unrelated moves from AWS, Cloudflare, Microsoft, Google, and Anthropic all snap into a single coherent picture.
The Infrastructure Layer: Cloud Backends Are Being Re-Engineered for Agents
The first place the rebuild shows up is the least visible: the cloud infrastructure that serves data and compute. Agents do not behave like web browsers, and the databases, search indices, and serverless platforms built for human-scale request patterns are buckling under the new load profile. The major cloud providers have each shipped a distinct response in the past few months.
AWS: OpenSearch Serverless Rebuilt to Scale to Zero
Amazon Web Services has launched a next-generation version of OpenSearch Serverless explicitly designed for agentic AI, according to TechCrunch. The headline architectural change is the decoupling of compute and storage: the system can scale compute up instantly when an agent burst hits, and scale to zero when there is no load. That elasticity directly targets the burstiness problem. A traditional always-on cluster either over-provisions (paying for idle capacity) or falls over under a spike. A scale-to-zero design absorbs the spike and costs nothing in the quiet periods between agent requests.
Tia White, GM of Amazon OpenSearch Service, framed the motivation directly. "The timing is straightforward," she told TechCrunch. "Agents are moving from experimentation into production, and they create traffic patterns that previous infrastructure simply wasn't designed for." That single sentence is the thesis of the entire infrastructure rebuild: the move from experiment to production is what changes everything, because production agents run continuously and at scale.
Cloudflare: Persistent Environments for Agents
Cloudflare has introduced cloud infrastructure that gives agents persistent environments with instant scalability, per TechCrunch. This addresses the second property of agent traffic: persistence. A human session is ephemeral. An agent may need a stable, addressable environment that survives across many calls, holds state, and remains instantly reachable. Building that at the edge, close to where requests originate, is exactly the kind of problem Cloudflare's network is positioned to solve. The agent stops being a transient visitor and becomes a long-lived resident on the network.
Microsoft Azure: Bursts and Inter-Agent Memory Sharing
Microsoft has rolled out Azure updates aimed at two things: handling AI agent traffic spikes, and enabling memory sharing between agents, according to TechCrunch. The first is the burst problem again. The second is more forward-looking and arguably more interesting: it implies a future where agents are not isolated processes but a coordinated population that can pool context. When one agent learns something, another can use it. That is a meaningful step toward multi-agent systems operating as a unit rather than as a swarm of strangers.
Databricks and Snowflake: Becoming the Memory of AI
The data warehouse giants are repositioning too. Databricks and Snowflake are recasting themselves as AI memory and retrieval systems for enterprise data, per TechCrunch. The strategic logic is sharp. If agents are the new primary consumers of enterprise data, then the platform that stores, indexes, and serves that data to agents becomes the platform agents depend on. "Memory" and "retrieval" are not new database features dressed in AI language; they are a bid to be the substrate that every enterprise agent reads from and writes to. Whoever owns agent memory owns a structural position in the agentic web.
Taken together, these four moves are not coordinated, but they rhyme. Every major infrastructure player is reacting to the same three pressures (burstiness, persistence, machine-precision) from a different angle. You can read the full landscape in TechCrunch's reporting on how the internet is being rebuilt for machines, which is the primary source for the infrastructure data in this section.
The Protocol Layer: How Agents and Websites Will Talk to Each Other
Rebuilding the backend solves the load problem. It does not solve the interaction problem. Even a perfectly elastic cloud does not tell an agent how to use a website. Today, most agents interact with the web by doing a crude impression of a human: they read the rendered HTML, guess where the buttons are, and click. It is slow, brittle, and breaks the moment a site changes its layout. The protocol layer exists to fix this by giving agents structured, machine-friendly ways to act. Three standards matter most right now: MCP, A2A, and WebMCP.
MCP (Model Context Protocol): The Agent-to-Tool Standard
The Model Context Protocol, introduced by Anthropic, is the standard for connecting an AI agent to external tools and data sources. Think of MCP as a universal adapter: instead of every application building a one-off integration for every model, MCP defines a common way for an agent to discover and call tools (a database, a file system, a SaaS API) through a single, consistent interface. It has become the de facto plumbing for agent tool use across the industry, to the point that the SDK infrastructure underneath it is now a strategic asset, as we covered when Anthropic acquired Stainless and the SDK plumbing behind OpenAI, Google, and Cloudflare. MCP answers one question: how does an agent use a tool?
A2A (Agent2Agent): The Agent-to-Agent Standard
Agent2Agent, introduced by Google, is the standard for letting agents communicate with other agents. Where MCP connects an agent to a tool, A2A connects an agent to a peer. In a world where Azure is already enabling memory sharing between agents and where complex tasks get decomposed across specialized agents, you need a common language for agents to delegate, negotiate, and hand off work to each other. A2A is the attempt to standardize that conversation so an agent built by one company can collaborate with an agent built by another. It is the connective tissue for the multi-agent systems that are already shipping, like the autonomous research and self-improving stacks we have tracked in coverage of Anthropic's managed agents and self-improving agent stack.
WebMCP: The Agent-to-Website Standard
WebMCP is the newest and, for the open web, the most consequential of the three. Announced by Google at I/O 2026 on May 19, WebMCP is a proposed open web standard that lets a website expose structured tools (JavaScript functions and annotated HTML form elements) directly to browser-based agents. Instead of an agent scraping a rendered page and guessing how to interact with it, the site itself publishes machine-friendly functions the agent can call to complete a task in seconds.
The example Google uses is travel. A user planning a multi-city vacation today forces an agent to click through dozens of forms across multiple sites. With WebMCP, the agent calls the booking site's exposed functions directly, completing a complex multi-city, multi-passenger itinerary with far fewer steps and far fewer chances to break. The experimental WebMCP origin trial begins in Chrome 149, and Gemini in Chrome will support WebMCP APIs, according to the Chrome for Developers team. You can read Google's own write-up in the Chrome for Developers WebMCP documentation.
Who Is Already Building on WebMCP
WebMCP is not a thought experiment. Nine global consumer brands are already experimenting with it: Expedia, Booking.com, Shopify, Credit Karma, TurboTax, Redfin, Etsy, Instacart, and Target. That list is telling. These are companies whose entire business is transactional, multi-step, and form-heavy: exactly the workflows that are painful for agents to navigate by scraping and clicking. They have the most to gain from exposing clean, callable tools, and the most to lose if agents bounce off their sites because the interaction is too brittle. Early adoption by transactional consumer brands is the clearest signal that WebMCP is solving a real, money-shaped problem.
How the Three Protocols Fit Together
The cleanest way to understand the protocol layer is by what each standard connects. MCP connects an agent to a tool. A2A connects an agent to another agent. WebMCP connects an agent to a website. They are not competitors fighting for the same job; they are complementary layers of the same emerging stack. An agent might use WebMCP to interact with a retailer's site, MCP to query a private database for the user's preferences, and A2A to hand a sub-task off to a specialized payments agent, all inside a single user request.
This layered picture also explains why no single company controls the agentic web. Anthropic seeded MCP, Google seeded A2A and WebMCP, and the infrastructure beneath them spans AWS, Cloudflare, Microsoft, Databricks, and Snowflake. For these standards to actually interoperate at internet scale, they need a neutral home. That is where formal standardization comes in: the W3C's AI Agent Protocol Community Group is working toward formal standards expected in the 2026 to 2027 window. Standardization is slow and unglamorous, but it is what turns a set of vendor proposals into the durable, vendor-neutral fabric the open web runs on. It is the difference between a proprietary feature and a protocol.
Why This Matters for the Open Web
The shift to an agentic web reorganizes who the web is for. For thirty years, the implicit deal was simple: publishers create pages, humans visit them, and attention (and ad revenue, and conversions) flows from that visit. When the visitor becomes an agent that extracts an answer or completes a task without ever rendering your page for a human, that deal changes. We have already watched the early edge of this in search, where AI-driven answers are compressing the traditional click economy, a dynamic we unpacked when Google AI Mode crossed one billion monthly users and effectively retired the 2024 SEO playbook.
The agentic web extends that logic from search to everything. If an agent can call your booking function directly via WebMCP, your carefully designed landing page becomes optional. If an agent retrieves your enterprise data from Snowflake on a user's behalf, the dashboard you built becomes optional. This is not necessarily a loss, but it is a reframe. The unit of value shifts from "a human looked at my page" to "an agent successfully used my capability." Sites that expose clean, reliable, well-documented tools will be the ones agents prefer to transact with. Sites that only offer a human-shaped interface risk being either scraped clumsily or skipped entirely.
The Trust and Control Problem
There is a harder question underneath all of this: who is allowed to act, and on whose behalf? When an agent fills a form, makes a purchase, or shares data through WebMCP, the website needs a way to know it is dealing with a legitimate agent acting for a real user, not a malicious bot or an unauthorized scraper. The same structured access that makes agents efficient also makes the surface area for abuse larger and more powerful. This is precisely why persistent identity, authentication, and authorization for agents are becoming first-class infrastructure concerns rather than afterthoughts. The agentic web cannot scale on capability alone; it needs trust primitives that are still being designed.
Where the OS-Native Agents Fit
The browser is not the only frontier. A parallel push is happening at the operating-system level, where agents are being given control of the whole machine rather than a single tab. When agents live at the OS layer, they can orchestrate across applications, files, and services, which raises the same protocol and trust questions WebMCP raises for the browser, just one level deeper. We saw an early production example of this when Perplexity's OS-native Personal Computer agent opened to Pro and Enterprise users on Mac. And the always-on, autonomous direction of these systems is visible in the competition between Google's Gemini Spark, Claude's Routines, and ChatGPT Pulse for 24/7 autonomous agents.
The infrastructure rebuild and the protocol rebuild both serve this trajectory. Persistent cloud environments (Cloudflare), shared memory (Azure), scale-to-zero search (AWS), agent memory layers (Databricks, Snowflake), and structured tool access (MCP, A2A, WebMCP) are all prerequisites for agents that run continuously, coordinate with each other, and act across the web and the OS without a human babysitting each step.
How the Agentic Web Is Different From the Web We Know
It helps to name the contrast directly. The human web optimizes for presentation: layout, copy, imagery, and a funnel designed to guide a person toward an action. The agentic web optimizes for capability: structured functions, clean data, reliable execution, and verifiable identity. The human web measures success in pageviews, time on site, and clicks. The agentic web measures success in successful tool calls and completed tasks. The human web tolerates ambiguity because humans fill in gaps. The agentic web demands precision because machines do not improvise gracefully.
None of this means the human web disappears. People will still read, watch, shop, and browse. But the two webs will increasingly run side by side on the same infrastructure, and the smart move for builders is to serve both: a human-facing experience and a machine-facing set of capabilities, exposed through the emerging protocols. The companies experimenting with WebMCP today are not abandoning their human customers. They are adding an agent-shaped front door alongside the human one.
Is the Agentic Web the Same as Web3?
It is worth clearing up a common confusion: the agentic web is not Web3. Web3 was a bet about ownership, built around blockchains, tokens, and decentralized finance. The agentic web is a bet about action, built around AI agents, structured protocols, and elastic cloud infrastructure. Where Web3 asked "who owns the data and the network," the agentic web asks "who can act on it, and how reliably." The two share almost no technical DNA. WebMCP, MCP, and A2A are not on a blockchain; they are plain web standards and SDKs layered on the existing internet, which is exactly why the largest incumbent platforms (Google, Anthropic, AWS, Microsoft, Cloudflare) are driving them rather than a wave of startups. The agentic web is less a new internet and more a new layer added to the one we already have.
That distinction matters for how seriously to take it. Web3 struggled because it required users to migrate to new infrastructure and new mental models. The agentic web requires neither: the agent does the work on top of the sites and services that already exist, and the protocols are designed to be adopted incrementally, one site and one tool at a time. Expedia adding a WebMCP endpoint does not break its normal site. That low switching cost is why the rebuild can happen quietly and fast, without a single dramatic flag-day cutover.
What to Watch Next
Three signals will tell you how fast the agentic web is arriving. First, watch the Chrome 149 WebMCP origin trial: how many sites enroll beyond the initial nine, and whether the early adopters report real wins or quietly back off. Second, watch the W3C AI Agent Protocol Community Group: formal standards expected in the 2026 to 2027 window will determine whether these protocols interoperate cleanly or fragment into vendor silos. Third, watch the traffic numbers: if non-human traffic actually approaches the forecast crossover point in the first half of 2027, the economic pressure to serve agents well becomes impossible to ignore.
We will keep tracking each of these. The agentic web is not a single launch with a date on it; it is a slow, structural migration happening across infrastructure and protocols at the same time. The most important thing to understand in 2026 is that the rebuild is already underway, it is being funded and shipped by the largest companies on the internet, and the assumptions baked into the human web (that your visitor is a person, that they will look at your page, that presentation is the product) are quietly becoming optional.
The Bottom Line
The internet is being re-architected for machines in two layers at once. The infrastructure layer (AWS OpenSearch Serverless scaling to zero, Cloudflare's persistent agent environments, Azure's burst handling and inter-agent memory, Databricks and Snowflake becoming AI memory systems) is being rebuilt to survive agent traffic that already makes up roughly 31% of HTTP requests per TechCrunch. The protocol layer (Anthropic's MCP for agent-to-tool, Google's A2A for agent-to-agent, and Google's WebMCP for agent-to-website, trialing in Chrome 149) is being built to let agents act with precision instead of scraping and guessing. The W3C is working to make these standards neutral and durable by 2026 to 2027.
For builders, the takeaway is concrete: assume some of your future visitors are agents, and design capabilities, not just pages, for them. For the open web, the takeaway is bigger and still unresolved: the deal that funded the web for thirty years is being renegotiated in real time, and the terms are being set right now by the companies rebuilding the plumbing. The crossover where machines outnumber humans on the internet is, for now, a forecast. Everything being built to prepare for it is already real.
Frequently Asked Questions
What is the agentic web?
The agentic web is the re-architecture of internet infrastructure and web protocols around autonomous AI agents instead of human users. It includes cloud backends rebuilt for agent traffic (AWS, Cloudflare, Microsoft Azure) and open protocols like MCP, A2A, and WebMCP that let agents call structured tools instead of scraping pages. As of May 2026, bots already make up roughly 31% of global HTTP traffic over the prior six months, according to data cited by TechCrunch.
What is WebMCP and when does it launch?
WebMCP is a proposed open web standard from Google, announced at I/O 2026 on May 19, that lets websites expose structured tools (JavaScript functions and annotated HTML form elements) directly to browser-based agents. The experimental WebMCP origin trial begins in Chrome 149, and Gemini in Chrome will support WebMCP APIs, according to the Chrome for Developers team.
What is the difference between MCP, A2A, and WebMCP?
The three protocols connect different things. MCP (Model Context Protocol, by Anthropic) connects an agent to a tool or data source. A2A (Agent2Agent, by Google) connects an agent to another agent. WebMCP (by Google) connects an agent to a website. They are complementary layers of the same stack, not competitors: an agent can use all three inside a single user request.
How much of internet traffic comes from bots and AI agents?
According to data cited by TechCrunch in its May 28, 2026 report, bots accounted for roughly 31% of all HTTP traffic over the last six months, and AI crawlers, search engines, and assistants made up approximately a quarter of all bot requests. TechCrunch also cites a forecast that non-human traffic will exceed human traffic sometime in the first half of 2027. That crossover is a prediction, not a measured fact.
Which companies are already using WebMCP?
Nine global consumer brands are already experimenting with WebMCP: Expedia, Booking.com, Shopify, Credit Karma, TurboTax, Redfin, Etsy, Instacart, and Target. These are transactional, form-heavy businesses (travel, e-commerce, finance) whose workflows are hardest for agents to navigate by scraping, which is why they have the most to gain from exposing clean, callable tools.
What is AWS doing for agentic AI infrastructure?
AWS launched a next-generation version of OpenSearch Serverless designed for agentic AI, according to TechCrunch. Its key architectural change is decoupling compute and storage so the system can scale compute up instantly during agent traffic bursts and scale to zero when idle. Tia White, GM of Amazon OpenSearch Service, said agents "are moving from experimentation into production, and they create traffic patterns that previous infrastructure simply wasn't designed for."
What is Cloudflare doing for AI agents?
Cloudflare has introduced cloud infrastructure that gives agents persistent environments with instant scalability, according to TechCrunch. This addresses the persistence problem: unlike ephemeral human sessions, a long-running agent may need a stable, addressable environment that survives across many calls and holds state, ideally close to where requests originate at the network edge.
What is Microsoft Azure doing for agent traffic?
Microsoft rolled out Azure updates aimed at handling AI agent traffic spikes and enabling memory sharing between agents, according to TechCrunch. The memory-sharing capability is notable because it points toward multi-agent systems that operate as a coordinated population, pooling context so what one agent learns another can use, rather than running as isolated processes.
How are Databricks and Snowflake involved in the agentic web?
Databricks and Snowflake are repositioning themselves as AI memory and retrieval systems for enterprise data, according to TechCrunch. The strategic logic is that if agents become the primary consumers of enterprise data, the platform that stores, indexes, and serves that data to agents becomes structurally essential. Owning agent memory means owning a durable position in the agentic web.
Who created MCP and A2A?
The Model Context Protocol (MCP) was introduced by Anthropic and has become the de facto standard for connecting AI agents to external tools and data sources. Agent2Agent (A2A) was introduced by Google as a standard for agents to communicate, delegate, and hand off work to other agents. Both are part of the broader protocol stack the W3C AI Agent Protocol Community Group is working to formalize.
Will AI agents replace traditional websites?
Not entirely, but they change the role of websites. As agents call structured tools directly through standards like WebMCP, the carefully designed landing page becomes optional for agent-driven tasks while remaining essential for human visitors. The smart approach for builders is to serve both: a human-facing experience and a machine-facing set of capabilities exposed through the emerging protocols. The unit of value shifts from "a human viewed my page" to "an agent successfully used my capability."
When will agentic web protocols become official standards?
The W3C AI Agent Protocol Community Group is working toward formal standards expected in the 2026 to 2027 window. Standardization is what turns vendor proposals like MCP, A2A, and WebMCP into a durable, vendor-neutral fabric for the open web, ensuring that an agent built by one company can interoperate with tools and agents built by another rather than fragmenting into proprietary silos.
