Providing a way to monetize web content in a (semi-)automated fashion for AI agents: that is how Cloudflare interprets its Pay per Crawl initiative.
In an experimental phase since July 2025, the approach leans on HTTP status code 402. This non-standard code is reserved for uses related to micro-payments. It sits at the core of a machine-to-machine (M2M) transactions protocol project launched by Coinbase: x402.
To promote the adoption of its Pay per Crawl, Cloudflare joined this project. Among other things, by offering a deferred payment system… and by helping establish the x402 Foundation.
Blockchains and a stablecoin Under the Hood
The transaction flow proposed by Coinbase can be sketched out in five major steps:
- A client attempts to access an “x402-compatible” resource
- The server responds with 402 and payment instructions
- The client resubmits its request, authorizing the payment
- A third-party server validates the request and ensures the settlement of the transaction
- The main server returns the resource to the client along with a payment receipt
Under the hood, there is blockchain activity. Four networks are currently supported: the Base testnet and mainnet (layer 2 on Ethereum, developed by Coinbase), as well as Avalanche’s layer 1 networks. The protocol presently handles USDC, as well as other ERC-20 tokens that would implement EIP-3009 (a contract type enabling the transfer of fungible assets via EIP-712 signatures).
Third-Party Servers to Validate Transactions
According to the initial draft of the x402 specification (v0.1, dated August 29, 2025), the 402 response sent by the server must include an “error” field explaining why a payment is required. It should also present a payment scheme. The only scheme currently supported is described as “exact.” It enables transferring a specific amount in a single shot.
The server must also specify the blockchain network identifier that will record the payment, the maximum amount demanded (in tokens), the contract address and the recipient wallet address, the URL of the protected resource and a description, as well as the payment window. A JSON schema describing the response format may also be provided, along with the expected MIME type.
The client signals its intent to pay via a X-PAYMENT header. This header notably carries an EIP-712 signature, UNIX timestamps defining the validity window of the authorization, and a random 32-byte nonce to prevent replay attacks.
To indicate whether the payment succeeded, the server returns an HTTP 200 response with a X-PAYMENT-RESPONSE header that includes the blockchain transaction hash.
The interaction with the third-party server – dubbed the “facilitator” – that validates the transactions occurs via a REST API. This enables either delegating operations to trusted third parties or hosting the endpoints in-house. Nominal gas fees (0.0001 USD) are handled at this level.
The Promise of a “Google for Agenticity”
The protocol includes a resource discovery mechanism for x402: the “Bazaar,” which is expected, in time, to resemble “a Google for agent endpoints.” For now, it enables listing all resources registered with a facilitator. The client can limit the number of results (0 to 100; 20 by default). For each resource, the server communicates the URL, the type, the supported protocol version, the payment methods, and the UNIX timestamp of the last update.
We are promised support for several thousand transactions per second, settled in roughly 200 ms. All of this without dependency on the traditional banking system, although future versions of the protocol could extend beyond stablecoins.
The reference implementations of x402 are in Python and TypeScript. Server-side integration is achievable with a single line of code. Express.js, FastAPI/Flask, Hono and Next.js are supported.
The protocol incorporates SIWE authentication (Sign-In with Ethereum), opening the door to user-specific pricing models.
From Pay-Per-Use to Deferred Payment, x402’s Roadmap
A usage-based payment model (tokens, bandwidth, etc.) is on the roadmap. Its inclusion in the spec is expected by the end of 2025. The roadmap also includes:
- Open-sourcing Coinbase’s facilitator code (Go; possibly TypeScript)
- Managing flows between remote URLs (targeting web clients like ChatGPT; currently the system performs better in desktop applications)
- Support for Solana wallets
- Refund and escrow flows
- Handling of EIP-712 flows with tokens other than USDC
- Routing between multiple facilitators
- In the Bazaar, MCP management and the A2A protocol, filtering and ranking systems, and the integration of Ethereum’s ERC-8004 (Trustless Agents, which extends A2A with identity and reputation registries to select agents without prior trust)
The deferred payment framework proposed by Cloudflare should help address potential disputes and aggregate billing when immediate settlement is not required (Pay per Crawl supports this from logs). Typically, it is intended for subscription-style models. It relies on an identifier provided during the handshake.
Complementary reading:
A2A-MCP: a duo of agentic protocols taking hold in production
SaaS vs. agentic AI: four scenarios to avoid obsolescence
Code assistants: a volatile market where prices are hard to read
Adobe pushes its own agentic stack
The Linux Foundation welcomes AGNTCY, Cisco-made agent project
