Cloudflare & GoDaddy back AI bot identity standards

Cloudflare and GoDaddy have formed a partnership focused on how AI systems access website content.

The deal centres on new controls for site owners and support for standards aimed at identifying AI agents.

Under the agreement, GoDaddy will add Cloudflare's AI Crawl Control to its website hosting platform. The tool is intended to let website owners decide how automated AI crawlers interact with their content.

Site operators will be able to allow, block or require payment for AI crawler access. The integration is intended to give businesses, creators and smaller website owners more visibility into automated traffic reaching their sites.

The partnership also extends beyond crawler management to include identity standards for AI agents. The two companies will support Agent Name Service, or ANS, and Web Bot Auth, which are designed to help websites verify who is operating an AI agent and whether that agent is legitimate.

GoDaddy introduced ANS as an open standard for naming, verification and discovery of AI agents across systems. It uses the Domain Name System and Public Key Infrastructure technologies to help distinguish identified AI agents from unknown or impersonating ones.

Cloudflare supports ANS as part of a broader set of standards for verifiable agent identity. It has also introduced Web Bot Auth, which uses cryptography to verify bot and agent traffic, and a Signature Agent Card intended to let agent developers share identity and purpose details.

Shift online

The partnership comes as internet companies respond to a broader shift in online behaviour. The rise of AI-generated answers and AI agents acting on behalf of users has created new pressure on publishers, merchants and hosting providers to understand who is accessing content and on what terms.

For many website operators, the issue is no longer limited to traditional search indexing or familiar bot traffic. AI crawlers and autonomous agents can collect information, make requests and potentially carry out commercial actions, raising questions about consent, attribution and payment.

Both companies argued that the current web lacks consistent methods for identifying such systems. Without clearer standards, website owners may struggle to distinguish legitimate agents from malicious bots or impersonators.

Access terms

GoDaddy's decision to integrate AI Crawl Control into its hosting service puts those controls directly into a platform used by small businesses and independent website owners. That could broaden access to tools that have so far been more prominent in discussions among larger technology groups, publishers, and infrastructure providers.

The model is permission-based, with website owners setting their own terms for AI access. The companies also pointed to audit logs and signed identity methods as part of an effort to create clearer records of automated interactions.

The commercial backdrop is becoming more important as AI systems increasingly surface information without sending users to the original website. That has fuelled concern across parts of the online economy that long-standing traffic-based models may weaken if content is used to generate answers elsewhere.

Cloudflare and GoDaddy presented their work as one response to that shift, linking site-level controls with identity verification for AI agents. Their approach suggests the next phase of the web may depend not only on blocking or allowing bots, but on distinguishing between different types of automated actors and assigning terms to each.

Industry standards

The emphasis on open standards is also notable. Efforts such as ANS and Web Bot Auth reflect attempts to avoid fragmented, platform-specific systems for AI identity at a time when developers, hosting companies, and website owners are all trying to define how machine-to-machine interactions should work.

Whether such standards gain broad adoption will depend on support from a wider set of companies beyond the two partners. In practice, any identity framework for AI agents would need backing from infrastructure providers, developers and online services if it is to become a meaningful layer of trust on the web.

Stephanie Cohen, Chief Strategy Officer at Cloudflare, said the shift is already underway. "The Internet is evolving into a high-velocity, AI-driven ecosystem, and that requires a new kind of transparent infrastructure," she said.

"By putting tools like AI Crawl Control and open standards into the hands of website owners, we are providing essential underpinnings for a new Internet business model. We want to ensure that every creator has the tools to verify who is interacting with their site, while giving legitimate AI agents a secure, transparent way to participate in a thriving open web," Cohen added.

Jared Sine, Chief Strategy Officer at GoDaddy, linked the effort to the company's customer base and to the development of a broader framework for AI identity. "By working with Cloudflare on AI Crawl Control and championing the Agent Name Service, an open standard giving every agent a verifiable identity built on DNS, we are providing our customers the transparency they need to thrive in an AI-first world," he said.

"We move at the speed of the Internet, and we're working with the broader industry to ensure the agentic open web does too," Sine said.