agentsready

About

Why this exists

AI agents (Claude, GPT, Cursor, Perplexity, and dozens more) increasingly read websites not with human eyes but through special access standards: robots.txt with AI-bot rules, llms.txt with instructions, MCP for functional APIs, OAuth for authorization.

These standards emerged in the last 1–2 years, and most websites haven't adapted. As a result, they're either invisible to agents, return HTML instead of machine-readable formats, or block valuable traffic through overly restrictive rules.

Our scanner is a public audit that shows where your site stands on this scale and what specifically to change.

What does 'agent-ready' mean and what about GEO?

An agent-ready site is one where an AI agent can:

  • Discover content (via robots.txt, sitemap, llms.txt, ai-agent.json, link headers)
  • Read content (via Markdown for Agents, Schema.org)
  • Understand access policy (AI bot rules, Content Signals)
  • Interact with APIs (OAuth, MCP, Agent Skills, A2A, WebMCP)
  • Complete transactions (x402, MPP, UCP, ACP — international only for now)

The first three blocks are the technical foundation of GEO (Generative Engine Optimization). Without it, your site is invisible in ChatGPT, Perplexity, and Gemini answers.

The remaining blocks (API, transactions) are about agent-driven workflows: the agent doesn't just read your site, it uses its features on behalf of users. For these, we offer turnkey implementation, not just an audit.

Who we are

An independent team of developers and consultants. We specialize in implementing AI-access standards for websites and APIs. Open to feedback and suggestions for new checks.

Contact: hello@agentsready.io

What we check

23 open standards across 5 categories. Glossary →

Discoverability

  • robots.txt /robots.txt

    Publish /robots.txt with clear rules for search and AI crawlers.

  • Sitemap /sitemap.xml

    Publish a sitemap and reference it from robots.txt.

  • Link headers /

    Include Link response headers (RFC 8288) for agent discovery.

  • llms.txt /llms.txt

    Publish /llms.txt with structured instructions for LLMs.

  • ai-agent.json /.well-known/ai-agent.json

    Publish /.well-known/ai-agent.json — a unified manifest for AI agents.

Content Accessibility

  • Markdown for Agents /

    Support content negotiation: return Markdown when requested with Accept: text/markdown.

  • Schema.org JSON-LD /

    Use Schema.org JSON-LD for structured content semantics.

Bot Access Control

  • AI bot rules /robots.txt

    Explicitly configure rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot).

  • Content Signals /robots.txt

    Declare AI usage policy via Content Signals in robots.txt.

  • Web Bot Auth /.well-known/http-message-signatures-directory

    (Optional) Publish Web Bot Auth directory for crypto-identification of AI bots.

API / Auth / MCP

  • API Catalog /.well-known/api-catalog

    Publish API Catalog (RFC 9727) for agent discovery.

  • OAuth discovery /.well-known/openid-configuration

    Publish OAuth/OIDC discovery endpoint.

  • OAuth Protected Resource /.well-known/oauth-protected-resource

    Publish OAuth Protected Resource Metadata (RFC 9728).

  • auth.md /auth.md

    Publish /auth.md — agent registration instructions (WorkOS open auth.md protocol).

  • MCP Server Card /.well-known/mcp/server-card.json

    Publish MCP Server Card for discovery of your MCP server.

  • A2A Agent Card /.well-known/agent-card.json

    Publish A2A Agent Card for agent-to-agent interaction.

  • Agent Skills /.well-known/agent-skills/index.json

    Publish an Agent Skills index.

  • WebMCP /

    Implement WebMCP API for browser-based AI agents.

  • RateLimit headers /

    (Optional) Publish RateLimit headers for throttling prevention.

Commerce

  • x402

    Declare HTTP 402 support for agentic micropayments.

  • MPP

    Declare Merchant Payment Protocol (MPP) support.

  • UCP

    Declare Universal Commerce Protocol (UCP) support.

  • ACP

    Declare Agent Commerce Protocol (ACP) support.

More

→ Run a scan → Read FAQ