Cloudflare shipped developer-experience improvements to AI Search, its managed retrieval layer for agents. The update collapses five primitives — Workers AI, AI Gateway, Vectorize, R2, and Browser Run — into a single indexed namespace, adds a no-sitemap crawl mode, public MCP endpoints, custom-domain routing, and zero-cost embedding and reranking when using Workers AI models.
| Primitive | Pipeline Role | Previously Required |
|---|---|---|
| Workers AI | Embedding generation & reranking (zero-cost with designated models) | Manual API calls + token-cost management |
| AI Gateway | API management & observability layer | Separate configuration & billing surface |
| Vectorize | Vector storage & nearest-neighbour retrieval | Manual vector store writes & query wiring |
| R2 | Raw document & chunk object storage | Separate storage provisioning & lifecycle |
| Browser Run | Headless web crawling & page rendering | Custom crawler jobs + link-following logic |
Before this release, connecting an agent to proprietary data meant wiring those five primitives yourself. Now a single `wrangler ai-search instance create` command points AI Search at a source URL, handles crawling via Browser Run, chunking, embedding, and vector storage in one step. For sites without a sitemap, the `--parse-type discover` flag follows links to find pages. The result is one instance per data surface, bundled into a namespace.
The Cloudflare Dev Stack MCP, available in Cloudflare's AI Playground, demonstrates the deployment. It covers 10 surfaces: Cloudflare Docs, Blog, API Docs, Community, Astro, Vite, Vitest, Hono, Replicate, and OpenNext. Each gets one AI Search instance. A namespace binding in `wrangler.jsonc` lets a Worker make a single `AI_SEARCH.search()` call that fans out to all 10 simultaneously. Results return as `res.chunks` with citation metadata and instance tags, with reranking enabled and up to 10 results per call.
Two deployment paths exist. Option A: bind the namespace to a Worker, register a tool, and fan out with one call—the path Cloudflare used for the Dev Stack MCP. Option B: enable public URLs in the dashboard and immediately get `/search` and `/mcp` endpoints that query all instances in the namespace with no code and no auth by default. For sensitive data, put Cloudflare Access in front of a custom domain and require login; Access tokens work for both humans and agents.
Embedding and reranking cost zero per token when using designated Workers AI models. That eliminates the operational overhead of predicting token counts for search workloads where query volume spikes unpredictably. Storage and compute outside these two operations follow a separate "predictable and scalable" cost model still in preview.
Most teams building retrieval-augmented agents today maintain custom pipelines: crawler jobs, chunking, embedding calls to third-party APIs, vector store writes, and query interfaces. Each component has its own failure mode and billing surface. Cloudflare's managed namespace with a single CLI call and Worker binding cuts surface area significantly—but trades flexibility for lock-in. Teams with non-Cloudflare infrastructure or data sources not on their Cloudflare account hit ownership-verification constraints. Cloudflare says more verification methods are coming.
If your agents need semantic search over documentation, internal wikis, or content you own on Cloudflare, AI Search eliminates the pipeline-stitching work. The free-embedding pricing means you can index aggressively without per-query costs punishing high-volume agents.