Cloudflare Workers AI vs the OpenAI API: where each wins

Cloudflare Workers AI is a serverless inference platform that runs models on GPUs inside Cloudflare’s own network, billed per unit of compute rather than per GPU hour. You call it from a Worker through a binding, or over HTTP from anywhere. The decision most teams actually face is narrower than edge versus cloud: for this one feature, does the call go to Workers AI, or to a hosted API like OpenAI’s?

Route by request shape. Short, high-volume, latency-sensitive calls sitting next to code that already runs on Cloudflare belong on Workers AI. Long-context reasoning, multi-step tool calling, and anything where the model ceiling decides whether the feature works at all belong on a hosted frontier API. Most production systems we build use both, and the routing rule gets written once.

What Workers AI is in 2026

The platform launched as a catalog of small open-weight models plus a promise about latency. It has moved since. Cloudflare now lists more than 50 open-source models alongside third-party frontier models served on the same billing rail: Moonshot Kimi K2.6 and K2.7-code, Z.ai GLM 5.2 and 5.3, DeepSeek V4 Pro. Several of those need a paid billing method or prepaid AI Gateway credits before they will answer.

Billing runs on Neurons, Cloudflare’s unit for GPU compute, at $0.011 per 1,000 Neurons. Both the Free and the Paid Workers plan include 10,000 Neurons per day at no charge, and the allocation resets at 00:00 UTC. Since the August 2026 pricing update the docs also publish per-model rates in tokens, so comparing against a hosted API is reading two numbers instead of doing conversion arithmetic.

The price spread inside the catalog is wider than the gap between platforms. IBM Granite 4.0 H Micro costs $0.017 per million input tokens. GLM 5.3 costs $1.400. That is 82x, on one account, behind one API token. Picking the wrong model on Workers AI costs more than picking the wrong provider.

Cloudflare Workers AI vs the OpenAI API: the five axes that decide it

• Model ceiling. Hosted frontier APIs lead. Epoch AI measured the most capable open-weight models trailing frontier closed models by an average of four months since January 2026, an 8-point gap on its Capabilities Index.
• Unit cost on small tasks. Workers AI leads by roughly an order of magnitude. Llama 3.1 8B in the fp8-fast build runs at $0.045 per million input tokens and $0.384 per million output tokens.
• Network position. Workers AI leads when the caller is already a Worker. The request stays inside the network the code runs in. No third-party key to scope, rotate, or leak.
• Rate limits. Hosted APIs lead on headroom. Workers AI text generation defaults to 300 requests per minute per account, and the frontier models on the platform sit at 20 per minute, or 50 with prepaid AI Gateway credits.
• Switching cost. Even, and lower than most teams assume. Workers AI serves OpenAI-compatible endpoints at /v1/chat/completions and /v1/embeddings, so moving a call means changing a base URL and a model string.

When should you run inference on Workers AI?

The call is short, frequent, and boring

Classification, moderation, tagging, language detection, embeddings, first-pass summarization. These are the calls that arrive thousands of times an hour and never need frontier reasoning. On a hosted flagship model they are the line item that quietly eats the margin on an AI feature. On Workers AI a small instruct model handles them for cents.

The test is not the prompt length. It is whether a human reviewing the output would notice the difference between a 3B model and a frontier one. For a spam flag or a topic tag, usually not. For a legal summary, immediately.

The caller is already a Worker

When the inference call comes from code running on Cloudflare, the AI binding keeps the request inside the same network. That removes a public-internet round trip, a second vendor from the data path, and one more credential from the rotation schedule. When a compliance reviewer asks where customer text goes, the answer is shorter by one company.

You want a free floor under the prototype

10,000 Neurons a day is enough to build and demo a feature without a card on file, on the Free plan. That matters less for a funded team and quite a lot for a prototype that has to survive a week of internal testing before anyone approves a budget.

When should you keep the hosted API?

The model ceiling is the feature

If the product promise is an agent that plans across steps, reads a 200-page contract, or writes code that compiles, the open-weight tier is a downgrade the user will feel. Epoch’s four-month lag is an average across benchmarks, and it narrows every quarter, but averages do not help when a specific reasoning task fails at a specific quality level. Test the actual prompt before assuming parity.

Traffic is bursty and frontier-shaped

20 requests per minute is a real ceiling. A coding assistant with 40 concurrent users will hit it. Prepaid AI Gateway credits raise it to 50, which helps, and the limits exist because agentic requests take longer to complete. If your load profile is spiky and frontier-dependent, budget for a hosted provider with a higher quota and keep Workers AI for the cheap tier underneath.

You need the surrounding tooling

Structured outputs, function calling, batch endpoints, fine-tuning pipelines, evaluation tooling, and a support contract with a name on it. The open-weight ecosystem covers most of this, unevenly. Hosted providers cover it with documentation and an account manager. That is worth money to some teams and nothing to others.

How much does the difference actually cost?

Take a support-inbox feature: 500,000 calls a month, about 1,200 input tokens and 200 output tokens each. That is 600 million input tokens and 100 million output tokens. Using the published Workers AI rates:

• Llama 3.1 8B fp8-fast: $27.00 input plus $38.40 output, so about $65 a month.
• gpt-oss-120b: $210.00 input plus $75.00 output, about $285 a month.
• GLM 5.3: $840.00 input plus $440.00 output, about $1,280 a month.

Same platform, same API token, same code path. A 20x range decided by one string in the model field. Average load here is 11.6 requests per minute, comfortable against the 300 per minute text-generation limit and impossible against the 20 per minute frontier limit if the traffic arrives in bursts.

Run this arithmetic before the feature ships, not after the first invoice. We wrote the longer version of that argument in budgeting token cost before it eats your margin.

What we ship

Two tiers, one router. The cheap tier (embeddings, classification, moderation, short summaries) goes to a small Workers AI model. The expensive tier (reasoning, agents, anything user-visible and long-form) goes to a hosted frontier model. Both sit behind AI Gateway for caching, spend caps, and fallback, which means the routing decision is observable instead of theoretical.

The OpenAI-compatible endpoint is what makes this safe. Because both sides speak the same request shape, the tier assignment is config rather than architecture. If a small model turns out to be good enough for a task we assumed needed frontier quality, the change is one line and a redeploy. If it turns out worse, the same.

The limits worth knowing before you commit

Inference does not run everywhere. Cloudflare reported GPUs in over 180 cities after doubling capacity in a year, against a network spanning 330 cities. The request from a Worker still travels to a GPU location. It is a short, private hop rather than a public-internet call to another vendor, and the practical latency win is real, but it is not inference in the same rack as the user.

Neurons add an indirection. The docs now show token prices next to Neuron prices, so the mental model is easier than it was, though the bill still arrives denominated in compute units. Budget in tokens, reconcile in Neurons.

And the frontier models on Workers AI are not Cloudflare models. They are third-party weights served on Cloudflare infrastructure, with the vendor’s quality and the platform’s rate limits. That combination is often the right one. It is worth naming, because "running it on Cloudflare" and "running an open model" stopped meaning the same thing this year. The same reasoning applies one layer down when you choose the edge runtime or the Node runtime for the code doing the calling.

Sources

• Workers AI pricing, Cloudflare Docs
• Workers AI limits, Cloudflare Docs
• OpenAI compatible API endpoints, Cloudflare Docs
• Cloudflare’s bigger, better, faster AI platform, Cloudflare Blog
• Cloudflare’s AI Platform: an inference layer designed for agents, Cloudflare Blog
• Open models lag state-of-the-art closed models by 4 months, Epoch AI