Baseten review: model APIs, dedicated deployments, and pricing

The two Basetens

Almost every confused Baseten evaluation comes from conflating these. They share a console and nothing else.

If you want GLM 5.2 behind a URL, use Model APIs and you are done in five minutes at the same price everyone else charges. If you want your own fine-tune, a diffusion model, an embedding model, or a whisper variant with custom pre-processing, use dedicated deployments and accept that you now own a deployment.

Model APIs: what is served and what it costs

Thirteen models, deliberately curated rather than exhaustive, with context windows from 128k up to 1,048k tokens and maximum outputs between 32k and 384k. Every model supports tool calling, structured outputs, and JSON mode; vision is available on the GLM 5.2, Inkling, and Kimi K2.6, K2.7, and K3 variants; only Inkling handles audio. Reasoning is on by default for DeepSeek and Kimi K3 and opt-in elsewhere.

The compatibility surface is the practical selling point: endpoints speak both the OpenAI Chat Completions shape and the Anthropic Messages shape, so an existing SDK or coding agent points at Baseten with a base-URL change. See what OpenAI-compatible actually means for the parts of the shape that providers routinely get wrong.

Dedicated deployments and Truss

Truss is Baseten's open-source CLI for packaging a model. You start with configuration only, add custom Python when you need it, and drop to a custom Docker container at the far end. The development loop is the part users actually praise.

# push a development deployment and watch it
truss push --watch

# sync local changes to the running dev deployment in seconds
truss watch

# promote to production, with full autoscaling
truss push

Development deployments are limited to one replica and run slightly slower than published ones; production deployments get full autoscaling. That distinction matters when you benchmark: measuring a dev deployment and extrapolating is how teams talk themselves into the wrong instance size.

Per-minute GPU pricing

Baseten is the only provider in this comparison that quotes GPUs per minute rather than per hour or per second. The hourly column below is our conversion, not a Baseten figure.

Training runs on the same GPU and CPU rates as deployments, so there is no separate training rate card to reason about. Plans are Basic at $0 pay-as-you-go, Pro with volume discounts, and Enterprise at custom pricing; new accounts get complimentary credits, though Baseten does not publish the amount.

Model API or dedicated: the break-even

Because Baseten sells both surfaces, the obvious question is when a dedicated deployment beats the Model API for a model that appears on both. The arithmetic is unforgiving.

Dedicated H100 80GB: $0.10833/min x 60 x 730 hr = $4,744.85/month
Model API, GLM-5.2 input:                        $1.40 per million

Break-even (input only) = 4,744.85 / 1.40
                        = ~3.4 BILLION input tokens/month
                        = ~1,290 input tokens/second, sustained

Even a dedicated box running only 2 hours a day
  60 hr x $6.50 = $390/month
  ...which buys 278M input tokens on the Model API instead.

The conclusion holds across every provider in this category and is worth stating plainly: a dedicated deployment of a model the API already serves is almost never a cost decision. Choose it when you need isolation, a private fine-tune, a model outside the 13, non-LLM inference, data residency, or a latency floor you can guarantee. Those are all good reasons. Saving money is not one of them.

The latency story

Baseten's most defensible claim is independently measured rather than self-published. On the Artificial Analysis provider leaderboard in August 2026 it held the lowest time-to-first-token figures on the board, at 0.49 seconds on Inkling Small and 0.75 seconds on DeepSeek V4 Flash.

That is a different axis from raw throughput, where Groq's 457 tokens per second and Nebius's 453 lead. Low TTFT matters most when a workload issues many short calls rather than a few long ones, which is exactly the shape of an agentic coding loop: twenty short model calls separated by tool execution, where every extra 300ms of first-token latency is multiplied twenty times.

Where Baseten is the wrong choice

• You want the broadest catalog. Thirteen models is a curation decision, not an oversight, but if the model you need is not among them you are into dedicated deployment territory.
• You want cheap GPUs. $6.50 per effective H100 hour is roughly triple DeepInfra and 63 percent above Together clusters.
• You want frontier models. No Claude Opus 5, no gpt-5.6-sol. Open weights only.
• You want to avoid operating anything. The Model API is genuinely zero-ops. A dedicated deployment is not, and the Truss ergonomics can make it feel lighter than it is.

Questions people ask

Keep reading

Continue through Inference providers with Modal review: serverless GPUs, and when a team needs them.