Baseten review: the inference provider for shipping custom models to production
Answer in brief: Baseten ranks #6 among the best AI inference providers. It is the strongest choice for companies that run their own or fine-tuned models on dedicated GPUs, with cheap shared Model APIs alongside. Its gpt-oss-120b price, $0.10 input / $0.50 output per 1M tokens, is the lowest list price we found. It supports OpenAI Chat Completions and Responses, plus Anthropic Messages in beta. Default rate limits are modest until you verify or upgrade your account.
Independent research, checked 2026-11-09 against Baseten's pricing, docs and funding announcement. We did not test the product hands-on.
Two ways to run: Model APIs and dedicated deployments
Model APIs are hosted language and decision models you call with a Baseten API key, billed per million input and output tokens. Dedicated deployments are for when you need another model, dedicated GPUs, or control over hardware and scaling. Baseten also offers training. Fast variants use the same weights as their base models but run on dedicated capacity for higher sustained throughput. They have separate slugs, prices and rate limits, and fall back to base capacity when Fast capacity is unavailable.
Who Baseten fits best
Baseten fits AI product companies shipping their own or fine-tuned models, and teams that want low per-token prices on popular open models. It also suits coding-agent workloads: Baseten recognizes session IDs from Claude Code, Codex and OpenCode for cache-friendly routing.
Why teams pick Baseten
- Price leadership: gpt-oss-120b at $0.10 / $0.50 per 1M tokens.
- API flexibility: OpenAI Chat Completions, OpenAI Responses (including the OpenAI Agents SDK) and Anthropic Messages (beta), all at inference.baseten.co/v1.
- Automatic prompt caching at a discount, plus session affinity that routes related requests to the same replica.
- Data and region controls: zero data retention for synchronous Model API requests, and a compliance policy that restricts which regions can serve your traffic.
- Cost governance: workspace budgets and cost attribution by API key, user, model or service tier.
- Funding: a $300M Series E at a $5B valuation on January 23, 2026, led by IVP and CapitalG, with NVIDIA among the participants.
The honest drawbacks
- Modest default rate limits: 15 RPM and 100K TPM unverified; 120 RPM and 500K TPM verified; 120 RPM and 1M TPM on Pro. Enterprise limits are custom.
- The Model API catalog is smaller than Together AI's or DeepInfra's.
- Shared endpoints can return 529 Overloaded even within your rate limits.
- Serving regions for shared Model APIs can change unless you enable a compliance policy.
Pricing at a glance
| Model API | Input / cached / output per 1M |
|---|---|
| gpt-oss-120b | $0.10 / not listed / $0.50 |
| GLM-5.3 | $1.40 / $0.14 / $4.40 |
| Kimi K3 | $3.00 / $0.30 / $15.00 |
| Dedicated deployments | Billed for compute used |
Best Baseten alternatives
- Together AI: a broader serverless catalog plus Provisioned Throughput.
- Fireworks AI: per-request speed modes and Reserved Throughput with an SLA.
- Modal: write your own serving code on per-second serverless GPUs.
Baseten FAQ
Is Baseten the cheapest place to run gpt-oss-120b?
On 2026-11-09, Baseten's $0.10 / $0.50 per 1M tokens was the lowest list price we found for gpt-oss-120b.
Does Baseten store my prompts?
For synchronous Model API requests, Baseten says it does not store inputs or outputs under its zero data retention policy.
How do I raise Baseten's rate limits?
Verify your account, upgrade to Pro, or submit a rate-limit increase request. Enterprise limits are custom.
Sources
- Baseten pricing
- Baseten gpt-oss-120b model page
- Baseten Kimi K3 model page
- Baseten Model APIs docs
- Baseten pricing and limits docs
- Baseten $300M Series E (January 23, 2026)
← Back to the ranked list of the best AI inference providers
Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.