What are the best AI inference providers in 2026?
Answer in brief: The best AI inference providers in 2026 are Together AI (best overall for open-weight models), Fireworks AI (best for production traffic, with per-request speed and cost modes), Groq and Cerebras (fastest vendor-reported speeds), and DeepInfra and Baseten (some of the lowest per-token list prices). For gpt-oss-120b, Baseten lists $0.10 input / $0.50 output per 1M tokens, and Together, Fireworks and Groq list $0.15 / $0.60. Choose by the exact model you need, your latency target and your traffic pattern.
Top pick: Together AI. One platform takes you from your first serverless call to reserved and dedicated capacity at scale. It has one of the broadest open-model catalogs we found, Provisioned Throughput with a 99% uptime SLA, single-tenant dedicated GPUs and built-in fine-tuning. Read the Together AI review ·
Last updated: 2026-11-09
This is independent editorial research. We built the ranking from each provider's own pricing pages, documentation, model pages and announcements, read on 2026-11-09. We did not benchmark or test these products ourselves, and every speed figure below is the vendor's own. See how we ranked and the side-by-side pricing comparison.
The ranked list
Together AI: best overall open-model inference platform
What it is: A full-stack platform for open-source AI. It covers serverless per-token inference for text, image, video and audio models, Provisioned Throughput (reserved capacity), dedicated single-tenant GPU endpoints, GPU clusters, and LoRA and full fine-tuning.
Best for: Teams that want one vendor from prototype to large-scale production on open-weight models.
Strengths: One of the broadest serverless catalogs we found, including Kimi, GLM, DeepSeek, Qwen, MiniMax, gpt-oss and NVIDIA Nemotron. Cached-input pricing and a Batch API are available. Provisioned Throughput launched on July 8, 2026 with a 99% uptime SLA. Dedicated endpoints support custom models and autoscaling. Together announced an $800M Series C on July 1, 2026.
Drawbacks: The pricing surface is large and changes often. Reserved hardware and several newer GPU types are contact-sales only. Dedicated H100 endpoints ($5.49/GPU-hr on-demand) cost more than Together's own on-demand GPU-cluster rate ($3.99/GPU-hr).
Pricing model: Serverless is priced per 1M tokens; for example, gpt-oss-120B is $0.15 input / $0.60 output and GLM-5.3 is $1.40 / $4.40. Provisioned Throughput is sold in throughput units (PTUs). Dedicated GPUs are billed per GPU-hour (H100 $5.49, B200 $8.99 on-demand).
Fireworks AI: best for production traffic with flexible serving modes
What it is: An inference and training platform. It offers shared serverless capacity, Reserved Throughput, on-demand GPU deployments, and Fireworks Sovereign, which runs Fireworks-managed inference inside your own cloud or data center (private preview).
Best for: Product teams with steady production traffic that want to trade speed against cost per request, and enterprises with compliance requirements.
Strengths: You can pick a serverless mode per request: Standard, Priority, Fast (100+ tokens/sec), Ultra-fast (preview), Batch (50% off) or Spot (30% off on select models). Prompt caching is automatic, cached input costs half price or less, and there is no cache-write fee. Reserved Throughput comes with a throughput SLA and 99.9% uptime terms. Data retention is zero by default. Fireworks lists SOC 2 Type II, ISO 27001, 27701 and 42001, and HIPAA with a BAA. It is a first-party provider in Microsoft Foundry, so usage counts toward an Azure consumption commitment. At its Forge event on November 3, 2026, Fireworks put Ultra-fast into public preview and reported up to 1,500 tokens/sec on DeepSeek V4.1 Flash.
Drawbacks: On-demand GPU rates ($8.00/hr for H100, $13.00/hr for B200) are higher than several rivals. Region-restricted deployments cost 1.5x. US-only serverless costs 50% more on select models. Spot throughput can drop to zero. Ultra-fast and Sovereign are still in preview.
Pricing model: Prepaid credits drawn down per token on serverless; for example, gpt-oss-120B is $0.15 input / $0.015 cached / $0.60 output per 1M tokens. On-demand deployments are billed per GPU-second. Reserved Throughput is billed in dollars per minute.
Groq: best for low latency with simple self-serve pricing
What it is: An inference cloud. It pioneered the LPU chip and now pairs its LPX systems with NVIDIA GPUs. Access is through GroqCloud, which has an OpenAI-compatible API.
Best for: Real-time apps, voice and agent loops where response speed matters, on the models Groq offers self-serve.
Strengths: Groq lists about 500 tokens/sec for gpt-oss-120b and about 1,000 tokens/sec for gpt-oss-20b, at low prices ($0.15/$0.60 and $0.075/$0.30 per 1M tokens). Developer-plan limits for both are 250K tokens and 1K requests per minute. Whisper Large V3 Turbo speech-to-text costs $0.04 per hour. gpt-oss-120b has built-in browser search and code execution. In August 2026 Groq reported 13 data centers and more than six million developers.
Drawbacks: The self-serve priced catalog is small. Llama 3.1 8B and Llama 3.3 70B are Enterprise and contact sales, as are the preview models Muse Spark Open and MiniMax M2.7. Groq says preview models are for evaluation only and may be discontinued at short notice. Its $350M Series A, announced August 17, 2026 at a $3.5B valuation, was subject to customary closing conditions.
Pricing model: Per token for self-serve models, with developer-plan rate limits. Enterprise contracts cover the rest.
DeepInfra: best low-cost, broad-catalog inference
What it is: An inference cloud serving more than 150 open-source models through OpenAI-compatible APIs. It owns and operates its GPUs across eight US data centers and also rents GPU instances.
Best for: Cost-sensitive teams and high-volume workloads that want many models at low list prices.
Strengths: Some of the lowest list prices we saw: DeepSeek-V4-Flash at $0.09 / $0.18 per 1M tokens, Llama 3.1 8B Turbo at $0.02 / $0.04, and Llama 3.3 70B Turbo at $0.10 / $0.32. DeepSeek V4 models are listed with 1024k context. The catalog also covers embeddings, rerankers, speech, image and video models, plus proprietary Gemini and Claude models. There are no long-term contracts. DeepInfra announced a $107M Series B on May 4, 2026 and reported 25x token growth since its Series A. It states zero data retention and SOC 2 and ISO 27001 certification.
Drawbacks: It publicly describes fewer reserved-capacity and SLA products than Together or Fireworks. Temporary discounts (for example, 38% off GLM-5.3) make the effective price move. The announced data-center footprint is US-only. Most non-LLM models are billed by execution time, which is harder to estimate in advance.
Pricing model: Per token for many language models, per execution time for most others, and pay as you go.
Cerebras: fastest vendor-reported speeds, narrow shared catalog
What it is: Inference on Cerebras wafer-scale systems. A self-serve Developer tier is billed pay-as-you-go, and Enterprise and Dedicated Inference add more model families, production capacity and custom weights.
Best for: Latency-critical coding agents and real-time reasoning, on the models Cerebras serves.
Strengths: Cerebras lists about 3,000 tokens/sec for gpt-oss-120b and about 1,850 tokens/sec for Qwen 3.8 27B. It says all Shared Inference models are original, unpruned versions, and that activations, attention and KV cache stay in full precision. Cerebras is also available through AWS Marketplace, OpenRouter, Hugging Face and Vercel.
Drawbacks: Only two models are in the shared catalog. Its gpt-oss-120b price ($0.35 / $0.75) is above GPU clouds that charge $0.15 / $0.60 for the same model. Cerebras labels the Developer tier as for development and evaluation, not production. The one-time $5 promotional credit expires after 30 days. Production capacity, custom weights and other models need an Enterprise or Dedicated agreement.
Pricing model: Per token on the Developer tier (gpt-oss-120b $0.35 / $0.75; Qwen 3.8 27B $0.99 / $1.49 per 1M tokens). Enterprise pricing is quoted.
Baseten: best for production deployment of custom and fine-tuned models
What it is: An inference platform with two parts: Model APIs, which are shared and priced per token, and dedicated deployments for any other model on dedicated GPUs. It also offers Training.
Best for: Companies shipping their own or fine-tuned models that need dedicated GPUs, autoscaling and hardware control, with low-cost Model APIs alongside.
Strengths: The lowest gpt-oss-120b list price we found: $0.10 input / $0.50 output per 1M tokens. It supports OpenAI Chat Completions and Responses, and Anthropic Messages in beta. Fast variants run on dedicated capacity. Synchronous Model API requests are covered by zero data retention. A compliance policy can restrict which regions serve your requests. Baseten also offers session affinity for cache reuse, plus budgets and cost attribution. It raised a $300M Series E at a $5B valuation on January 23, 2026.
Drawbacks: Default rate limits are modest: 15 requests/min and 100K tokens/min unverified, 120 RPM and 500K TPM verified, and 1M TPM on Pro. The Model API catalog is smaller than Together's or DeepInfra's. Shared endpoints can return 529 Overloaded, and their serving regions can change unless you have a compliance policy.
Pricing model: Per 1M tokens on Model APIs (for example, GLM-5.3 $1.40 / $4.40, Kimi K3 $3.00 / $15.00). Dedicated deployments are billed for the compute they use.
SambaNova Cloud: best for enterprise and on-premises inference
What it is: SambaNova builds its own AI systems (SN40 and SN50). It offers the SambaNova Cloud API, as well as systems for on-premises deployments.
Best for: Enterprises evaluating on-premises inference, and developers who want a pay-as-you-go API.
Strengths: Pay-as-you-go prices include gpt-oss-120b at $0.22 / $0.59 and Llama 3.3 70B at $0.60 / $1.20 per 1M tokens. Enterprise adds production rate limits, on-request models, BYOC and custom rate limits. On July 8, 2026 SambaNova announced the first close of a $1B Series F at an $11B post-money valuation. It also said JPMorganChase selected its SN40 and SN50 systems for secure on-premises inference, and by October 2026 it described that deployment as running inside JPMorganChase's firewall.
Drawbacks: The public price list has only four models. The Free plan still requires a payment method and purchased credits, so it is not a no-cost tier. Production rate limits are Enterprise-only.
Pricing model: Per token on Free and Developer. Enterprise uses subscription pricing.
Nebius Token Factory: best path from shared endpoints to region-pinned dedicated capacity
What it is: Nebius's inference and post-training platform. It has OpenAI-compatible public serverless endpoints in Fast and Base configurations, dedicated endpoints, fine-tuned model hosting and sandboxes.
Best for: Teams that prototype on shared endpoints, then move to isolated dedicated endpoints in EU or US regions, or to custom weights.
Strengths: Nebius's public GLM page lists GLM-5.3 at $1.40 / $4.40 and GLM-5.3-Flash at $0.15 / $0.50 per 1M tokens. Cached-input discounts apply automatically on Serverless. Dedicated endpoints come with a 99.9% SLA and optional EU or US deployment. They are billed per GPU-hour at per-minute granularity, with no charge at zero replicas. Nebius offers a zero-retention mode, lists SOC 2 Type II, HIPAA and ISO 27001, and acquired Inferize on October 1, 2026 to strengthen its production inference stack.
Drawbacks: The serverless catalog changes. Nebius's November 2026 deprecation notice removes 11 models from Serverless on November 9, 2026, including GLM-5.1, DeepSeek-V4-Pro and Kimi-K2.6, and requests are not rerouted automatically. Nebius points production workloads that need stable model versions to dedicated endpoints. The full price catalog requires a console login.
Pricing model: Per token on public endpoints (gpt-oss-120b $0.15 / $0.60 per 1M tokens in Nebius's price catalog, visible after login) and per GPU-hour on dedicated endpoints. Enterprise terms are custom.
Modal: best for running your own inference code on serverless GPUs
What it is: A serverless compute platform. You write Python, deploy LLM, image, video or audio inference, and pay per second. It also offers sandboxes and training.
Best for: Engineers who want their own serving stack without managing clusters, especially for spiky traffic.
Strengths: Per-second billing with scale to zero. H100 is $0.001097/sec (about $3.95/hr) and B200 is $0.001736/sec. The Starter plan includes $30/month in free credits, and Team ($250/month) includes $100/month. Modal is SOC 2 compliant, offers HIPAA on Enterprise, and accepts AWS and GCP marketplace committed spend on Enterprise.
Drawbacks: You build and operate the serving stack yourself. Choosing a region costs 1.15–1.75x the base price, and non-preemptible execution costs 3x. GPU concurrency is capped at 10 on Starter and 50 on Team. Containers stay up 60 seconds after the last input by default, and that time is billed.
Pricing model: Per-second compute, plus an optional platform plan.
Replicate: best for image, video and community models
What it is: A model marketplace and API that runs thousands of community and proprietary models. You can deploy your own models with Cog, its open-source packaging tool.
Best for: Generative media and quick prototypes across many models.
Strengths: A very large catalog. Many models are priced per output (for example, FLUX 1.1 Pro at $0.04 per image). Public models bill only for processing time. Enterprise plans add performance SLAs, higher GPU limits and volume discounts.
Drawbacks: Private models run on dedicated hardware and bill for setup and idle time, except fast-booting fine-tunes. Hardware rates are high next to GPU clouds (H100 $5.49/hr). H200 and additional multi-GPU capacity require committed-spend contracts. It is not the cheapest place for LLM tokens: DeepSeek R1 is listed at $3.75 per 1M input tokens and $0.01 per 1K output tokens.
Pricing model: Per second of hardware time, or per input and output, depending on the model.
OpenRouter: best gateway across many providers
What it is: A unified API that routes requests to more than 600 models from more than 80 providers. It is a routing layer, not an inference operator.
Best for: Teams that want one API key, automatic fallback, and easy switching between providers.
Strengths: Inference is billed at each provider's list price. Features include auto-routing and preferred-provider selection, budgets and spend controls, routing based on data policies, zero-data-retention routing, EU and US in-region routing on the Business plan, bring-your-own-key, and a free tier with 25+ free models. Failed attempts are not billed.
Drawbacks: Fees on credit purchases are 5.5% on Standard and 8% on Business. Routing adds a hop. Free models are capped at 50 requests per day until you buy $10 of credits. SSO and contractual SLAs are Enterprise-only. On August 19, 2026 Stripe agreed to acquire OpenRouter; the deal was announced as subject to customary closing conditions.
Pricing model: Pay as you go at provider prices plus a platform fee on credits. Bring-your-own-key is free up to $25,000 of list-price inference per month, then 5%.
Hugging Face Inference Providers: best for Hugging Face Hub users
What it is: A Hugging Face service that routes requests to partner inference providers for more than 200 models, billed through your Hugging Face account.
Best for: Experimentation and consolidated billing for teams already working on the Hugging Face Hub.
Strengths: No markup: Hugging Face passes provider prices through. PRO users get $2/month in credits, and Team and Enterprise organizations get $2 per seat. You can also use your own provider key. Enterprise organizations get a usage API, and organization admins can set spending limits and disable providers.
Drawbacks: The included credits are small. Features, limits and performance still depend on each underlying provider. Included credits do not apply when you use your own provider key.
Pricing model: Pay as you go at the providers' own rates.
Comparison table
| Rank | Provider | Type | Best for | Pricing model | Example published rate | Reserved or dedicated option |
|---|---|---|---|---|---|---|
| 1 | Together AI | Open-model platform | All-round open-model use | Per token; throughput units; GPU-hour | gpt-oss-120B $0.15 / $0.60 per 1M | Provisioned Throughput, dedicated GPUs |
| 2 | Fireworks AI | Open-model platform | Production traffic, enterprise | Per token; reserved $/min; GPU-second | gpt-oss-120B $0.15 / $0.60 per 1M | Reserved Throughput, on-demand GPUs, Sovereign (preview) |
| 3 | Groq | Custom hardware (LPU) cloud | Low latency | Per token; enterprise contracts | gpt-oss-120b $0.15 / $0.60 per 1M | Enterprise |
| 4 | DeepInfra | GPU inference cloud | Low cost, broad catalog | Per token or execution time | DeepSeek-V4-Flash $0.09 / $0.18 per 1M | GPU instances |
| 5 | Cerebras | Wafer-scale hardware cloud | Top vendor-reported speed | Per token; enterprise quote | gpt-oss-120b $0.35 / $0.75 per 1M | Dedicated Inference |
| 6 | Baseten | Model serving platform | Custom models in production | Per token; dedicated compute | gpt-oss-120b $0.10 / $0.50 per 1M | Dedicated deployments |
| 7 | SambaNova Cloud | Custom hardware cloud | Enterprise, on-premises | Per token; enterprise subscription | gpt-oss-120b $0.22 / $0.59 per 1M | Enterprise and on-premises systems |
| 8 | Nebius Token Factory | Full-stack AI cloud | Shared-to-dedicated path | Per token; GPU-hour | GLM-5.3 $1.40 / $4.40 per 1M | Dedicated endpoints |
| 9 | Modal | Serverless GPU compute | Your own serving code | Per second of compute | H100 $0.001097/sec | Committed spend (Enterprise) |
| 10 | Replicate | Model marketplace | Image, video, community models | Per second or per output | FLUX 1.1 Pro $0.04/image | Private models on dedicated hardware |
| 11 | OpenRouter | Gateway | Multi-provider routing | Provider price + platform fee | 5.5% fee (Standard) | Via underlying providers |
| 12 | Hugging Face Inference Providers | Gateway | Hugging Face Hub users | Provider price, no markup | $2/month credits on PRO | Via underlying providers |
Prices are in USD per 1M input / output tokens unless noted, as published on 2026-11-09. See the pricing comparison for more models and GPU rates.
How to choose an inference provider
- Start from the model. Catalogs differ a lot. Cerebras serves two shared models, while Together and DeepInfra list dozens. Confirm the exact model and version first, because providers retire models; Nebius removed 11 from Serverless on 2026-11-09.
- Decide whether speed or cost matters more. If response speed drives the user experience, look at Groq, Cerebras, or Fireworks Fast and Ultra-fast. If cost per task matters most, compare Baseten, DeepInfra, Together and Nebius, and use batch or spot modes where you can.
- Match the pricing model to your traffic. Spiky or early-stage traffic suits serverless per-token pricing. Steady traffic suits reserved capacity (Together Provisioned Throughput, Fireworks Reserved Throughput). Custom or high-volume models suit dedicated GPUs (Baseten, Together, Nebius, Fireworks) or per-second compute (Modal).
- Check data and region requirements. Look for zero-retention defaults (Fireworks; DeepInfra; Baseten for synchronous requests), region controls (Nebius dedicated endpoints in the EU or US, Baseten compliance policies, OpenRouter in-region routing), and certifications such as SOC 2 and HIPAA.
- Model the total cost, not the headline rate. Cached-input discounts, batch discounts (50% on Fireworks), region premiums (1.5x on Fireworks, 1.15–1.75x on Modal) and gateway fees (5.5–8% on OpenRouter) change what you actually pay.
- Keep switching easy. Many providers here document OpenAI-compatible endpoints, so moving between them is mostly a change of base URL and model name. A gateway such as OpenRouter or Hugging Face Inference Providers gives you fallback across providers.
- Test on your own prompts. Vendor speed figures and public benchmarks may not match your traffic. Run your real workload on two or three shortlisted providers before you commit.
Frequently asked questions
What is an AI inference provider?
An inference provider runs trained AI models on its own hardware and sells access through an API. You send a prompt and get the model's output back, usually paying per token or per second of compute, without managing GPUs yourself.
Which inference provider is the cheapest?
It depends on the model. For gpt-oss-120b, Baseten had the lowest list price we found on 2026-11-09 ($0.10 input / $0.50 output per 1M tokens). Together AI, Fireworks AI and Groq were at $0.15 / $0.60. DeepInfra had some of the lowest prices on other models, such as DeepSeek-V4-Flash at $0.09 / $0.18. Batch and spot modes can cut costs further.
Which inference provider is the fastest?
By vendor-reported figures, Cerebras is fastest on the models it serves, listing about 3,000 tokens/sec for gpt-oss-120b. Groq lists about 500 tokens/sec for gpt-oss-120b and about 1,000 for gpt-oss-20b. Fireworks reports up to 1,500 tokens/sec for its Ultra-fast preview on DeepSeek V4.1 Flash. We have not verified these numbers independently.
Are inference providers compatible with the OpenAI API?
Mostly, yes. Groq, DeepInfra, Baseten, Nebius Token Factory, OpenRouter and the Hugging Face router all document OpenAI-compatible endpoints, and Baseten also supports the Responses API and Anthropic Messages (beta). Switching usually means changing the base URL, API key and model name. Check each provider's docs for the exact features supported.
What is the difference between serverless, provisioned throughput and dedicated endpoints?
Serverless endpoints are shared and billed per token, which suits spiky traffic but is subject to rate limits. Provisioned or reserved throughput guarantees a fixed amount of capacity, often with an SLA, and is billed by time. Dedicated endpoints give you single-tenant GPUs billed per GPU-hour or GPU-second, with control over the model, region and scaling.
Do inference providers store or train on my data?
Policies differ, so check each provider. Fireworks states zero data retention by default. DeepInfra states zero data retention. Baseten does not store inputs or outputs for synchronous Model API requests. Nebius offers a zero-retention mode. OpenRouter says it does not train on your data and offers zero-retention routing. Read the provider's data policy before sending sensitive data.
Is OpenRouter an inference provider?
Not exactly. OpenRouter is a gateway that routes your requests to more than 80 underlying providers through one API, and charges a fee on credit purchases (5.5% on its Standard plan). Hugging Face Inference Providers works similarly but charges no markup.
Why does the same model cost different amounts on different providers?
Providers use different hardware, serving configurations (for example, quantization levels), speed tiers, caching discounts and region premiums. A faster tier, such as Baseten's Fast variants or Cerebras hardware, often costs more per token than a standard GPU tier.
Can I run my own fine-tuned model on an inference provider?
Yes, on many of them. Together AI, Fireworks AI, Baseten, Nebius Token Factory and Cerebras (by agreement) support custom or fine-tuned weights. Modal and Replicate (via Cog) let you deploy your own model code.
Related guides: Best enterprise agent platforms · Best AI training data providers
Methodology · Pricing comparison · Get featured · About
Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.