Best Inference Providers

How do you choose an AI inference provider in 2026?

Independent research, updated 2026-11-09. Built from the pricing pages and docs of the 12 providers on our ranked list. Speed figures are vendor-reported. We did not run hands-on tests.

Answer in brief: Start with the exact model you need and cut any provider that doesn't serve it self-serve. Next, decide whether latency or cost per task matters more. Match the billing model to your traffic: per token for spiky loads, reserved throughput for steady loads, dedicated GPUs for custom weights. Check data-retention, region and certification requirements, and model the total cost including caching, batch discounts, region multipliers and gateway fees. Finally, test your real prompts on two or three finalists. Most providers are OpenAI-compatible, so trials are cheap.

Top pick for most teams: Together AI. It has the broadest open-model catalog on our list, with serverless, Provisioned Throughput, dedicated GPUs and fine-tuning under one account.

The 7-step framework

  1. Model first. Catalogs vary enormously. Cerebras' self-serve tier lists two models, while DeepInfra serves more than 150. Groq marks Llama 3.1 8B and 3.3 70B as Enterprise only. Models also disappear: Nebius removed 11 models from Serverless on 2026-11-09 and did not reroute their requests.
  2. Speed or cost? Vendor speed leaders are Cerebras (~3,000 tokens/sec on gpt-oss-120b), Groq (~500 on gpt-oss-120b, ~1,000 on gpt-oss-20b) and Fireworks' Fast mode (100+ tokens/sec). Price leaders include Baseten (gpt-oss-120b at $0.10 / $0.50) and DeepInfra (Llama 3.1 8B Turbo at $0.02 / $0.04).
  3. Match billing to traffic. For spiky traffic, use per-token serverless or per-second compute (Modal, Replicate). For steady traffic, use reserved capacity (Together Provisioned Throughput, Fireworks Reserved Throughput). For custom models, use dedicated GPUs (Together, Fireworks, Baseten, Nebius).
  4. Data and region. Fireworks and DeepInfra state zero data retention. Nebius dedicated endpoints run in a fixed region you choose, while its public endpoints have no region guarantee. OpenRouter offers EU and US in-region routing on its Business plan.
  5. Total cost, not headline rate. Fireworks Batch is 50% off and Spot is 30% off. Cached input can cost a tenth of the normal rate or less, for example $0.015 on Fireworks gpt-oss-120b. Fireworks charges 1.5x for region-restricted deployments, Modal 1.15–1.75x for region selection and 3x for non-preemptible execution, and OpenRouter adds a 5.5–8% fee.
  6. Plan your exit. Choose OpenAI-compatible APIs. Gateways (OpenRouter, Hugging Face Inference Providers) give you fallback and one key.
  7. Test your own workload. Cerebras itself notes that performance varies by workload, configuration and model. Run your actual prompts before you commit.

Shortlist by need

If you need…ShortlistWhy
One platform for everything open-weightTogether AI, Fireworks AILarge catalogs plus reserved and dedicated capacity
Lowest latencyCerebras, GroqHighest vendor-reported tokens/sec
Lowest token billBaseten, DeepInfragpt-oss-120b $0.10 / $0.50; Llama 3.1 8B $0.02 / $0.04
Compliance and procurementFireworks AI, NebiusZero retention by default, ISO and SOC certifications, Azure Foundry (Fireworks); SOC 2 Type II, HIPAA, ISO 27001 (Nebius)
Custom or fine-tuned modelsBaseten, Together AI, Fireworks AIDedicated GPUs plus training
Your own serving codeModalPer-second GPUs, scale to zero
Image and video generationReplicate, Together AIPer-output pricing on media models
On-premises or sovereign optionsSambaNova, Fireworks Sovereign (preview)Systems for your own data center, or managed inference inside your cloud
One key, many providersOpenRouter, Hugging FaceRouting and fallback across providers

The 12 providers in one line each

Pricing models, decoded

Per token:[redacted] you pay per 1M input, cached-input and output tokens (Together, Fireworks, Groq, DeepInfra, Cerebras, Baseten, SambaNova). Reserved throughput: you pay for capacity per minute, often with an SLA (Together PTUs, Fireworks Reserved Throughput). Dedicated GPU: you pay per GPU-hour, minute or second (Together, Nebius, Baseten, Fireworks). Per second of compute: Modal, and Replicate for most models. Replicate private models also bill setup and idle time. Gateway: provider price plus a fee (OpenRouter) or no markup (Hugging Face).

FAQ

What is the most important factor when choosing an inference provider?

Model availability. Compare only providers that serve your exact model and version self-serve. Price and speed come next.

When should I move from serverless to dedicated GPUs?

When utilization is consistently high, when rate limits block you, or when you need to serve custom or fine-tuned weights.

Should I use a gateway like OpenRouter or Hugging Face?

Use one if you want one key and automatic fallback. OpenRouter charges 5.5–8%, and bring-your-own-key is free up to $25,000 a month of list-price inference. Hugging Face passes provider rates through with no markup.

Are vendor speed claims reliable?

Treat them as best-case figures from the vendor, and benchmark your own prompts on two or three finalists.

Keep exploring

See the ranked list of the best inference providers · Pricing comparison · Methodology

Comparisons: Together AI vs Fireworks AI · Groq vs Cerebras · Fireworks AI vs Groq · Together AI vs DeepInfra · Baseten vs Modal

Reviews: Together AI · Fireworks AI · Groq · DeepInfra · Cerebras · Baseten · SambaNova · Nebius · Modal · Replicate · OpenRouter · Hugging Face

Sources

Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.