Groq review: a low-latency AI inference provider with simple self-serve pricing
Answer in brief: Groq ranks #3 among the best AI inference providers. It is our pick for real-time apps, voice and agent loops that need fast responses at low prices. GroqCloud lists about 500 tokens/sec for gpt-oss-120b at $0.15 input / $0.60 output per 1M tokens, and about 1,000 tokens/sec for gpt-oss-20b at $0.075 / $0.30. The catch is a small self-serve catalog: several popular models are Enterprise-only.
Independent research, checked 2026-11-09 against GroqCloud's model docs and Groq's newsroom. Speed figures are Groq's own; we did not test the product hands-on.
Speed you can feel, at GPU-cloud prices
Groq pioneered the LPU, a chip designed for inference. It now pairs its LPX systems with NVIDIA GPUs and describes itself as a neocloud for fast inference. Developers reach it through GroqCloud, which exposes an OpenAI-compatible API at api.groq.com/openai/v1. Groq's pitch is that speed and affordability no longer have to be traded off. On gpt-oss-120b its price matches Together AI and Fireworks while listing about 500 tokens/sec.
Who Groq fits best
Groq fits voice assistants, chat UIs, coding tools and multi-step agents, where every second of latency compounds, on the models Groq offers self-serve. It also suits speech pipelines: Whisper Large V3 Turbo transcription is $0.04 per hour of audio.
Why teams pick Groq
- Vendor-listed speed: about 500 tokens/sec on gpt-oss-120b, about 1,000 on gpt-oss-20b, and about 450 on the Qwen3.8-27B preview.
- Low prices: gpt-oss-120b $0.15 / $0.60; gpt-oss-20b $0.075 / $0.30; Whisper Large V3 $0.111 per hour; Whisper Large V3 Turbo $0.04 per hour.
- Generous developer limits on gpt-oss models: 250K tokens per minute and 1K requests per minute.
- Built-in tools: gpt-oss-120b on Groq includes browser search and code execution.
- Scale: in August 2026 Groq said it operates 13 data centers across North America, Europe, the Middle East and Asia Pacific, and serves more than six million developers. It also says it is an NVIDIA Cloud Partner.
The honest drawbacks
- Small self-serve catalog. Llama 3.1 8B and Llama 3.3 70B are marked Enterprise and contact sales, as are the preview models Muse Spark Open and MiniMax M2.7.
- Preview models are not for production. Groq says they may be discontinued at short notice.
- Company in transition. On August 17, 2026 Groq announced a $350M Series A at a $3.5B valuation, led by Disruptive with planned NVIDIA participation, on top of $650M raised in June 2026. The release said the round was subject to customary closing conditions.
Pricing at a glance
| Model | Speed (vendor) | Price |
|---|---|---|
| gpt-oss-120b | ~500 tokens/sec | $0.15 / $0.60 per 1M |
| gpt-oss-20b | ~1,000 tokens/sec | $0.075 / $0.30 per 1M |
| Qwen3.8-27B (preview) | ~450 tokens/sec | $0.80 / $4.00 per 1M |
| Whisper Large V3 Turbo | n/a | $0.04 per audio hour |
| Llama 3.3 70B | ~280 tokens/sec | Enterprise, contact sales |
Best Groq alternatives
- Cerebras: higher vendor-listed speed (about 3,000 tokens/sec on gpt-oss-120b) at a higher token price.
- Fireworks AI: Fast and Ultra-fast modes on a much broader catalog.
- SambaNova: custom-silicon inference with an enterprise and on-premises focus.
Groq FAQ
Is Groq OpenAI-compatible?
Yes. GroqCloud exposes OpenAI-compatible endpoints, including a models list at api.groq.com/openai/v1/models.
Which models can I use on Groq without a contract?
Self-serve priced models include gpt-oss-120b, gpt-oss-20b, Whisper Large V3 and V3 Turbo, plus several preview models. Llama 3.1 8B and 3.3 70B are Enterprise.
How fast is Groq?
Groq lists about 500 tokens/sec for gpt-oss-120b and about 1,000 for gpt-oss-20b. These are vendor figures; your speeds depend on your workload.
Sources
- GroqCloud supported models
- Groq homepage
- Groq $350M Series A announcement (August 17, 2026)
← Back to the ranked list of the best AI inference providers
Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.