LLM Tokens per Second Calculator
Estimate local language model generation speed from GPU memory bandwidth and model quantization.
How to use
- Select a GPU or enter its published memory bandwidth from the dropdown.
- Enter the model parameter count and quantization.
- Adjust efficiency to reflect real-world utilization; lower values give more conservative estimates.
Formula and example
Estimated tokens/second ≈ memory bandwidth (GB/s) × efficiency ÷ (parameters × bytes per parameter). This bandwidth-bound estimate assumes each generated token reads the model weights once.
Example: RTX 4090 at 1,008 GB/s, 8B model in INT4 (0.5 byte/parameter) and 65% efficiency: 1,008 × 0.65 ÷ (8 × 0.5) ≈ 164 tokens/second.
Static memory bandwidth dataset, reviewed November 18, 2026. Actual speed depends on kernels, KV cache, context, batching, thermals and software.
FAQ
Is this a benchmark?
No. It is a theoretical estimate, especially useful for comparing hardware and quantization choices.
Why does actual output speed differ?
Attention, context length, memory overhead, multi-GPU communication and runtime optimizations affect token generation.
Will the model fit in VRAM?
Check the LLM VRAM Calculator; memory capacity and memory bandwidth are separate constraints.
Related: LLM VRAM Calculator · LLM Cost Calculator.
See also
- Need to know how many tokens a prompt or answer is? Use the token calculator.
- Speed also depends on fitting the model in memory; see the LLM memory calculator.
Related Tools
- Self-Host vs API Cost Calculator (LLM Break-Even)Compare monthly LLM API cost with self-hosting on a rented or owned GPU and find the break-even token volume. All assumptions shown.
- GPU Rental Cost CalculatorEstimate cloud GPU cost per job and per month from hourly rate, GPU count, job and idle hours, storage and egress. Example rates are dated and editable.
- LLM API Cost CalculatorCompare GPT, Claude and Gemini API rates and estimate monthly token costs.
- RAG Chunking and Embedding Cost CalculatorEstimate RAG chunk count, one-off embedding (indexing) cost and monthly query cost from document size, chunk size, overlap, embedding price and retrieval settings. Prices are dated and editable.
- AI Token CounterCount tokens, words and characters in your prompts for GPT, Claude, Gemini and Llama.
- Tokens to Words, Pages and Books ConverterHow many tokens in a word, page or 1,000 words? Convert words, pages and books to tokens and back for six languages.