Kimi K3 (MoE, VLM) needs 1529.0 GB of VRAM for its weights at AWQ 4-bit — 1539.8 GB once a KV cache is loaded. Below is the full breakdown: which GPUs fit, how many you need, what it serves, and what it costs per million tokens. Every figure comes from tokcalc's open-source capacity model.
| Quantization | Weights | With KV cache | Smallest single GPU |
|---|---|---|---|
| FP16 | 5559.9 GB | 5570.7 GB | — needs multi-GPU |
| FP8 (E4M3) | 2779.9 GB | 2790.8 GB | — needs multi-GPU |
| AWQ 4-bitrecommended | 1529.0 GB | 1539.8 GB | — needs multi-GPU |
| GPU | GPUs | Total VRAM | Decode tok/s | Aggregate tok/s | Cost / 1M tokens |
|---|---|---|---|---|---|
| AMD MI300X 192GB | 8× | 1536 GB | 360.4 | 34598 | $0.1606 |
| AMD MI325X 288GB | 8× | 2304 GB | 408.0 | 39168 | $0.1986 |
| B200 SXM 192GB | 8× | 1536 GB | 544.0 | 52224 | $0.2553 |
| B300 SXM 288GB | 8× | 2304 GB | 544.0 | 52224 | $0.3404 |
| Mac M2 Ultra (192GB unified) | 8× | 1536 GB | 54.40 | 4154 | — |
| Mac M2 Ultra (192GB) | 8× | 1536 GB | 54.40 | 4154 | — |
AMD MI300X 192GB has no NVLink, so 8-way tensor parallelism runs over PCIe. tokcalc applies a flat 0.85× multi-GPU efficiency factor for this, which is realistic for NVLink-equipped datacenter cards but optimistic for PCIe scale-out — treat the throughput and cost figures for this configuration as an upper bound, and measure before committing.
| Context | Concurrent users | KV per request | Fits? |
|---|---|---|---|
| 4K | 21 | 10.82 GB | ✓ fits |
| 8K | 15 | 21.65 GB | ✓ fits |
| 33K | 6 | 86.60 GB | ✓ fits |
| 66K | 3 | 173.19 GB | ✓ fits |
| 131K | 1 | 346.38 GB | ✓ fits |
Not at AWQ 4-bit with any GPU in the current catalog — the weights alone need 1529 GB. You will need multiple GPUs, or a quantisation below AWQ 4-bit.
At AWQ 4-bit with 8,192-token context and 8 concurrent users, the cheapest configuration that fits is 8× AMD MI300X 192GB (1536 GB total VRAM). $0.1606 per 1M output tokens.
On 8× B200 SXM 192GB at AWQ 4-bit, the aggregate throughput is 52224 tokens/sec with continuous batching, at $0.2553 per 1M output tokens.
Weights alone: 5559.9 GB at FP16, 1529.0 GB at AWQ 4-bit. Add KV cache for your context length and concurrency — every concurrent request at 8,192 tokens costs extra VRAM, so budget headroom above the weight figure.
FP8 halves the weight footprint (5560 GB → 2780 GB) and is faster than FP16 on Hopper and Blackwell, where it has native kernels. FP16 remains the reference quality bar; FP8 loses a small amount of accuracy in exchange for roughly doubling the batch size you can serve per GPU.
Kimi K3 (MoE, VLM) is a mixture-of-experts model: 104B parameters are active per token, but all 2780B experts must stay resident in VRAM. Capacity is bounded by the resident set, while decode throughput is bounded by the active set — so it serves fast per token but needs the full footprint.
At 8× AMD MI300X 192GB the marginal cost is $0.1606 per 1M output tokens, before you add idle capacity, ops staff, or egress. That is typically cheaper than frontier API pricing at sustained volume, but API calls win when your load is bursty or too small to keep a GPU busy. The Build vs Buy tab on tokcalc prices this against a specific provider for you.
The calculator uses the exact same formulas as this page — change the workload and watch every number update.
| Total parameters | 2779.93B |
| Active parameters per token | 104.00B |
| Architecture | Mixture-of-Experts |
| Attention layout | Hybrid: 69 KDA + 24 MLA |
| Transformer layers | 93 |
| Hidden dimension | 7168 |
| Attention heads | 96 query / 96 KV (GQA) |
| Head dimension | 74 |
| Vocabulary size | 163,840 |
| Max context (catalog) | 1M |
| KV cache per token @ 8K | 27.0 KB |
| Fixed recurrent state | 222.3 MB per request |
| Native checkpoint size | 1.52 TB |
| Native weight format | mxfp4 |
| License | kimi-k3 |
linear-attention layers keep a fixed recurrent state instead of a growing key/value cache; MLA layers store one compressed latent plus a small decoupled RoPE key, not a full key/value pair.