Kimi K3 (MoE, VLM) at AWQ 4-bit needs 1529.0 GB for its weights and 1532.5 GB in total at 8K context with 8 concurrent users. That means it needs 8× B200 SXM 192GB. Every figure is computed by tokcalc's open-source capacity model.
| GPUs | Total VRAM | VRAM needed | Decode tok/s | Aggregate tok/s | TTFT | Cost / 1M tok | Fits? |
|---|---|---|---|---|---|---|---|
| 1× B200 | 192 GB | 1532.5 GB | 0.000 | 0.000 | — | — | ✗ |
| 2× B200 | 384 GB | 1532.5 GB | 0.000 | 0.000 | — | — | ✗ |
| 4× B200 | 768 GB | 1532.5 GB | 0.000 | 0.000 | — | — | ✗ |
| 8× B200 | 1536 GB | 1532.5 GB | 544.0 | 52224 | 315.8 ms | $0.2553 | ✓ |
| Context | Concurrent users | KV per request | Fits? |
|---|---|---|---|
| 4K | 21 | 10.82 GB | ✓ fits |
| 8K | 15 | 21.65 GB | ✓ fits |
| 33K | 6 | 86.60 GB | ✓ fits |
| 66K | 3 | 173.19 GB | ✓ fits |
| 131K | 1 | 346.38 GB | ✓ fits |
Yes, but it needs 8× B200 SXM 192GB. At AWQ 4-bit the total footprint is 1532.5 GB (1529.0 GB of weights plus 3.5 GB of KV cache at 8K × 8 concurrent users), against 1536 GB of available VRAM across 8 GPUs.
544.0 tokens/sec per stream and 52224 tok/s aggregate with continuous batching at 8 concurrent users. Time to first token is 315.8 ms, and inter-token latency is 1.8 ms.
$0.2553 per 1M output tokens at 8× B200 SXM 192GB ($6.00/hr). That is the marginal hardware cost — add idle capacity, ops, and egress for the real figure.
1529.0 GB for the weights at AWQ 4-bit, plus 3.5 GB of KV cache for 8 concurrent requests at 8K context — 1532.5 GB total. Every extra concurrent user at long context costs more, so leave headroom.
It works. Compare $0.2553 per 1M tokens against the alternatives below — the cheapest option that fits the same workload is 8× AMD MI300X 192GB at $0.1606, and the fastest is 8× B200 SXM 192GB at 52224 tok/s.
Same formulas, your context length, concurrency, and quantisation.