DeepSeek V3 671B (MoE) needs 369.1 GB of VRAM for its weights at AWQ 4-bit — 385.4 GB once a KV cache is loaded. Below is the full breakdown: which GPUs fit, how many you need, what it serves, and what it costs per million tokens. Every figure comes from tokcalc's open-source capacity model.
| Quantization | Weights | With KV cache | Smallest single GPU |
|---|---|---|---|
| FP16 | 1342.0 GB | 1358.4 GB | — needs multi-GPU |
| FP8 (E4M3) | 671.0 GB | 687.4 GB | — needs multi-GPU |
| AWQ 4-bitrecommended | 369.1 GB | 385.4 GB | — needs multi-GPU |
| GPU | GPUs | Total VRAM | Decode tok/s | Aggregate tok/s | Cost / 1M tokens |
|---|---|---|---|---|---|
| A100 80GB SXM4 | 8× | 640 GB | 382.3 | 36698 | $0.0727 |
| Intel Gaudi 3 128GB | 8× | 1024 GB | 630.7 | 60552 | $0.0734 |
| H100 SXM5 80GB | 8× | 640 GB | 640.3 | 61469 | $0.0904 |
| H200 SXM5 141GB | 8× | 1128 GB | 917.4 | 88075 | $0.1009 |
| RTX PRO 6000 Blackwell 96GB | 8× | 768 GB | 342.5 | 32881 | $0.1014 |
| AMD MI300X 192GB | 4× | 768 GB | 506.5 | 24312 | $0.1143 |
| H100 PCIe 80GB | 8× | 640 GB | 382.3 | 36698 | $0.1211 |
| AMD MI325X 288GB | 4× | 1152 GB | 573.4 | 27523 | $0.1413 |
| B200 SXM 192GB | 4× | 768 GB | 764.5 | 36698 | $0.1817 |
| B300 SXM 288GB | 4× | 1152 GB | 764.5 | 36698 | $0.2422 |
| TPU v5p (per-chip, 95GB) | 8× | 760 GB | 156.5 | 15028 | $0.6211 |
| Mac M2 Ultra (192GB unified) | 4× | 768 GB | 76.45 | 2919 | — |
| Context | Concurrent users | KV per request | Fits? |
|---|---|---|---|
| 4K | 16 | 16.37 GB | ✓ fits |
| 8K | 8 | 32.75 GB | ✓ fits |
| 33K | 2 | 131.00 GB | ✓ fits |
| 66K | 1 | 261.99 GB | ✓ fits |
| 131K | 0 | 523.99 GB | ✗ does not fit |
Not at AWQ 4-bit with any GPU in the current catalog — the weights alone need 369 GB. You will need multiple GPUs, or a quantisation below AWQ 4-bit.
At AWQ 4-bit with 8,192-token context and 8 concurrent users, the cheapest configuration that fits is 8× A100 80GB SXM4 (640 GB total VRAM). $0.0727 per 1M output tokens.
On 8× H200 SXM5 141GB at AWQ 4-bit, the aggregate throughput is 88075 tokens/sec with continuous batching, at $0.1009 per 1M output tokens.
Weights alone: 1342.0 GB at FP16, 369.1 GB at AWQ 4-bit. Add KV cache for your context length and concurrency — every concurrent request at 8,192 tokens costs extra VRAM, so budget headroom above the weight figure.
FP8 halves the weight footprint (1342 GB → 671 GB) and is faster than FP16 on Hopper and Blackwell, where it has native kernels. FP16 remains the reference quality bar; FP8 loses a small amount of accuracy in exchange for roughly doubling the batch size you can serve per GPU.
DeepSeek V3 671B (MoE) is a mixture-of-experts model: 37B parameters are active per token, but all 671B experts must stay resident in VRAM. Capacity is bounded by the resident set, while decode throughput is bounded by the active set — so it serves fast per token but needs the full footprint.
At 8× A100 80GB SXM4 the marginal cost is $0.0727 per 1M output tokens, before you add idle capacity, ops staff, or egress. That is typically cheaper than frontier API pricing at sustained volume, but API calls win when your load is bursty or too small to keep a GPU busy. The Build vs Buy tab on tokcalc prices this against a specific provider for you.
The calculator uses the exact same formulas as this page — change the workload and watch every number update.
| Total parameters | 671.00B |
| Active parameters per token | 37.00B |
| Architecture | Mixture-of-Experts |
| Attention layout | Full attention (GQA) |
| Transformer layers | 61 |
| Hidden dimension | 7168 |
| Attention heads | 128 query / 128 KV (GQA) |
| Head dimension | 128 |
| Vocabulary size | 102,400 |
| Max context (catalog) | 66K |
| KV cache per token @ 8K | 3.81 MB |
Every layer caches keys and values for the whole context, so the KV cache grows linearly with conversation length.