CodeMixture-of-Experts236B params21B active131K max context

DeepSeek Coder V2 236B (MoE) VRAM Requirements: How Many GPUs & What It Costs

DeepSeek Coder V2 236B (MoE) needs 129.8 GB of VRAM for its weights at AWQ 4-bit — 130.6 GB once a KV cache is loaded. Below is the full breakdown: which GPUs fit, how many you need, what it serves, and what it costs per million tokens. Every figure comes from tokcalc's open-source capacity model.

Weights (AWQ 4-bit)
129.8 GB
130.6 GB with KV cache
Fits on 1 GPU?
Yes
B200 SXM 192GB or larger
Cheapest config
$0.0192
8× RTX 5090 32GB / 1M tokens
Fastest config
161646
tok/s on 4× Cerebras CS-3 (WSE-3)
VRAM requirement by quantization
Weight footprint = params × bytes per param. The “with KV cache” column adds one request at 4K context.
QuantizationWeightsWith KV cacheSmallest single GPU
FP16472.0 GB472.8 GB— needs multi-GPU
FP8 (E4M3)236.0 GB236.8 GBB300 SXM 288GB (288 GB)
AWQ 4-bitrecommended129.8 GB130.6 GBH200 SXM5 141GB (141 GB)
Which GPU should you use?
Reference workload: 8K context, 8 concurrent users, continuous batching, AWQ 4-bit. Ranked by fitting first, then by tokens per dollar-hour.
GPUGPUsTotal VRAMDecode tok/sAggregate tok/sCost / 1M tokens
RTX 5090 32GB8×256 GB603.557934$0.0192
RTX 3090 24GB8×192 GB315.227124$0.0205
RTX 4090 24GB8×192 GB339.532588$0.0273
V100 SXM2 32GB8×256 GB303.129096$0.0382
A10G 24GB8×192 GB202.119397$0.0412
TPU v6e Trillium (32GB)8×256 GB275.826478$0.0671
A100 40GB SXM44×160 GB261.012528$0.0798
L4 24GB8×192 GB101.09699$0.0916
L40S 48GB4×192 GB145.56983$0.1114
RTX 6000 Ada 48GB4×192 GB161.67759$0.1146
A100 80GB SXM42×160 GB168.44041$0.1650
Intel Gaudi 3 128GB2×256 GB277.86668$0.1666

RTX 5090 32GB has no NVLink, so 8-way tensor parallelism runs over PCIe. tokcalc applies a flat 0.85× multi-GPU efficiency factor for this, which is realistic for NVLink-equipped datacenter cards but optimistic for PCIe scale-out — treat the throughput and cost figures for this configuration as an upper bound, and measure before committing.

How concurrency changes with context length
On 8× RTX 5090 32GB at AWQ 4-bit. KV cache scales linearly with context, so long contexts cut the number of users you can serve from the same GPUs.
ContextConcurrent usersKV per requestFits?
4K160807.5 MB✓ fits
8K801.58 GB✓ fits
33K206.31 GB✓ fits
66K1012.62 GB✓ fits
131K525.23 GB✓ fits
Frequently asked questions

Can DeepSeek Coder V2 236B (MoE) run on a single GPU?

Yes. At AWQ 4-bit the model needs 130.6 GB of VRAM, so it fits on a single B200 SXM 192GB or anything with more VRAM.

How many GPUs does DeepSeek Coder V2 236B (MoE) need?

At AWQ 4-bit with 8,192-token context and 8 concurrent users, the cheapest configuration that fits is 8× RTX 5090 32GB (256 GB total VRAM). $0.0192 per 1M output tokens.

How fast can DeepSeek Coder V2 236B (MoE) be served?

On 4× Cerebras CS-3 (WSE-3) at AWQ 4-bit, the aggregate throughput is 161646 tokens/sec with continuous batching. That hardware has no published rental price, so its cost per million tokens is not comparable.

How much VRAM does DeepSeek Coder V2 236B (MoE) need?

Weights alone: 472.0 GB at FP16, 129.8 GB at AWQ 4-bit. Add KV cache for your context length and concurrency — every concurrent request at 8,192 tokens costs extra VRAM, so budget headroom above the weight figure.

Should I run DeepSeek Coder V2 236B (MoE) in FP8 or FP16?

FP8 halves the weight footprint (472 GB → 236 GB) and is faster than FP16 on Hopper and Blackwell, where it has native kernels. FP16 remains the reference quality bar; FP8 loses a small amount of accuracy in exchange for roughly doubling the batch size you can serve per GPU.

Why does DeepSeek Coder V2 236B (MoE) need more VRAM than its active parameter count suggests?

DeepSeek Coder V2 236B (MoE) is a mixture-of-experts model: 21B parameters are active per token, but all 236B experts must stay resident in VRAM. Capacity is bounded by the resident set, while decode throughput is bounded by the active set — so it serves fast per token but needs the full footprint.

Is it cheaper to self-host DeepSeek Coder V2 236B (MoE) than to call an API?

At 8× RTX 5090 32GB the marginal cost is $0.0192 per 1M output tokens, before you add idle capacity, ops staff, or egress. That is typically cheaper than frontier API pricing at sustained volume, but API calls win when your load is bursty or too small to keep a GPU busy. The Build vs Buy tab on tokcalc prices this against a specific provider for you.

Try DeepSeek Coder V2 236B (MoE) with your own context and concurrency

The calculator uses the exact same formulas as this page — change the workload and watch every number update.

Open in calculator

DeepSeek Coder V2 236B (MoE) on specific GPUs

DeepSeek Coder V2 236B (MoE) on L40S 48GBDeepSeek Coder V2 236B (MoE) on A100 80GB SXM4DeepSeek Coder V2 236B (MoE) on H100 SXM5 80GBDeepSeek Coder V2 236B (MoE) on AMD MI300X 192GBDeepSeek Coder V2 236B (MoE) on H200 SXM5 141GBDeepSeek Coder V2 236B (MoE) on AMD MI325X 288GBDeepSeek Coder V2 236B (MoE) on B200 SXM 192GBDeepSeek Coder V2 236B (MoE) on B300 SXM 288GB
Model architecture
The numbers behind the KV-cache and throughput math.
Total parameters236.00B
Active parameters per token21.00B
ArchitectureMixture-of-Experts
Attention layoutFull attention (GQA)
Transformer layers47
Hidden dimension6144
Attention heads64 query / 8 KV (GQA)
Head dimension128
Vocabulary size102,400
Max context (catalog)131K
KV cache per token @ 8K188.0 KB

Every layer caches keys and values for the whole context, so the KV cache grows linearly with conversation length.