TextMixture-of-Experts753.33B params19B active1M max context

GLM-5.3 (MoE) VRAM Requirements: How Many GPUs & What It Costs

GLM-5.3 (MoE) needs 414.3 GB of VRAM for its weights at AWQ 4-bit — 430.0 GB once a KV cache is loaded. Below is the full breakdown: which GPUs fit, how many you need, what it serves, and what it costs per million tokens. Every figure comes from tokcalc's open-source capacity model.

Weights (AWQ 4-bit)
414.3 GB
430.0 GB with KV cache
Fits on 1 GPU?
No
Needs 8× at AWQ 4-bit
Cheapest config
$0.0375
8× A100 80GB SXM4 / 1M tokens
Fastest config
118970
tok/s on 8× H100 SXM5 80GB
VRAM requirement by quantization
Weight footprint = params × bytes per param. The “with KV cache” column adds one request at 4K context.
QuantizationWeightsWith KV cacheSmallest single GPU
FP161506.7 GB1522.4 GB— needs multi-GPU
FP8 (E4M3)753.3 GB769.0 GB— needs multi-GPU
AWQ 4-bitrecommended414.3 GB430.0 GB— needs multi-GPU
Which GPU should you use?
Reference workload: 8K context, 8 concurrent users, continuous batching, AWQ 4-bit. Ranked by fitting first, then by tokens per dollar-hour.
GPUGPUsTotal VRAMDecode tok/sAggregate tok/sCost / 1M tokens
A100 80GB SXM48×640 GB739.971027$0.0375
H100 SXM5 80GB8×640 GB1239118970$0.0467
RTX PRO 6000 Blackwell 96GB8×768 GB662.963640$0.0524
AMD MI300X 192GB4×768 GB980.347055$0.0590
H100 PCIe 80GB8×640 GB739.971027$0.0626
Intel Gaudi 3 128GB4×512 GB610.429299$0.0758
B200 SXM 192GB4×768 GB148071027$0.0939
H200 SXM5 141GB4×564 GB887.842616$0.1043
AMD MI325X 288GB2×576 GB554.913318$0.1460
B300 SXM 288GB2×576 GB739.917757$0.2503
TPU v5p (per-chip, 95GB)8×760 GB303.029086$0.3209
Mac M2 Ultra (192GB unified)4×768 GB148.05649—
How concurrency changes with context length
On 8× A100 80GB SXM4 at AWQ 4-bit. KV cache scales linearly with context, so long contexts cut the number of users you can serve from the same GPUs.
ContextConcurrent usersKV per requestFits?
4K61315.70 GB✓ fits
8K30631.41 GB✓ fits
33K76125.63 GB✓ fits
66K38251.26 GB✓ fits
131K19502.51 GB✓ fits
Frequently asked questions

Can GLM-5.3 (MoE) run on a single GPU?

Not at AWQ 4-bit with any GPU in the current catalog — the weights alone need 414 GB. You will need multiple GPUs, or a quantisation below AWQ 4-bit.

How many GPUs does GLM-5.3 (MoE) need?

At AWQ 4-bit with 8,192-token context and 8 concurrent users, the cheapest configuration that fits is 8× A100 80GB SXM4 (640 GB total VRAM). $0.0375 per 1M output tokens.

How fast can GLM-5.3 (MoE) be served?

On 8× H100 SXM5 80GB at AWQ 4-bit, the aggregate throughput is 118970 tokens/sec with continuous batching, at $0.0467 per 1M output tokens.

How much VRAM does GLM-5.3 (MoE) need?

Weights alone: 1506.7 GB at FP16, 414.3 GB at AWQ 4-bit. Add KV cache for your context length and concurrency — every concurrent request at 8,192 tokens costs extra VRAM, so budget headroom above the weight figure.

Should I run GLM-5.3 (MoE) in FP8 or FP16?

FP8 halves the weight footprint (1507 GB → 753 GB) and is faster than FP16 on Hopper and Blackwell, where it has native kernels. FP16 remains the reference quality bar; FP8 loses a small amount of accuracy in exchange for roughly doubling the batch size you can serve per GPU.

Why does GLM-5.3 (MoE) need more VRAM than its active parameter count suggests?

GLM-5.3 (MoE) is a mixture-of-experts model: 19B parameters are active per token, but all 753B experts must stay resident in VRAM. Capacity is bounded by the resident set, while decode throughput is bounded by the active set — so it serves fast per token but needs the full footprint.

Is it cheaper to self-host GLM-5.3 (MoE) than to call an API?

At 8× A100 80GB SXM4 the marginal cost is $0.0375 per 1M output tokens, before you add idle capacity, ops staff, or egress. That is typically cheaper than frontier API pricing at sustained volume, but API calls win when your load is bursty or too small to keep a GPU busy. The Build vs Buy tab on tokcalc prices this against a specific provider for you.

Try GLM-5.3 (MoE) with your own context and concurrency

The calculator uses the exact same formulas as this page — change the workload and watch every number update.

Open in calculator

GLM-5.3 (MoE) on specific GPUs

GLM-5.3 (MoE) on B200 SXM 192GBGLM-5.3 (MoE) on H200 SXM5 141GB
Model architecture
The numbers behind the KV-cache and throughput math.
Total parameters753.33B
Active parameters per token19.12B
ArchitectureMixture-of-Experts
Attention layoutHybrid: 78 MLA
Transformer layers78
Hidden dimension6144
Attention heads64 query / 64 KV (GQA)
Head dimension192
Vocabulary size154,880
Max context (catalog)1M
KV cache per token @ 8K87.8 KB
Native checkpoint size755.62 GB
Native weight formatfp8
Licenseglm-5.3

MLA layers store one compressed latent plus a small decoupled RoPE key, not a full key/value pair. This model's architecture is only partially modelled — the memory figure above is a lower bound, because one architecture-specific cache is not modelled. Throughput should be read as an upper bound.

Limitations of these numbers
Where the source does not fully establish the values above. The figures are a lower bound — real usage is at least this.
  • •DeepSeek-style sparse attention (DSA): 78 layers carry a per-token index cache (index_n_heads=32, index_head_dim=128) that this engine does not model, so memory figures UNDERSTATE the real requirement at long context. The MLA KV cache itself is modelled exactly.
  • •Active parameter count (19.117B) is taken from the model card; the checkpoint config does not state it.
  • •Checkpoint carries 1 multi-token-prediction head layer — a draft head, not part of the per-token decode path, so it is not counted as active parameters.