EmbeddingDense0.57B params8K max context

BGE-M3 (embed, 1024-dim) VRAM Requirements: How Many GPUs & What It Costs

BGE-M3 (embed, 1024-dim) needs 1.1 GB of VRAM for its weights at FP16 — 1.5 GB once a KV cache is loaded. Below is the full breakdown: which GPUs fit, how many you need, what it serves, and what it costs per million tokens. Every figure comes from tokcalc's open-source capacity model.

Weights (FP16)
1.1 GB
1.5 GB with KV cache
Fits on 1 GPU?
Yes
RTX 3080 10GB or larger
Cheapest config
$0.0106
1× RTX 3080 10GB / 1M tokens
Fastest config
137324
tok/s on 1× Cerebras CS-3 (WSE-3)
VRAM requirement by quantization
Weight footprint = params × bytes per param. The “with KV cache” column adds one request at 4K context.
QuantizationWeightsWith KV cacheSmallest single GPU
FP16recommended1.1 GB1.5 GBRTX 3080 10GB (10 GB)
FP8 (E4M3)0.6 GB1.0 GBRTX 3080 10GB (10 GB)
AWQ 4-bit0.3 GB0.7 GBRTX 3080 10GB (10 GB)
Which GPU should you use?
Reference workload: 8K context, 8 concurrent users, continuous batching, FP16. Ranked by fitting first, then by tokens per dollar-hour.
GPUGPUsTotal VRAMDecode tok/sAggregate tok/sCost / 1M tokens
RTX 3080 10GB1×10 GB434.95218$0.0106
RTX 3090 24GB1×24 GB535.66427$0.0108
RTX 5090 32GB1×32 GB102512304$0.0113
RTX 4090 24GB1×24 GB576.86921$0.0161
RTX 4080 16GB1×16 GB410.34923$0.0169
AMD MI300X 192GB1×192 GB303336391$0.0191
V100 SXM2 32GB1×32 GB515.06180$0.0225
A100 40GB SXM41×40 GB886.910643$0.0235
AMD MI325X 288GB1×288 GB343341197$0.0236
A100 80GB SXM41×80 GB114413732$0.0243
A10G 24GB1×24 GB343.34120$0.0243
Intel Gaudi 3 128GB1×128 GB188822658$0.0245
How concurrency changes with context length
On 1× RTX 3080 10GB at FP16. KV cache scales linearly with context, so long contexts cut the number of users you can serve from the same GPUs.
ContextConcurrent usersKV per requestFits?
4K22412.3 MB✓ fits
8K11824.6 MB✓ fits
33K23.22 GB✓ fits
66K16.44 GB✓ fits
131K012.88 GB✗ does not fit
Frequently asked questions

Can BGE-M3 (embed, 1024-dim) run on a single GPU?

Yes. At FP16 the model needs 1.5 GB of VRAM, so it fits on a single RTX 3080 10GB or anything with more VRAM. At full FP16 it needs 1.5 GB, which fits on one RTX 3080 10GB or larger.

How many GPUs does BGE-M3 (embed, 1024-dim) need?

At FP16 with 8,192-token context and 8 concurrent users, the cheapest configuration that fits is 1× RTX 3080 10GB (10 GB total VRAM). $0.0106 per 1M output tokens.

How fast can BGE-M3 (embed, 1024-dim) be served?

On 1× Cerebras CS-3 (WSE-3) at FP16, the aggregate throughput is 137324 tokens/sec with continuous batching. That hardware has no published rental price, so its cost per million tokens is not comparable.

How much VRAM does BGE-M3 (embed, 1024-dim) need?

Weights alone: 1.1 GB at FP16, 1.1 GB at FP16. Add KV cache for your context length and concurrency — every concurrent request at 8,192 tokens costs extra VRAM, so budget headroom above the weight figure.

Is it cheaper to self-host BGE-M3 (embed, 1024-dim) than to call an API?

At 1× RTX 3080 10GB the marginal cost is $0.0106 per 1M output tokens, before you add idle capacity, ops staff, or egress. That is typically cheaper than frontier API pricing at sustained volume, but API calls win when your load is bursty or too small to keep a GPU busy. The Build vs Buy tab on tokcalc prices this against a specific provider for you.

Try BGE-M3 (embed, 1024-dim) with your own context and concurrency

The calculator uses the exact same formulas as this page — change the workload and watch every number update.

Open in calculator

BGE-M3 (embed, 1024-dim) on specific GPUs

BGE-M3 (embed, 1024-dim) on RTX 3080 10GBBGE-M3 (embed, 1024-dim) on RTX 3090 24GBBGE-M3 (embed, 1024-dim) on RTX 5090 32GBBGE-M3 (embed, 1024-dim) on RTX 4090 24GBBGE-M3 (embed, 1024-dim) on RTX 4080 16GBBGE-M3 (embed, 1024-dim) on AMD MI300X 192GBBGE-M3 (embed, 1024-dim) on V100 SXM2 32GBBGE-M3 (embed, 1024-dim) on A100 40GB SXM4
Model architecture
The numbers behind the KV-cache and throughput math.
Total parameters0.57B
Active parameters per token0.57B
ArchitectureDense
Attention layoutFull attention (GQA)
Transformer layers24
Hidden dimension1024
Attention heads16 query / 16 KV (GQA)
Head dimension64
Vocabulary size250,002
Max context (catalog)8K
KV cache per token @ 8K96.0 KB

Every layer caches keys and values for the whole context, so the KV cache grows linearly with conversation length.