Groqlpu2024

GroqLPU (per-chip, 230MB SRAM): Specs, VRAM, Throughput & Cost

GroqLPU (per-chip, 230MB SRAM) has 0.23 GB of VRAM and 9,000 GB/s of memory bandwidth. Most models in the catalog need multi-GPU configurations on this accelerator. All figures are computed by tokcalc's open-source capacity model.

VRAM
0.23 GB
Memory bandwidth
9,000
GB/s
FP16/BF16
750 TF
Price
—
quote required
Hardware specifications
VendorGroq
Categorylpu
VRAM0.23 GB
Memory bandwidth9,000 GB/s
FP16/BF16 dense compute750 TFLOPS
NVLink bandwidthnone
Release year2024
Typical cloud pricequote required
NoteLPU architecture; SRAM-based, very different from HBM GPUs. Use cited model-specific benchmarks.
Models that fit on a single GroqLPU (per-chip, 230MB SRAM)
At each model's recommended quantization, 8K context and 8 concurrent users. Cost is per 1M output tokens.

No model in the tokcalc catalog fits on a single GroqLPU (per-chip, 230MB SRAM) at its recommended quantization. Use multi-GPU tensor parallelism or context parallelism.

Llama 3.3 70B across every accelerator
Same model (fp8), same workload, every GPU in the catalog.
GPUAggregate tok/sCost / 1M tok$/hr
Cerebras CS-3 (WSE-3)45076——
RTX 4080 16GB6464$0.1031$0.3
RTX A4000 16GB4039$0.1651$0.3
RTX 5090 32GB4039$0.1376$0.5
A100 40GB SXM43493$0.2863$0.9
RTX 4090 24GB2272$0.1956$0.4
V100 SXM2 32GB2028$0.2739$0.5
RTX 3090 24GB2017$0.1377$0.25
H100 SXM5 80GB1888$0.7358$2.5
TPU v6e Trillium (32GB)1846$0.4816$0.8
T4 16GB1836$0.2421$0.2
A10G 24GB1352$0.2958$0.36
Frequently asked questions

What is GroqLPU (per-chip, 230MB SRAM) used for?

GroqLPU (per-chip, 230MB SRAM) is a lpu accelerator from Groq with 0.23 GB of VRAM and 9,000 GB/s of memory bandwidth. LPU architecture; SRAM-based, very different from HBM GPUs. Use cited model-specific benchmarks.. It is aimed at specialised inference workloads.

Which models fit on a single GroqLPU (per-chip, 230MB SRAM)?

No model in the tokcalc catalog fits on a single GroqLPU (per-chip, 230MB SRAM) at its recommended quantization — even the smallest models here need more than 0.23 GB once weights and KV cache are loaded, or they only fit at a much lower precision than recommended.

How does GroqLPU (per-chip, 230MB SRAM) compare to other accelerators?

Memory bandwidth drives decode speed and VRAM drives capacity. GroqLPU (per-chip, 230MB SRAM) has 9,000 GB/s of bandwidth and 0.23 GB of VRAM, plus 750 TFLOPS of dense FP16/BF16 compute. The comparison table on this page ranks every accelerator in the catalog on the same reference workload so you can see where it lands.

Size a real deployment on GroqLPU (per-chip, 230MB SRAM)

Pick your model, context length, and concurrency — the calculator computes the rest with the same formulas.

Open in calculator