TPU v6e Trillium (32GB) has 32 GB of VRAM and 819 GB/s of memory bandwidth. It runs 12 models from the catalog on a single card — up to GPT-OSS 20B (MoE) — and serves Llama 3.3 70B at 1846 tok/s aggregate. All figures are computed by tokcalc's open-source capacity model.
| Vendor | |
| Category | tpu |
| VRAM | 32 GB |
| Memory bandwidth | 819 GB/s |
| FP16/BF16 dense compute | 918 TFLOPS |
| NVLink bandwidth | none |
| Release year | 2024 |
| Typical cloud price | $0.8/hr |
| Model | Params | Quant | VRAM needed | Aggregate tok/s | Cost / 1M tok |
|---|---|---|---|---|---|
| GTE Large (1024-dim) | 0.44B | fp16 | 7.32 GB | 7259 | $0.0306 |
| Multilingual E5 Large (1024-dim) | 0.56B | fp16 | 7.56 GB | 5704 | $0.0390 |
| BGE-M3 (embed, 1024-dim) | 0.57B | fp16 | 7.58 GB | 5623 | $0.0395 |
| SmolLM2 1.7B Instruct | 1.70B | fp16 | 16.28 GB | 1879 | $0.1183 |
| Qwen 3 4B | 4B | fp16 | 17.66 GB | 798.5 | $0.2783 |
| Qwen 2.5-VL 7B (VLM) | 7B | fp16 | 17.76 GB | 456.3 | $0.4870 |
| Mistral 7B v0.3 | 7.24B | fp16 | 23.07 GB | 441.2 | $0.5037 |
| Qwen2 7B | 7.62B | fp16 | 19.00 GB | 419.2 | $0.5301 |
| Qwen 3 8B | 8B | fp16 | 25.66 GB | 399.3 | $0.5566 |
| Llama 3 8B | 8.03B | fp16 | 24.65 GB | 397.8 | $0.5587 |
| Falcon 3 10B Instruct | 10B | fp16 | 30.74 GB | 319.4 | $0.6957 |
| GPT-OSS 20B (MoE) | 20.91B | fp8 | 22.55 GB | 2662 | $0.0835 |
| GPU | Aggregate tok/s | Cost / 1M tok | $/hr |
|---|---|---|---|
| Cerebras CS-3 (WSE-3) | 45076 | — | — |
| RTX 4080 16GB | 6464 | $0.1031 | $0.3 |
| RTX A4000 16GB | 4039 | $0.1651 | $0.3 |
| RTX 5090 32GB | 4039 | $0.1376 | $0.5 |
| A100 40GB SXM4 | 3493 | $0.2863 | $0.9 |
| RTX 4090 24GB | 2272 | $0.1956 | $0.4 |
| V100 SXM2 32GB | 2028 | $0.2739 | $0.5 |
| RTX 3090 24GB | 2017 | $0.1377 | $0.25 |
| H100 SXM5 80GB | 1888 | $0.7358 | $2.5 |
| TPU v6e Trillium (32GB) | 1846 | $0.4816 | $0.8 |
| T4 16GB | 1836 | $0.2421 | $0.2 |
| A10G 24GB | 1352 | $0.2958 | $0.36 |
TPU v6e Trillium (32GB) is a tpu accelerator from Google with 32 GB of VRAM and 819 GB/s of memory bandwidth, typically around $0.8/hr on cloud rental. It is aimed at specialised inference workloads.
12 models in the tokcalc catalog fit on one TPU v6e Trillium (32GB), up to GPT-OSS 20B (MoE) (21B params) at fp8. Below that, throughput and cost per million tokens improve as the model gets smaller.
At fp8 with 8 concurrent users, 4× TPU v6e Trillium (32GB) serves 1846 tokens/sec aggregate (38.46 tok/s per stream), with a 686 ms time to first token. That works out to $0.4816 per 1M output tokens.
Around $0.8/hr for on-demand cloud rental (a typical 2025–2026 estimate across RunPod, Lambda, Modal, AWS, GCP and similar providers). Actual pricing varies by region, commitment, and spot availability — the Build vs Buy tab in the calculator pulls live quotes from several providers.
Memory bandwidth drives decode speed and VRAM drives capacity. TPU v6e Trillium (32GB) has 819 GB/s of bandwidth and 32 GB of VRAM, plus 918 TFLOPS of dense FP16/BF16 compute. The comparison table on this page ranks every accelerator in the catalog on the same reference workload so you can see where it lands.
Pick your model, context length, and concurrency — the calculator computes the rest with the same formulas.