Qwen3

Qwen3-30B-A3B

The Qwen3-30B-A3B model card says 30.5B total parameters and 3.3B activated.

Apache-2.0 · 30.5B total, 3.3B active · context 32,768 · YaRN 131,072

The model card says 32,768 tokens natively and 131,072 with YaRN. config.json max_position_embeddings is 40,960. The Frankenstein (HP ProLiant DL380p Gen8) tag sets num_ctx 65536; the measured runs used 2048 and 8192.

From the model card

Strengths

  • Mixture-of-experts: the card says 3.3B parameters are active per token, out of 30.5B total.
  • The same Qwen3 card describes thinking and non-thinking modes, and presents the family as a step up on reasoning, math, coding, and agent tasks.

Weaknesses

  • Memory follows the 30.5B total. All experts are stored. The 3.3B figure is compute, not disk.
  • This repository is a text model.
  • Context past 32,768 needs YaRN. The measured runs did not load a 65,536 context.

Sources: Qwen3-30B-A3B model card · Qwen3 GitHub · config.json

Memory at 8,192 tokens

Computed, not measured. Weights use llama.cpp’s whole-file averages (Q4_K_M = 4.8944 bits/weight, Q8_0 = 8.5008) on the total parameter count. KV is fp16 and does not depend on the quant. The formula, including the sliding-window and latent-KV cases, is on the methodology page.

Computed memory for Qwen3-30B-A3B at 8192 context
Q4_K_MQ8_0
Weights17.38 GiB30.18 GiB
KV at 8,1920.75 GiB0.75 GiB
Total18.13 GiB Computed30.93 GiB Computed

48 layers, 4 KV heads, head dim 128. Every layer stores the full context.

Tokens per second

These are the Frankenstein (HP ProLiant DL380p Gen8) rows whose model name matches this card. Generation tok/s is the median of three runs. Prompt tok/s is uncached prefill, or n/a when that pass was not captured. Thread-sweep rows are included and labeled in the notes. They are not the headline number in the tiers below.

DeviceQuantContextGen tok/sPrompt tok/sDate
CPUQ4_K_M2,0489.58 Measuredn/a2026-11-18
CPUQ4_K_M8,1928.73 Measured53.22026-11-18
Tesla P40 (GDDR5)Q4_K_M2,04865.29 Measured2552026-11-18
Tesla P40 (GDDR5)Q4_K_M8,19257.64 Measured8712026-11-18

The full notes, including RSS and leftover models, are on the benchmark table.

Hardware tiers

Minimum is the smallest RAM that holds Q4_K_M plus 8k KV. Suggested does not name a second machine or a speed this lab has not run. Performance means a CUDA-class card with 12 GB or more of GDDR6 or GDDR6X and the weights fully on the card. The Tesla P40 is 24 GB of GDDR5, so a P40 speed is not the performance-tier number.

TierExample hardwareQuantFittok/s
MinimumCPU-only, with at least 20.13 GiB of system RAM for the Q4_K_M weights, 8k KV, and a 2 GiB planning allowance.Q4_K_MCPU-only, if the RAM is thereCPU: 8.73 gen tok/s, 53.2 prompt tok/s, context 8,192, 2026-11-18, measured
SuggestedA machine you are willing to wait on, or a GPU that can hold the quant. This lab has not measured a separate “comfortable” machine, so this row does not invent one.Q4_K_M until a Q8 run existsWhichever of the machines above actually holds the weightsnot yet measured
PerformanceA CUDA-class card with enough GDDR6 or GDDR6X to hold about 19.13 GiB (Q4_K_M weights, 8k KV, and a 1 GiB planning allowance). 12 GB is not enough for that total.Q4_K_MFull VRAM only if the card is larger than 12 GBnot yet measured

Measured on the Tesla P40 in Frankenstein (HP ProLiant DL380p Gen8), which is not the performance tier: 57.64 gen tok/s at context 8,192, Q4_K_M, 2026-11-18. GDDR5, 24 GB.

Where it sits

Best-for is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking. The matching build is CPU-only used Xeon. Score it against a machine you have on What can my rig run?