Qwen3

Qwen3-14B

The Qwen3-14B model card says 14.8B parameters.

Apache-2.0 · 14.8B · context 32,768 · YaRN 131,072

The model card says 32,768 tokens natively and 131,072 with YaRN. config.json max_position_embeddings is 40,960.

From the model card

Strengths

  • The Qwen3 card describes one model with a thinking mode and a non-thinking mode.
  • The card presents the family as stronger than Qwen2.5 on reasoning, math, coding, and agent tasks, and as covering 100+ languages.

Weaknesses

  • This repository is a text model. Vision is a different Qwen checkpoint.
  • Context past 32,768 needs the YaRN setup the card documents. It is not the default.
  • On the Ollama tag this lab pulled, the model info showed thinking enabled. The measured rows turned it off.

Sources: Qwen3-14B model card · Qwen3 GitHub · config.json

Memory at 8,192 tokens

Computed, not measured. Weights use llama.cpp’s whole-file averages (Q4_K_M = 4.8944 bits/weight, Q8_0 = 8.5008) on the total parameter count. KV is fp16 and does not depend on the quant. The formula, including the sliding-window and latent-KV cases, is on the methodology page.

Computed memory for Qwen3-14B at 8192 context
Q4_K_MQ8_0
Weights8.43 GiB14.65 GiB
KV at 8,1921.25 GiB1.25 GiB
Total9.68 GiB Computed15.9 GiB Computed

40 layers, 8 KV heads, head dim 128. Every layer stores the full context.

Tokens per second

These are the Frankenstein (HP ProLiant DL380p Gen8) rows whose model name matches this card. Generation tok/s is the median of three runs. Prompt tok/s is uncached prefill, or n/a when that pass was not captured. Thread-sweep rows are included and labeled in the notes. They are not the headline number in the tiers below.

DeviceQuantContextGen tok/sPrompt tok/sDate
CPUQ4_K_M2,0484.41 Measuredn/a2026-11-18
CPUQ4_K_M8,1923.02 Measured17.12026-11-18
CPU · thread sweepQ4_K_M2,0484.24 Measuredn/a2026-11-18
CPU · thread sweepQ4_K_M2,0481.33 Measuredn/a2026-11-18
Tesla P40 (GDDR5)Q4_K_M2,04825.42 Measured3642026-11-18
Tesla P40 (GDDR5)Q4_K_M8,19221.85 Measured4342026-11-18

The full notes, including RSS and leftover models, are on the benchmark table.

Hardware tiers

Minimum is the smallest RAM that holds Q4_K_M plus 8k KV. Suggested does not name a second machine or a speed this lab has not run. Performance means a CUDA-class card with 12 GB or more of GDDR6 or GDDR6X and the weights fully on the card. The Tesla P40 is 24 GB of GDDR5, so a P40 speed is not the performance-tier number.

TierExample hardwareQuantFittok/s
MinimumCPU-only, with at least 11.68 GiB of system RAM for the Q4_K_M weights, 8k KV, and a 2 GiB planning allowance.Q4_K_MCPU-only, if the RAM is thereCPU: 3.02 gen tok/s, 17.1 prompt tok/s, context 8,192, 2026-11-18, measured
SuggestedA machine you are willing to wait on, or a GPU that can hold the quant. This lab has not measured a separate “comfortable” machine, so this row does not invent one.Q4_K_M until a Q8 run existsWhichever of the machines above actually holds the weightsnot yet measured
PerformanceA CUDA-class card with 12 GB or more of GDDR6 or GDDR6X, weights fully on the card.Q4_K_MFull VRAM on a 12 GB or larger GDDR6-class cardnot yet measured

Measured on the Tesla P40 in Frankenstein (HP ProLiant DL380p Gen8), which is not the performance tier: 21.85 gen tok/s at context 8,192, Q4_K_M, 2026-11-18. GDDR5, 24 GB.

Where it sits

Best-for is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking. The matching build is CPU coding box. Score it against a machine you have on What can my rig run?