Benchmarks
What ran, and how fast
One row is one run: a machine, a model, a quant, a context length, and the software that served it. Generation tokens per second and prompt tokens per second are both recorded, because a long prompt and a short answer are different jobs.
- Machine
- The lab name, plus enough hardware to tell two boxes apart.
- Model
- The model that ran, not a size class.
- Quant
- The quantization of that file.
- Size
- Parameter size or on-disk size. Say which in the notes if it is ambiguous.
- Context
- The context length of the run.
- Gen tok/s
- Output tokens per second.
- Prompt tok/s
- Prompt-processing tokens per second.
- RAM
- What the run used, or installed RAM. Say which in the notes.
- Date
- The day of the run.
- Software
- Runtime and version. The notes sit under it.
| Machine | Model | Quant | Size | Context | Gen tok/s | Prompt tok/s | RAM | Date | Software |
|---|---|---|---|---|---|---|---|---|---|
| Frankenstein | qwen3:4b-instruct-2507-q4_K_M | Q4_K_M | 4.0B dense, 2.5 GB | 2048 | 12.36 | n/a | 3.1 GiB | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0, Ollama default threads. Short prompt (~110 tokens). Clean RSS. Uncached prefill was not captured. |
| Frankenstein | qwen3:4b-instruct-2507-q4_K_M | Q4_K_M | 4.0B dense, 2.5 GB | 8192 | 6.62 | 52.6 | 4.9 GiB | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0. Long prompt. Prefill is the first calibration pass (~3552 tokens), approximate. Clean RSS. |
| Frankenstein | qwen3:8b | Q4_K_M | 8.2B dense, 5.2 GB | 2048 | 6.91 | n/a | 10.4 GiB observed | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0, default threads. Short prompt. RSS includes a leftover 4B. Uncached prefill was not captured. |
| Frankenstein | qwen3:8b | Q4_K_M | 8.2B dense, 5.2 GB | 8192 | 4.66 | 31.3 | 12.3 GiB observed | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0. Long prompt (~4012 tokens). Prefill is the first uncached run. RSS includes a leftover 4B. |
| Frankenstein | qwen3:14b | Q4_K_M | 14.8B dense, 9.3 GB | 2048 | 4.41 | n/a | 9.4 GiB clean | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0, Ollama default threads. Short prompt. This is the default-thread cell. During the run RSS was 21.7 GiB with other models still loaded. 9.4 GiB is the later peak with 14B alone. Uncached prefill was not captured. |
| Frankenstein | qwen3:14b | Q4_K_M | 14.8B dense, 9.3 GB | 8192 | 3.02 | 17.1 | 23.8 GiB observed | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0. Long prompt. Prefill is the first uncached run. RSS on run 1 includes a leftover 4B. |
| Frankenstein | qwen3:14b | Q4_K_M | 14.8B dense, 9.3 GB | 2048 | 4.24 | n/a | 9.4 GiB clean | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_thread=24 (physical cores), short prompt. Median of three runs: 4.12, 4.24, 4.43. Uncached prefill was not captured. |
| Frankenstein | qwen3:14b | Q4_K_M | 14.8B dense, 9.3 GB | 2048 | 1.33 | n/a | 9.4 GiB clean | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_thread=48 (all SMT threads), short prompt. Median of 1.08, 1.33, 1.35. About 3.2× slower than 24 threads. Uncached prefill was not captured. |
| Frankenstein | frank-qwen3-30b-64k | Q4_K_M | 30.5B MoE (3B active), 18 GB | 2048 | 9.58 | n/a | 27.2 GiB observed | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0, default threads. Qwen3-30B-A3B weights. The tag only sets num_ctx 65536; this run used 2048. Short prompt. RSS includes a leftover 14B. Uncached prefill was not captured. |
| Frankenstein | frank-qwen3-30b-64k | Q4_K_M | 30.5B MoE (3B active), 18 GB | 8192 | 8.73 | 53.2 | 28.5 GiB observed | Nov 18, 2026 | Ollama 0.35.0 CPU only, num_gpu=0. Long prompt (~4012 tokens). Prefill is the first uncached run. Context 65536 was not loaded. RSS includes a leftover 14B. |
| Frankenstein | qwen3:4b-instruct-2507-q4_K_M | Q4_K_M | 4.0B dense, 2.5 GB | 2048 | 69.57 | n/a | 3.1 GiB VRAM | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Short prompt. 5.6× the CPU generation rate. Clean VRAM. Uncached prefill was not captured. |
| Frankenstein | qwen3:4b-instruct-2507-q4_K_M | Q4_K_M | 4.0B dense, 2.5 GB | 8192 | 60.03 | 1,100 | 4.9 GiB VRAM | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Long prompt. 9.1× the CPU generation rate. Prefill ~1100 is the first calibration pass; load and prefill may overlap. Clean VRAM. |
| Frankenstein | qwen3:8b | Q4_K_M | 8.2B dense, 5.2 GB | 2048 | 44.18 | 391 | 10.3 GiB VRAM observed | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Short prompt. 6.4× the CPU generation rate. Prefill ~391 is run 1 and approximate. VRAM likely includes a leftover 4B. |
| Frankenstein | qwen3:8b | Q4_K_M | 8.2B dense, 5.2 GB | 8192 | 40.22 | 748 | 12.1 GiB VRAM observed | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Long prompt. 8.6× the CPU generation rate. Prefill 748 is the first uncached run. VRAM may include a leftover 4B. |
| Frankenstein | qwen3:14b | Q4_K_M | 14.8B dense, 9.3 GB | 2048 | 25.42 | 364 | 21.3 GiB VRAM observed | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Short prompt. 5.8× the CPU generation rate. Prefill ~364 is run 1 and approximate. VRAM is 14B plus a leftover 8B, nearly the full 24 GB. |
| Frankenstein | qwen3:14b | Q4_K_M | 14.8B dense, 9.3 GB | 8192 | 21.85 | 434 | 11.1 GiB VRAM | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Long prompt. 7.2× the CPU generation rate. Prefill 434 is the first uncached run. 11.1 GiB is the settled VRAM for 14B alone. |
| Frankenstein | frank-qwen3-30b-64k | Q4_K_M | 30.5B MoE (3B active), 18 GB | 2048 | 65.29 | 255 | 17.8 GiB VRAM | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Short prompt. 6.8× the CPU generation rate. Prefill ~255 is run 1 and approximate. Clean VRAM. Context 65536 was not loaded. |
| Frankenstein | frank-qwen3-30b-64k | Q4_K_M | 30.5B MoE (3B active), 18 GB | 8192 | 57.64 | 871 | 18.9 GiB VRAM | Nov 18, 2026 | Ollama 0.35.0 Tesla P40, num_gpu=99. Long prompt. 6.6× the CPU generation rate. Prefill 871 is the first uncached run. Clean VRAM, about 5 GiB free on the 24 GB card. Context 65536 was not loaded. |
No rows match that filter.
On a narrow screen, scroll the table sideways. The machine name stays put.
Frankenstein, 18 Nov 2026
These rows are from Frankenstein: two Xeon E5-2697 v2 CPUs (24 cores, 48 threads), 288 GB of DDR3 RDIMM at 1067 MT/s, and a Tesla P40 24 GB, running Ollama 0.35.0. Generation speed is the median of three runs. Prompt speed is the first uncached prefill. Identical prompts after that hit Ollama’s prefix cache and are not in this table. The write-up is CPU-only on a used Xeon: what actually runs.
Add a row by editing src/data/benchmarks.json. The file is checked at build time. A measured table cannot keep a sample row, and a measured row needs a generation speed and a date. Leave prompt tok/s null when the uncached prefill was not captured. The methodology is the rule for what belongs in a cell.