DeepSeek-R1-Distill-Qwen-14B
The DeepSeek-R1 card names this distill 14B, based on Qwen2.5-14B. The card does not publish a separate exact parameter total, so this page uses the 14B size class.
config.json max_position_embeddings is 131072. The sliding window in that config is 131072, so the KV formula treats every layer as full attention.
From the model card
Strengths
- The DeepSeek-R1 card says these dense models are distilled from DeepSeek-R1's reasoning traces.
- The 14B distill is based on Qwen2.5-14B.
Weaknesses
- Reasoning traces spend context and time.
- Two licenses apply: MIT on DeepSeek's files, and the base model's license.
- No Frankenstein (HP ProLiant DL380p Gen8) row yet. Text only.
Sources: DeepSeek-R1 model card · Distill model card · config.json
Memory at 8,192 tokens
Computed, not measured. Weights use llama.cpp’s whole-file averages (Q4_K_M = 4.8944 bits/weight, Q8_0 = 8.5008) on the total parameter count. KV is fp16 and does not depend on the quant. The formula, including the sliding-window and latent-KV cases, is on the methodology page.
| Q4_K_M | Q8_0 | |
|---|---|---|
| Weights | 7.98 GiB | 13.85 GiB |
| KV at 8,192 | 1.5 GiB | 1.5 GiB |
| Total | 9.48 GiB Computed | 15.35 GiB Computed |
48 layers, 8 KV heads, head dim 128. Every layer stores the full context.
Tokens per second
Not yet measured. This lab has no Frankenstein (HP ProLiant DL380p Gen8) row for DeepSeek-R1-Distill-Qwen-14B, so there is no estimated tok/s either.
Hardware tiers
Minimum is the smallest RAM that holds Q4_K_M plus 8k KV. Suggested does not name a second machine or a speed this lab has not run. Performance means a CUDA-class card with 12 GB or more of GDDR6 or GDDR6X and the weights fully on the card. The Tesla P40 is 24 GB of GDDR5, so a P40 speed is not the performance-tier number.
| Tier | Example hardware | Quant | Fit | tok/s |
|---|---|---|---|---|
| Minimum | CPU-only, with at least 11.48 GiB of system RAM for the Q4_K_M weights, 8k KV, and a 2 GiB planning allowance. | Q4_K_M | CPU-only, if the RAM is there | CPU: not yet measured |
| Suggested | A machine you are willing to wait on, or a GPU that can hold the quant. This lab has not measured a separate “comfortable” machine, so this row does not invent one. | Q4_K_M until a Q8 run exists | Whichever of the machines above actually holds the weights | not yet measured |
| Performance | A CUDA-class card with 12 GB or more of GDDR6 or GDDR6X, weights fully on the card. | Q4_K_M | Full VRAM on a 12 GB or larger GDDR6-class card | not yet measured |
Where it sits
Best-for is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking. The matching build is CPU coding box. Score it against a machine you have on What can my rig run?