Start here · Step 4 of 7
Step 4 of 7 · about 5 minutes
Pick a model
Start from what you have, not from a leaderboard. Open What can my rig run? and type the RAM and the GPU. It stays in the browser.
A sensible first pick for most people is a small instruct model in the 4B–8B range at Q4_K_M. These are suggestions by size and license, not a ranking of quality, and not a promise of speed on your machine.
| Model | License | Q4_K_M + 8k |
|---|---|---|
| Qwen3-4B | Apache-2.0 | 3.4 GiB Computed |
| Qwen3-8B | Apache-2.0 | 5.8 GiB Computed |
| Llama 3.1 8B Instruct | Llama 3.1 Community License | 5.6 GiB Computed |
| Gemma 3 4B | Gemma terms | 2.5 GiB Computed |
Qwen3-4B and Qwen3-8B are Apache-2.0, and I have already run them on Frankenstein (HP ProLiant DL380p Gen8). Llama 3.1 8B Instruct is the name many people already know; read its license before you download. Gemma 3 4B takes text and an image; its terms are not Apache-2.0.
Match the job: chat, coding, reasoning traces, or a home assistant that can see an image. The “best for” field on each model is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking.
Read the license before you download. Apache-2.0 and MIT are straightforward. Llama Community, Gemma terms, and “Apache plus a usage policy” are not the same thing.
If the computed Q4_K_M + 8k total does not fit, pick a smaller model or a shorter context. Do not start with a 70B.