I’m brand new
Never installed a model. Not sure my computer can do it.
Start hereLocal AI, in plain words
I’ll walk you through it one step at a time: what you need, what to install, and which model to try first. When I give you a number, I tell you where it came from.
Free. No account needed. About an hour to start.
Pick the one that sounds like you.
Never installed a model. Not sure my computer can do it.
Start hereWant a better model, your own documents, or a better format.
Browse guidesI want real speeds and dated prices before I spend money.
Builds and benchmarksApache-2.0
Computed Q4_K_M weights plus 8,192 tokens of KV: 3.4 GiB Computed
6.62 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured
Apache-2.0
Computed Q4_K_M weights plus 8,192 tokens of KV: 5.8 GiB Computed
4.66 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured
Apache-2.0
Computed Q4_K_M weights plus 8,192 tokens of KV: 9.7 GiB Computed
3.02 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured
Apache-2.0
Computed Q4_K_M weights plus 8,192 tokens of KV: 18.1 GiB Computed
8.73 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured
Memory is computed from each model’s published size at a 4-bit quant, plus KV. It is not a measurement of the file on disk. Speeds are only the rows I have actually run. How I work this out.
The only guide written so far is It will not run. The rest of the list is not published yet, so I am not filling this page with headlines I have not written.
No news posts yet. When I write one, it will have a date and a link to the source.
From my bench
My test machine is Frankenstein (HP ProLiant DL380p Gen8). Each speed names the model, the quant, the context, and the day I ran it. If I have not measured something, the cell stays empty.
See all benchmarks| Model | On | Speed | Badge |
|---|---|---|---|
| Qwen3-8B | Frankenstein (HP ProLiant DL380p Gen8), CPU | 4.66 gen tok/s | Measured |
| Qwen3-14B | Frankenstein (HP ProLiant DL380p Gen8), CPU | 3.02 gen tok/s | Measured |
| Qwen3-30B-A3B | Frankenstein (HP ProLiant DL380p Gen8), CPU | 8.73 gen tok/s | Measured |
| Llama 3.3 70B Instruct | — | — | Not yet measured |