Local AI, in plain words

Run an AI model on your own computer.

I’ll walk you through it one step at a time: what you need, what to install, and which model to try first. When I give you a number, I tell you where it came from.

Free. No account needed. About an hour to start.

Your first hour

  1. 1
    What is a local LLM
    The weights stay on a machine you have
  2. 2
    Hardware
    VRAM, system RAM, and a CPU-only box
  3. 3
    Words
    The words on a model page
  4. 4
    Pick a model
    One that fits the machine you have
  5. 5
    Install
    One official runner
  6. 6
    First chat
    Send one prompt
  7. 7
    Next
    One useful next project

Where are you right now?

Pick the one that sounds like you.

I’m brand new

Never installed a model. Not sure my computer can do it.

Start here

I have one running

Want a better model, your own documents, or a better format.

Browse guides

Good first models

All models

Qwen3-4B

Apache-2.0

Computed Q4_K_M weights plus 8,192 tokens of KV: 3.4 GiB Computed

6.62 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured

Qwen3-8B

Apache-2.0

Computed Q4_K_M weights plus 8,192 tokens of KV: 5.8 GiB Computed

4.66 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured

Qwen3-14B

Apache-2.0

Computed Q4_K_M weights plus 8,192 tokens of KV: 9.7 GiB Computed

3.02 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured

Qwen3-30B-A3B

Apache-2.0

Computed Q4_K_M weights plus 8,192 tokens of KV: 18.1 GiB Computed

8.73 gen tok/s on Frankenstein (HP ProLiant DL380p Gen8), CPU, context 8,192, 2026-11-18. Measured

Memory is computed from each model’s published size at a 4-bit quant, plus KV. It is not a measurement of the file on disk. Speeds are only the rows I have actually run. How I work this out.

Popular guides

All guides

The only guide written so far is It will not run. The rest of the list is not published yet, so I am not filling this page with headlines I have not written.

Local AI news

All news

No news posts yet. When I write one, it will have a date and a link to the source.

From my bench

Numbers I measured myself

My test machine is Frankenstein (HP ProLiant DL380p Gen8). Each speed names the model, the quant, the context, and the day I ran it. If I have not measured something, the cell stays empty.

See all benchmarks
ModelOnSpeedBadge
Qwen3-8BFrankenstein (HP ProLiant DL380p Gen8), CPU4.66 gen tok/sMeasured
Qwen3-14BFrankenstein (HP ProLiant DL380p Gen8), CPU3.02 gen tok/sMeasured
Qwen3-30B-A3BFrankenstein (HP ProLiant DL380p Gen8), CPU8.73 gen tok/sMeasured
Llama 3.3 70B Instruct——Not yet measured