Build 03Placeholder
RAG and a home assistant
Query your own notes and manuals on hardware you control, with a small chat model and an embedder.
Chat can be a modest model. The extra cost is memory for a second model left resident, disk for the index, and idle watts if the box stays on.
- Price
- Placeholder
- Price pulled on
- Placeholder
- Source
- Placeholdername the sold listing or retailer page on the day it is pulled
- Idle
- Placeholder
- Load
- Placeholder
- Model
- Placeholder
- Quant
- Placeholder
- Context
- Placeholder
- Gen tok/s
- Placeholder
- Prompt tok/s
- Placeholder
Idle watts matter if this machine stays on for a home assistant. Meter idle separately from a query.
Record the chat model. Note in the same row whether the embedding model was resident.
Don’t buy this if
- You have not decided which documents are in scope. Buy disk after that, not before.
- The whole household will hit it at once. That is the family server, and it needs a queue story.
- You are shopping for a GPU only to run embeddings. Time the CPU embedder first and keep the number.
Parts
| Part | Item | Price (USD) | Source | Notes |
|---|---|---|---|---|
| Host | Placeholdera small always-on machine, or the CPU server if it is already running | Placeholder | Placeholder | Reuse a machine you already meter before buying another one. |
| Memory | Placeholderchat model plus embedding model resident, with OS headroom | Placeholder | Placeholder | The second model is the usual surprise, not the chat model alone. |
| Disk | Placeholderroom for source documents and the index | Placeholder | Placeholder | Capacity follows the corpus you actually have, not a round number. |
| Network | Placeholderwired Ethernet if the box lives in a closet | Placeholder | Placeholder | A home assistant that drops off Wi-Fi is a support call. |
No affiliate links on this page yet. If one is added, the source cell will say so. See the affiliate disclosure.
Gotchas
- Retrieval quality dominates the hardware. A faster machine will not repair a bad chunking setup. This site measures the model runtime, not the index.
- Keeping the embedder loaded changes the RAM story. Write down whether it was resident during the timed run.
- Context length still applies. Retrieved chunks have to fit in the window you benchmark.
RAG on a home machine is two workloads that people price as one. A chat model answers. An embedding model turns documents into vectors, and it may need to stay loaded if you reindex or query often. Memory planning that only names the chat model is incomplete. The parts table has a line for that, with the price left blank.
This site will not score your notes, your chunk size, or your vector database. Those choices change answers more than another 10% of tokens per second, and they are not what a wall meter shows. What the lab can log is the chat model’s prompt and generation speed while the embedder is or is not resident, plus idle watts for a box that is supposed to be there when you ask.
If the CPU-only Xeon is already on, it may be the host, and this page becomes a workload rather than a second purchase. That is a good outcome. A dedicated assistant box is justified when the server is too loud, too hungry at idle, or shared with jobs you do not want in the same queue.
The first real numbers belong in the benchmark table, with the software version. Home-assistant tooling moves. A speed without the version is not comparable to next month’s run.
Models that point here
A judgment from the model card and the size, not a speed ranking. Memory and the measured-or-not speeds are on each model page.
- Gemma 3 4B · 4B
- Gemma 3 12B · 12B