For years I said: local AI on consumer hardware is mostly not worth the time to test. Disappointment and waiting time. Until this week – two new variations really deliver, and my opinion has changed.
The price advantage is enormous: for free, private and on your own hardware – while the leading cloud model costs sixty dollars per million tokens.
The shift since August 2026
Until August, local AI on consumer hardware was mostly a compromise. Two new model variations turn it into a real option – for twelve as well as twenty-four gigabytes of graphics memory.
Two models change the equation
Qwen 3.8 – the dense model
For everyone with a modern graphics card with 24 GB of memory. Delivers quality you pay for per token at cloud providers.
Ornith 1.5 – the surprise
35 billion parameters, but only 3 billion are loaded per request. Thanks to expert technology it even runs on a 12 GB card that was outdated five years ago.
Without a cloud data center
Completely for free on your own hardware, private and without subscription. Your data never leaves your own machine.
Expert technology explained simply
My favorite comparison: you would not ask your mother what the latest trends are – and you would not ask your partner about your childhood. Both are experts for different areas, and that is exactly how an expert model works: not all of its knowledge is always active, only the part the request needs. That is why a 35 billion parameter model fits into the memory of an old 12 GB card.
What such a setup with two used cards looks like in practice is described in my article "Qwen 3.8 the cheap way".
A waste of time or the biggest shift in years?
The video explains expert technology with your mother, your partner and one everyday question – and leaves open the decisive question at the end: what is your experience with local AI?
➡️ All content in the embedded video was created with AI.