A project showed up on GitHub called kimi-k3-in-c.
Over 5,000 stars, written in pure C, claiming to run all 2.78 trillion parameters of Kimi K3 on an ordinary computer with 8 GB of RAM. No GPU, no cloud servers. I know this model pretty well — I wired it into my own tooling when it came out, and lately it's been my daily driver. My first reaction was the same as most of the comment section: so GPUs really aren't needed anymore?
After reading through the repo, my conclusion landed in exactly the opposite direction.
The project is real, and the engineering is solid
First, the fact-check.
| Item | What I found |
|---|---|
| Repo | Real — 5,485 stars / 892 forks as of today, Apache-2.0 |
| Code | ~250k lines of C99, zero dependencies, no math libraries borrowed, 22 kernel unit tests |
| Verification | A golden test compares logits against the official PyTorch implementation, with error within 8% of tolerance |
This is not a scam project. It has a tech report, full CI, and twelve memory configurations from 8 GB to 224 GB — and the author proved something genuinely hard: all twelve tiers produce byte-identical token ids. The 8 GB run and the 224 GB run give you exactly the same output.
But — each token takes 32.7 seconds. Ask it a question and you'll finish a cup of coffee before the answer arrives. And first you need a 1.7 TB local NVMe drive just to hold the model files.
The author says it himself at the end of the README: "The point was never the speed."
Four moves, not four compressions
Seeing "2.78T in 8 GB," most people's first thought is: it compressed the model, right?
No. Not a single weight was dropped or approximated. All it changes is where the bytes live.
A metaphor: you're writing an article in a library, and before every word you have to flip through the relevant encyclopedia volumes. That encyclopedia — the model's 2.78 trillion parameters — can sit in three places:
| Where it sits | What it maps to | Hauling speed |
|---|---|---|
| A huge workbench at your elbow | GPU memory (HBM — special memory soldered right next to the GPU chip) | 3–8 TB per second |
| Your desk | System RAM (DDR5) | 50–100 GB per second |
| A warehouse on basement level 3 | NVMe SSD | 3–7 GB per second |
What this project does is move the encyclopedia from "the workbench" down to "basement level 3." You save the cost of the workbench; the price is a trip downstairs before every single word.
Concretely, four steps.
Step one (5,560 → 1,560 GB) — not this project's doing. K3 ships in MXFP4 format, half a byte per weight. Moonshot did that when training the model; it has nothing to do with this project.
Step two (1,560 → 113 GB) — exploiting MoE sparsity. K3 has 93 layers, 92 of which hold 896 "experts" each — think of 896 separate volumes — but each token only consults 16 of them. At any moment, 96.3% of the model's parameters are asleep, and those sleeping experts make up 93% of the model file (1.447 TB). The move: let them keep sleeping on disk and fetch each volume on demand, keeping a small cache (~13 GB) in RAM — recently used experts stick around in case they're needed again.
Step three (113 → 8.24 GB) — even the backbone doesn't stay. The remaining 108.81 GB is the backbone: the router (which decides which volumes to consult), the attention machinery, the vocabulary — needed by every single token. The author doesn't keep even this in RAM. It's pre-packed into one contiguous file and streamed in layer by layer during inference, used and discarded. RAM never holds more than one layer's worth.
The enabling condition. K3 uses two attention optimizations (KDA and MLA) that squeeze the model's short-term memory down to a tiny footprint. Otherwise, just remembering what has already been said would blow past 8 GB on its own.
32 seconds a token, 80% of it waiting on disk
Once the move is done, the bill arrives.
The most honest chart in the README: at the 8 GB configuration, 80% of the 32 seconds per token is spent waiting for the disk to haul data up. The time spent on actual multiply-accumulate work is a sliver.
Does more RAM help? Not much.
Going from 8 GB to 64 GB makes it 14% faster. Going from 8 GB to 224 GB — 28 times the memory — makes it 1.70 times faster (32.69 → 19.21 seconds per token).
Because the bottleneck was never capacity. It's storage bandwidth.
Even at the 128 GB configuration, every token still reads 25.83 GB from disk. 25.83 GB ÷ NVMe's ~3 GB/s ≈ 9 seconds. That's physics; there's no magic.
Why GPUs are expensive
Now we can answer the question: does this project prove GPUs are unnecessary?
Exactly the opposite. It shows you in the plainest possible terms why GPUs cost money — not because they pack more compute, but because the memory sitting next to that compute moves data a thousand times faster.
Single-user LLM inference is a pure bandwidth problem: for every token, all the activated weights have to stream past the compute units once, and the arithmetic units are mostly idle. A GPU is expensive not for the logic etched on the chip but because HBM is physically soldered right beside it — extremely short wires, extremely wide buses — which makes data movement three orders of magnitude faster.
Back to the library: a GPU isn't a faster calculator. It's the workbench closest to the books. This project moved the books to basement level 3 and then demonstrated that no porter, however hardworking, keeps up with reading at the desk.
The idea itself isn't new — llama.cpp's mmap, KTransformers' MoE expert offloading, and DeepSeek-style CPU/GPU hybrid inference all do similar things. What's unique here is starting from scratch in C with no frameworks at all, and quantifying every step down to reproducible numbers.
Three boundaries
This method doesn't generalize to every model. Three preconditions:
It only works on extremely sparse MoE models. K3 activates 3.7% of its parameters per token — an extreme value. Switch to a dense model like Llama and there are no sleeping experts to leave on disk; every token has to haul every weight. The same trick helps there too (llama.cpp has been doing it forever), but the effect isn't in the same league.
It only holds when serving one person. Serve two users at once and they activate different experts, so disk reads double. GPU economics depend entirely on batching — one haul, hundreds of users sharing it. In a production setting, this approach's economics go straight to zero.
It isn't a usable system. It only continues text — this isn't a ChatGPT-style thing you can hold a conversation with — it only does greedy decoding (pick the highest-probability word at each step: deterministic but dull), and it can't see images. More importantly, no quality evaluation has been run at all. The author's own words: at 11 seconds per token on his machine, a single perplexity run (a standard measure of model output quality) would take days.
Wrapping up
I've written about K3 twice before: one post about the cost surprises after wiring it into my own tool, and one post about making it my daily driver. Both were about using K3.
This time is different. This time someone took K3 apart on the table, so you can see where every layer lives, how long one fetch takes, and why 96% of the books never need to move.
What this project is really worth remembering for: it decomposes the gap between "you need a data center" and "you need a desktop" into four engineering decisions about where bytes live — then proves, with 250k lines of hand-written C and byte-exact comparison, that all four of those decisions can be lossless. As an engineering demo, I'm impressed.
If you want to run big models at home, the right path is still picking a size that matches your VRAM — anything under 30B with 4-bit quantization runs on most consumer cards. Cramming 2.78 trillion parameters into 8 GB of RAM and then waiting half a minute per word is a beautiful experiment, not a practical route.
The README's closing line says it best: "The point was never the speed."
Repo: FareedKhan-dev/kimi-k3-in-c. Performance numbers are the author's own measurements from the README and tech report.