Spec sheets for AI machines lead with memory: 128 GB, 512 GB. Then a model that fits easily runs at a crawl. I wanted to understand why, concretely, so I built it up from the bottom: what a model is, where it lives, and what has to move for every single token.
The short answer: memory size decides whether a model fits. Memory bandwidth decides how fast it writes. Compute decides how fast it reads your prompt. Most disappointing machines are short on one of the last two.
The one-minute version
Step through it yourself
The interactive version has eleven steps, including a slider for the one number that decides everything: how many tokens share each trip from memory to the chip.
The numbers for Qwen3.8 27B
- Memory. 27 billion weights at 4 bits is about 16 GB. It fits on a 32 GB card but not on a 16 GB one, because the conversation needs room too.
- Writing. Every token reads all 16 GB from memory. An H100 moves 3,350 GB/s, so at best about 209 tokens/s. A Mac Studio M3 Ultra manages about 51, a DGX Spark about 17.
- Reading. A 2,000-token prompt takes about 108 trillion operations. An H100 gets through that in about 0.1 seconds, an M3 Ultra in about 4.
These are ideal figures from published peak specs. Real software reaches roughly 50 to 80% of them. Compute for the Macs and the DGX Spark is estimated, since Apple doesn't publish it and NVIDIA quotes the Spark in 4-bit.