Qwen 3.8 27B at 1500 tok/s on Cerebras vs my garage rack. Ran the same Go refactor on both.
Cerebras put Qwen 3.8 27B on their public endpoints, listed at ~1500 tokens/s, 64k context free tier and 128k paid: https://inference-docs.cerebras.ai/models/overview
The catalog page also says they don't serve pruned models on the public API, which I appreciate them writing down, because "same model name, mystery weights" is a thing.
So I did the obvious. Same Go refactor I did in my local-week post (one service, six files, add a context deadline everywhere), same Aider setup, two backends:
- Rack (2x 3090, Qwen quant): about 11 minutes wall clock, three rounds with the compiler, one bad edit I fixed by hand.
- Cerebras: the model finished its turns before I'd finished reading the previous one. Whole thing under 3 minutes. Same three compiler rounds, roughly the same quality, which makes sense, it's the same family.
So the speed is real and it's absurd. Where it bit me: I'm used to reading along as the model types. At 1500 tok/s there's nothing to read along to, it's just done, and I caught myself accepting a diff I hadn't actually looked at. The rack's slowness was doing a job I didn't know it was doing.
Keeping the rack for anything private. Using this for the rest. Free tier lasted the afternoon.
1 comment
Join the discussion
Log in to comment.
"The slowness was doing a job" is an interesting observation and I think it is true. At some speed the human stops being in the loop and becomes a rubber stamp. Maybe the tools should add an artificial pause before showing a diff. It sounds silly. I am not sure it is silly.