Pulled qwen2.5-coder:32b-q4_K_M into Ollama on an M2 Max (64GB). Activity Monitor said ~41GB before the refactor even started.
Halfway through rewriting a messy FastAPI router it just… died. No stack trace, Metal panic, fan screaming. Dropped to 14b and it finished in 6 minutes with one wrong import.
Anyone actually shipping with 70B-class local models on Mac, or is that mostly blog cosplay?
Switched our Flutter chat stub from a generic OpenAI compatible endpoint to Groq (llama 3.3 70b) on Friday. First reply landed in 280ms. Cool. Also broke the loading UI — the CircularProgressIndicato…
Spent last week on Continue + Ollama ( qwen2.5 coder:14b ) for a Rust CLI refactor. Config clearly pointed at local. Half the turns still billed Anthropic until I noticed the usage graph. Free Contin…
Was wiring a filesystem MCP into Cursor so the agent could peek at configs. Asked it to "check the project layout". It ran what looked like a recursive ls and dumped 40k tokens of node modules paths…
5 comments
Join the discussion
Log in to comment.
same energy here. i stopped pretending 70b local was "the workflow" after the third fan death.
14b + tight file context beats a half-loaded giant that ghost-OOMs. the blog posts never show Activity Monitor.
14b + tight context is exactly where i landed after a week of pretending otherwise.
tried 32b q4 on a 64GB M3 Pro for a Flutter codegen loop — worked until Xcode indexing kicked in. then same ghost OOM. Activity Monitor never shows the Metal reservation honestly.
We tried the same on a studio Mac for an internal agent demo. Looked great until someone opened Chrome.
Now we run 14b local for autocomplete and pay ~$20/mo for cloud when the agent needs a long context window. Curious — did you pin
num_ctxor just let Ollama default? That alone killed us once.yes,
num_ctxwas the silent killer for us also. default felt fine until agent started dumping whole repo into prompt.we pin 8192 now for local, and if need longer we just call cloud. paying $20/mo is cheaper than buying another Mac.
Shipping? Kinda. We run
qwen2.5-coder:14bon an M2 Ultra for autocomplete and keep a spare Linux box with 2×4090 for anything over 32b.Mac 70b-class is cosplay for me too. The second Cursor + Slack open, Metal starts lying about free RAM. I set
OLLAMA_MAX_LOADED_MODELS=1and still get surprise kills mid-refactor.