left Activity Monitor open like an idiot. LM Studio said llama3.1:14b-instruct-q5_K_M would fit with num_ctx 16384.
first 40 prompts: fine. prompt 41: process just vanishes. no crash dialog. memory pressure still green.
now I pin ctx to 8192 and keep vm_stat in a terminal. 70B demos can stay on twitter.
Left ollama running qwen2.5 coder:32b overnight on a Mac mini while Cursor agent chewed through a refactors queue. Woke up to a PR that looked fine until review. Half the diffs were the kind of vague…
left ollama running an agent loop overnight on a 200 line auth refactor. woke up to Activity Monitor showing 36GB "compressed" and a dead terminal. num ctx was still on the default 128k. dumb. anyone…
Tried Deepseek Harness Desktop on Friday after the API kept rate limiting me mid refactor. Electron idle sat at 3.1GB on my M2 Air. Workflows actually ran, but one hung for 11 minutes on a missing DE…
5 comments
Join the discussion
Log in to comment.
sorry english not perfect — same on my side with Qwen2.5-14B-Instruct. default context windows are silent killers.
if you do not set
num_ctxyourself, first 30 prompts look like magic, then Metal OOM. I measure tokens/sec AFTER first crash now. blog numbers before that are fanfiction.yeah — sticky note on my monitor now:
num_ctx 4096for anything over 13b on this m2.i wasted a whole cape town night chasing "bad prompts" when the real killer was the default window. blog tok/s numbers after the first OOM don't count.
on a 4090 with vLLM this is less dramatic but still real. leave
max_model_lenat default, GPU hits 98%, one long prompt and the queue freezes forever.pin it or your "eval" is mostly waiting. Activity Monitor green means nothing.
same class of bug on my m2 air last friday. lm studio said the q5_K_M 8b would "fit comfortably" with 12k ctx.
prompt 28: beach ball. no crash report. just gone.
i now treat the green memory bar like a marketing screenshot. if
vm_statand activity monitor disagree, trust neither and cut ctx in half.the fit estimator is doing UI cosplay. it shows system RAM free and quietly ignores Metal / GPU pressure.
i started screenshotting the status bar next to Activity Monitor before every eval. if those two numbers diverge by more than ~2GB I refuse to click Load. not a vibe, just fewer silent exits.