Woke up to qwen2.5:14b-instruct-q4_K_M answering like it had amnesia.
Turns out yesterday's Ollama bump wiped the Modelfile override. Activity Monitor still showed ~18GB resident so I kept yelling at the system prompt. It was just 8k context the whole time.
Anyone pinning PARAMETER num_ctx in a Modelfile that survives upgrades, or do you just re-apply it like a peasant every Monday?
spent friday night trying to rename a package with mlx community/Qwen2.5 32B Instruct 4bit on an M2 Max 64GB. got through the import graph, wrote a half finished pyproject.toml, then metal panic and…
Tried the new qwen2.5:14b instruct q4 K M in LM Studio on the Mac Studio last night. UI said "fits in 64GB" so I loaded it with a 32k context for a RAG eval. Ten minutes in, fans go nuclear, swap hit…
Sorry English. I pipe llama 3.3 70b through Groq for agent loops because latency look nice on demos. Yesterday: 900 tok/s for first 4 calls, then hard 429 for 40 minutes while I still had half a Type…
5 comments
Join the discussion
Log in to comment.
I pin it in a tiny
Modelfileand also set it in the OpenAI-compat client (num_ctx/options.num_ctx). Ollama upgrades have burned me twice — once on a meeting-booker agent that started summarizing mid-sentence because the history got truncated.Also: Mac Activity Monitor lying about resident memory is a classic. I only trust
ollama psnow.same. i keep num_ctx in a Modelfile checked into the repo and a mise task that runs
ollama create …before any long agent loop.activity monitor is useless for this. does anyone actually get ollama to keep custom models across major bumps without a recreate?
Same trap. I keep a one-liner in the repo:
ollama create my-qwen -f Modelfileafter every bump, then screenshotollama showbefore I trust a long RAG pass.The "model loaded = context is fine" vibe is how you ship a UI that forgets half the brief. Cleanup time: about 40 minutes of me redrawing copy the model truncated. Never again without the show command.
the
ollama showscreenshot habit is underrated. i burned a saturday (and another $20 on cloud fallback tokens) because the local model was truncating tool args mid-json.now it's on the friday ship checklist next to "did the webhook secret leak into the PR again"
Peasant mode here. I pin
PARAMETER num_ctx 32768in the Modelfile and a shell alias that diffsollama show my-qwen --modelfileagainst a checked-in copy after every brew upgrade.Yesterday's bump still dropped me to 8k once because I renamed the model tag.
ollama pssaid loaded; the model just forgot half the tool schema. Blamed Zod for 25 minutes before I ran show.