ran qwen2.5:14b-instruct-q4_K_M for a small customer demo yesterday. model kept cutting mid-sentence and inventing function args that were already in the prompt.
i rewrote the system prompt three times. then notice Ollama Modelfile still has PARAMETER num_ctx 2048. my docs were ~6k tokens. of course output was garbage.
sorry for english. pin your context. default will quietly lie to you.
running mistral small on an M2 Air for a cheap eval loop. hit 4k tokens into a batch and the whole thing just... vanished. no crash log. Activity Monitor still shows ollama at 0% CPU hugging the port…
spent tuesday night thinking my RAG eval was broken. i set PARAMETER num ctx 32768 in the Modelfile for qwen2.5:14b instruct q4 K M . ollama show printed 32768. then every long prompt still cut aroun…
opened LM Studio this morning and the chat still said qwen2.5 14b instruct . Activity Monitor said 9.1GB resident, which is fine for that tag on the Mac Studio. then the answers started sounding like…
3 comments
Join the discussion
Log in to comment.
mate same energy. windsurf cascade "fixed" my ollama setup by rewriting the Modelfile back to 2048 while i was in another tab. thought i was going mad. now i just
catthe Modelfile before every demo like a superstitionIf the UI does not show active context length next to the model tag, you will keep debugging the wrong layer. Print it. Or your "CI is green" demo will keep dying for no reason.
yeah the 2048 default bites everyone. on Mac Studio i
ollama show qwen2.5:14b --modelfile | rg num_ctxbefore any customer call now.also check
ollama ps— half the time you still have the old tag loaded with the tiny context and the UI is lying about which one is active.