vibehacker
Discuss
Finn
9 hours ago

ollama swapped my model mid-session and i only noticed from the latency

#rants#tooling#local-llm

was mid-refactor in Cursor last night with qwen2.5-14b via ollama. replies felt sharp for ~25 minutes, then answers got weirdly polite and slow.

checked the bar: still said qwen. ran ollama ps — it had quietly loaded tinyllama after a memory spike. no toast, no reconnect prompt.

i only caught it because token/sec dropped from ~38 to ~9. anyone pinning model ids in the client, or do you just watch ollama ps like a hawk?

5

5 comments

Join the discussion

Log in to comment.

Log in
  • Orin

    same trap friday. agent kept "fixing" a typescript error with tinyllama still loaded and i spent 40 minutes thinking my prompt was trash.

    i now pin OLLAMA_KEEP_ALIVE=-1 plus an exact model tag in the Cursor settings json. still not enough if ollama OOMs — it just picks whatever fits. do you get any log line when the swap happens, or is ollama ps really the only signal?

    • Owen

      KEEP_ALIVE=-1 helped until the M2 thermal-throttled and ollama still yeeted the 14b. same silent swap, no toast.

      i've been stuffing the full tag into the first system line every turn (MODEL: qwen2.5:14b-instruct-q4_K_M) and making the agent echo it before tools. if it says tinyllama i abort. ugly, but catches the mid-session flip.

  • Quiet Forge

    Pin the full tag (qwen2.5:14b-instruct-q4_K_M) in every agent config. Short aliases are how tinyllama sneaks in.

    I also run a tiny sidecar that polls ollama ps every 30s and pages me if the running digest != the expected one. Overkill for a solo box, quieter than debugging polite nonsense for half an hour.

    Silent fallback is worse than a hard crash. At least a crash makes you look.

    • Mason

      the sidecar idea is funny until you're on a friday deploy and the page goes off because chrome ate another 2GB.

      i just want ollama to hard-fail instead of quietly picking whatever fits. silent fallback is how my agent shipped a rewrite with a 1.1b that couldn't even keep the file path straight.

  • Sam Nguyen

    yeah the UI lying is the worst part. Activity Monitor still shows ~9GB for qwen while ollama ps already says tinyllama.

    i wrap agent calls with a ~200ms preflight that hits /api/tags and compares the digest. if it drifts i refuse to send the prompt. cheaper than a polite essay about sorting my laundry.

More like this

View all