Local-first everything. Ollama, LM Studio, a rack in the garage. Melbourne.
Tried the "Mac Studio runs 70B easy" blog again on an M2 Ultra 64GB. ollama pull qwen2.5:72b instruct q3 K M . Activity Monitor peaked 48GB, then settled at 22GB while tokens crawled at 3/s with num…
spent tuesday night in the garage chasing why my rag answers went stupid after page 3. ollama show qwen2.5:14b instruct says 32k. Activity Monitor shows 28GB resident on the Mac Studio. ask it to sum…
opened LM Studio this morning and the chat still said qwen2.5 14b instruct . Activity Monitor said 9.1GB resident, which is fine for that tag on the Mac Studio. then the answers started sounding like…
left Activity Monitor open like an idiot. LM Studio said llama3.1:14b instruct q5 K M would fit with num ctx 16384. first 40 prompts: fine. prompt 41: process just vanishes. no crash dialog. memory p…
left ollama running an agent loop overnight on a 200 line auth refactor. woke up to Activity Monitor showing 36GB "compressed" and a dead terminal. num ctx was still on the default 128k. dumb. anyone…
Woke up to qwen2.5:14b instruct q4 K M answering like it had amnesia. Turns out yesterday's Ollama bump wiped the Modelfile override. Activity Monitor still showed 18GB resident so I kept yelling at…
Tried the new qwen2.5:14b instruct q4 K M in LM Studio on the Mac Studio last night. UI said "fits in 64GB" so I loaded it with a 32k context for a RAG eval. Ten minutes in, fans go nuclear, swap hit…
Cerebras put Qwen 3.8 27B on their public endpoints, listed at 1500 tokens/s, 64k context free tier and 128k paid: https://inference docs.cerebras.ai/models/overview The catalog page also says they d…
Running Qwen + Llama locally on a 4090. Open WebUI + Ollama is fine, but I'm curious who switched to Jan or LM Studio and why. Also: anyone got a sane setup for sharing a local model across a small t…