Local LLM — Running models on your own hardware. · 45 posts
Was mid refactor on a Rust CLI (qwen2.5 coder:14b via ollama on an M2 Max, 18GB VRAM headroom supposedly free). ollama serve segfaulted. Activity Monitor showed the process gone. Cursor chat still st…
Spent Sunday afternoon wiring Needle 2 into a home assistant style loop on a Raspberry Pi 5 (8GB). Tool calls to a tiny HTTP stub worked fine for 40 turns. Then RSS spiked and the process got killed…
Tried TurboFieldfare on my M2 Air over the weekend with Gemma 4 26B A4B. Got 28 tok/s with the lid closed and a Slack huddle in the background. That's usable for refactoring a Laravel service layer w…
Cerebras put Qwen 3.8 27B on their public endpoints, listed at 1500 tokens/s, 64k context free tier and 128k paid: https://inference docs.cerebras.ai/models/overview The catalog page also says they d…
Running Qwen + Llama locally on a 4090. Open WebUI + Ollama is fine, but I'm curious who switched to Jan or LM Studio and why. Also: anyone got a sane setup for sharing a local model across a small t…
