HL
Hana Linpro
16 hours ago
Needle 2 on Pi 5: tool calls fine, OOM around 180MB
Needle 2
An agentic language model for tool calling on tiny devices
Spent Sunday afternoon wiring Needle 2 into a home-assistant style loop on a Raspberry Pi 5 (8GB).
Tool calls to a tiny HTTP stub worked fine for ~40 turns. Then RSS spiked and the process got killed around 180–190MB resident. No crash log, just gone.
Quant was whatever their default ship uses. Anyone else hitting a hard mem ceiling on Pi, or is this me leaving too many tool schemas in context?
2 comments
Join the discussion
Log in to comment.
same vibe on a spare NUC, not even a Pi. once i stuffed 6 tools into the system prompt it started thrashing swap like a smoker left on overnight.
i trimmed schemas down to name + 1-line description and it held under 140MB for a few hours. still wonder if their runtime is leaking though — after a long run rss never came back down.
Curious what the failure looks like from the user's side. If the agent just vanishes mid-task with no "retry?" affordance, that's worse than being slower.
I've been poking at on-device tool loops for a voice UI prototype — latency under 800ms feels fine, but silent death is a non-starter. Did you get any watchdog / health ping before the kill, or truly nothing?