LO
Lena Okada
17 hours ago
Needle 2 on a Pi 5: tool calls fine until the schema got nested
Needle 2
An agentic language model for tool calling on tiny devices
Spent Friday night wiring Needle 2 into a little home-assistant bridge on a Pi 5.
The 14 MB binary is real — session RAM sat around 30 MB, and plain tool calls (turn light on/off, read a sensor) were solid. Prefill felt snappy for something this small.
Then I handed it a nested JSON schema for a thermostat scene (rooms → devices → constraints). It started emitting half-valid args with a missing required field about 1 in 4 times. Confidence scores helped me escalate those, but the happy path wasn't reliable enough to leave unsupervised overnight.
Anyone else hitting schema depth limits on this, or am I just bad at prompting a 45M model?
2 comments
Join the discussion
Log in to comment.
same failure mode on my desk. flat tools were fine, anything with nested objects and Needle started inventing keys that weren't in the schema.
i flattened the schema into 2–3 sequential calls and it got way more reliable. kinda defeats "one shot agent" though — are you keeping it local-only or escalating the messy ones?
Ran it in Docker on a Pi 5 overnight for a door-sensor webhook. Tool calls were fine until the case hit 72C and decode dropped from "snappy" to molasses.
Also: if your thermostat schema has optional nested arrays, strip them before the prompt. Needle seems to hallucinate empty arrays into garbage strings. I ended up with a tiny pre-validator that rejects low-confidence args before they hit HA.