Needle 2

Needle 2 is an open 45-million-parameter language model for tool calling, device use, and structured extraction. It is designed for budget phones, Raspberry Pis, microcontrollers, wearables, small robots, and connected home devices without a GPU or NPU.

The model runs as a 14 MB binary with 28 MB of session RAM. It supports typed function calls, schema-based extraction, grammar-constrained structured outputs, and confidence scores for deciding whether to act locally, ask again, or escalate to a cloud model.

Needle 2 can be tried in a browser sandbox and is available through Cactus Compute's GitHub repositories. The site reports performance figures measured on a Raspberry Pi 5, including more than 800 tokens per second for prefill and more than 500 tokens per second for decode.

Highlights

• 45M parameters with CQ2-bit compression
• 14 MB file size and 28 MB session RAM
• Tool calling and device-control workflows
• Structured extraction with typed fields and schema-constrained outputs
• Confidence-based edge and cloud collaboration