cactus-compute/needle
Needle 3 runs tool calling, typed extraction, and embeddings locally in an 8–29 MB binary, giving agent builders an offline option for constrained devices.
Needle 3 packages tool calling, schema-constrained extraction, and text embeddings into a **2-bit, 8–29 MB binary**. Its 121M-parameter ladder supports deployable subnetworks from **2 to 20 layers**, with prebuilt engines under 1 MB.
For device-side agents, test whether its exact tool selection, typed outputs, and confidence score can replace a network round trip. Fine-tune a smaller subnetwork around a narrow tool set, and use the supplied runtime or C API for deployment.
Needle 3 packages tool calling, schema-constrained extraction, and text embeddings into a **2-bit, 8–29 MB binary**. Its 121M-parameter ladder supports deployable subnetworks from **2 to 20 layers**, with prebuilt engines under 1 MB. For device-side agents, test whether its exact tool selection, typed outputs, and confidence score can replace a network round trip. Fine-tune a smaller subnetwork around a narrow tool set, and use the supplied runtime or C API for deployment. The model deliberately trades away general chat ability, while the published task results use exact-match accuracy and field micro-F1. Local training exports at **4 bits**; the shipped 2-bit quantization uses Cactus infrastructure, and binary telemetry is on by default unless disabled.
Needle turns the edge-model argument into a deployable specialist: tool choice, typed extraction, and embeddings can fit in megabytes when general conversation is deliberately excluded. It reinforces fine-tuning tiny models for narrow device functions, while its confidence output overlaps Jev’s typed control-flow role. The remaining decision is operational: validate exact-task accuracy and quantization behavior before replacing a network call.