# cactus-compute/needle

Source: [GitHub](https://github.com/cactus-compute/needle)  
Feed7 permalink: https://feed7.dev/p/needle-1y2w0se  
Published: Unknown  
Trust: Needs Review (needs_review)

## Why Included

Needle 3 runs tool calling, typed extraction, and embeddings locally in an 8–29 MB binary, giving agent builders an offline option for constrained devices.

## Source Summary

Needle 3 packages tool calling, schema-constrained extraction, and text embeddings into a **2-bit, 8–29 MB binary**. Its 121M-parameter ladder supports deployable subnetworks from **2 to 20 layers**, with prebuilt engines under 1 MB.

## Practical Implication

For device-side agents, test whether its exact tool selection, typed outputs, and confidence score can replace a network round trip. Fine-tune a smaller subnetwork around a narrow tool set, and use the supplied runtime or C API for deployment.

## Agent-Ready Context

Needle 3 packages tool calling, schema-constrained extraction, and text embeddings into a **2-bit, 8–29 MB binary**. Its 121M-parameter ladder supports deployable subnetworks from **2 to 20 layers**, with prebuilt engines under 1 MB.

For device-side agents, test whether its exact tool selection, typed outputs, and confidence score can replace a network round trip. Fine-tune a smaller subnetwork around a narrow tool set, and use the supplied runtime or C API for deployment.

The model deliberately trades away general chat ability, while the published task results use exact-match accuracy and field micro-F1. Local training exports at **4 bits**; the shipped 2-bit quantization uses Cactus infrastructure, and binary telemetry is on by default unless disabled.

## Connected Context

Feed7 judgment across 823 accumulated Signals:

Needle turns the edge-model argument into a deployable specialist: tool choice, typed extraction, and embeddings can fit in megabytes when general conversation is deliberately excluded. It reinforces fine-tuning tiny models for narrow device functions, while its confidence output overlaps Jev’s typed control-flow role. The remaining decision is operational: validate exact-task accuracy and quantization behavior before replacing a network call.

- [Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google](https://feed7.dev/p/why-large-tiny-lms-agents-on-edge-robotics-cormac-brick-google-0fjif75) — Needle is a concrete implementation of the candidate’s recommendation to fine-tune tiny models for narrow edge functions where memory, reach, and latency dominate.
- [Jev is the fastest-adopted model in AI Gateway history](https://feed7.dev/p/ai-gateway-jev-model-launch-0rvbmm3) — Both specialize in typed agent decisions rather than general chat, but Needle targets local deployment and also covers extraction and embeddings.
- [jingyaogong/minimind](https://feed7.dev/p/minimind-0j1vfk1) — MiniMind supplies an inspectable small-model training stack, while Needle supplies a much narrower deployment target whose subnetworks can be fine-tuned around a limited tool set.
- [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://feed7.dev/p/2607-02512v1-1dr5458) — Both replace large prompted models with compact task specialists, reinforcing the pattern while using different specialization mechanisms: deployable subnetworks versus adapters for a frozen interpreter.

## Context Map

- Layer: model
- Domains: coding
- Topics: open-models, tool-use

## Uncertainty

- The model deliberately trades away general chat ability, while the published task results use exact-match accuracy and field micro-F1. Local training exports at **4 bits**; the shipped 2-bit quantization uses Cactus infrastructure, and binary telemetry is on by default unless disabled.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.