# Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google

Source: [AI Engineer](https://www.youtube.com/watch?v=hacEQHHhu2Q)  
Feed7 permalink: https://feed7.dev/p/why-large-tiny-lms-agents-on-edge-robotics-cormac-brick-google-0fjif75  
Published: 2026-07-25T17:00:06.000Z  
Trust: Source Linked (source_linked)

## Why Included

Edge-agent design is often memory-bound: use small prompted models when devices can spare gigabytes, or fine-tune tiny models for narrow functions when reach and latency matter more.

## Source Summary

Google’s optimized **2B model uses 841 MB for weights**, but runtime and KV cache can push it toward **2 GB of active RAM**. A fine-tuned **270M-parameter** model reached **45 tokens/sec and over 86% function-call reliability** across ten mobile actions.

## Practical Implication

Start with the device’s memory and latency budget. If a small prompted model fits, use it; otherwise choose a task-specific tiny model, generate representative synthetic training data, and fine-tune it for a narrow function such as voice-to-action.

## Agent-Ready Context

Google’s optimized **2B model uses 841 MB for weights**, but runtime and KV cache can push it toward **2 GB of active RAM**. A fine-tuned **270M-parameter** model reached **45 tokens/sec and over 86% function-call reliability** across ten mobile actions.

Start with the device’s memory and latency budget. If a small prompted model fits, use it; otherwise choose a task-specific tiny model, generate representative synthetic training data, and fine-tune it for a narrow function such as voice-to-action.

Tiny models are not drop-in general agents. Their reliability depends on task-specific data and tuning, while visual workloads and Raspberry Pi interactions can still miss real-time expectations. The reported figures apply to the demonstrated configurations.

## Context Map

- Layer: model
- Domains: None
- Topics: model-selection, tool-use, open-models

## Uncertainty

- Tiny models are not drop-in general agents. Their reliability depends on task-specific data and tuning, while visual workloads and Raspberry Pi interactions can still miss real-time expectations. The reported figures apply to the demonstrated configurations.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.