# Measure Before You Train: Typed Decision Models vs. a Classical Baseline vs. a Generative LLM for Routing a Personal Multilingual Inbox

Source: [arXiv](https://arxiv.org/abs/2610.32955v1)  
Feed7 permalink: https://feed7.dev/p/2610-32955v1-18vy54w  
Published: 2026-10-23T17:49:43.000Z  
Trust: Needs Review (needs_review)

## Why Included

A single-user inbox study finds that about 200 human labels let logistic regression outperform tested LLMs on classification accuracy, yet it missed more actionable emails. Measure action costs, not accuracy alone.

## Source Summary

The study evaluates nine inbox-routing systems on **739 Italian and English emails** with thread-grouped cross-validation. With about **200 labels**, logistic regression led on classification accuracy, while missing 18% of emails needing action versus 4% for the best LLM; a cost-based noise threshold reduced its miss rate by half.

## Practical Implication

If you build a routing agent, create human-labeled ground truth and measure the outcomes of its composed actions. For sender relationships, the study found information from other messages by a known contact was more useful than prompting from one email.

## Agent-Ready Context

The study evaluates nine inbox-routing systems on **739 Italian and English emails** with thread-grouped cross-validation. With about **200 labels**, logistic regression led on classification accuracy, while missing 18% of emails needing action versus 4% for the best LLM; a cost-based noise threshold reduced its miss rate by half.

If you build a routing agent, create human-labeled ground truth and measure the outcomes of its composed actions. For sender relationships, the study found information from other messages by a known contact was more useful than prompting from one email.

This is a **single-user case study**; the emails are private and not released. The classifier also never predicted the two rarest classes, so its accuracy advantage does not mean it handles every class well.

## Connected Context

Feed7 judgment across 910 accumulated Signals:

This single-user inbox study shows why routing systems should be judged on action-relevant errors as well as accuracy: a conventional classifier led on accuracy, while an LLM missed fewer actionable emails, and a cost-based threshold reduced those misses. The private dataset and absent predictions for rare classes limit how far to generalize the result.

- [Cost-Aware Abstention for LLM-Based IT Ticket Classification](https://feed7.dev/p/2610-31657v1-0pyz12o) — Both studies make routing errors and review costs central to system choice; the ticket study’s cost-aware abstention provides a related way to manage uncertain decisions, while the inbox study compares different model families and action-miss outcomes.

## Context Map

- Layer: benchmark
- Domains: data, research
- Topics: agent-evals, agent-reliability

## Uncertainty

- This is a **single-user case study**; the emails are private and not released. The classifier also never predicted the two rarest classes, so its accuracy advantage does not mean it handles every class well.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.