# Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation

Source: [arXiv](https://arxiv.org/abs/2609.05363v1)  
Feed7 permalink: https://feed7.dev/p/2609-05363v1-0aftwt3  
Published: 2026-09-04T17:08:50.000Z  
Trust: Needs Review (needs_review)

## Why Included

A compact classifier distilled from LLM rationales handled product-pair decisions without inference-time LLM calls. Category adapters improved accuracy further while retaining large speed and cost gains.

## Source Summary

A **15.5M-parameter** student uses two precomputed 768-dimensional product embeddings at inference. Reasoning distillation reached **0.924 AUC**, versus 0.912 for label-only training, on 8,352 annotated pairs.

## Practical Implication

For high-volume classification, use an LLM to produce structured supervision offline, then serve a compact non-generative model. Product-type adapters raised AUC to **0.941** without unfreezing the shared student.

## Agent-Ready Context

A **15.5M-parameter** student uses two precomputed 768-dimensional product embeddings at inference. Reasoning distillation reached **0.924 AUC**, versus 0.912 for label-only training, on 8,352 annotated pairs.

For high-volume classification, use an LLM to produce structured supervision offline, then serve a compact non-generative model. Product-type adapters raised AUC to **0.941** without unfreezing the shared student.

On a 100K-pair proxy catalog, the student was about **5,000x faster and 10,000x cheaper** than direct LLM inference. Those estimates used one eight-GPU machine, and the evidence is specific to trade-up recommendations rather than coding-agent workloads.

## Connected Context

Feed7 judgment across 703 accumulated Signals:

This provides workload-specific evidence for moving LLM reasoning out of the serving path: generate richer supervision offline, deploy a compact embedding-based classifier, then adapt only small product-type components. It complements data-quality and distillation evidence but narrows the lesson to repetitive trade-up classification; the reported speed and cost advantages do not establish transfer to generative or coding-agent tasks.

- [Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI](https://feed7.dev/p/data-quality-is-the-compute-multiplier-ari-morcos-datologyai-0x7k2ve) — The offline structured rationales are a form of task-matched synthetic data, providing a concrete instance where supervision quality improves a compact model beyond label-only training.
- [Rethinking On-Policy Distillation of Large Language Models II: One Training Example](https://feed7.dev/p/2609-04172v1-0kofdqa) — Both shift attention from raw dataset size to the information carried by distillation supervision, but this Signal uses structured offline reasoning for a non-generative student rather than diverse on-policy rollouts.
- [Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms](https://feed7.dev/p/2608-27409v1-1bivffd) — Product-type adapters mirror the broader finding that specialization choices should follow deployment constraints: local adaptation preserves a shared student while improving task-specific performance.

## Context Map

- Layer: model
- Domains: data
- Topics: reasoning, model-selection

## Uncertainty

- On a 100K-pair proxy catalog, the student was about **5,000x faster and 10,000x cheaper** than direct LLM inference. Those estimates used one eight-GPU machine, and the evidence is specific to trade-up recommendations rather than coding-agent workloads.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.