# onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

Source: [arXiv](https://arxiv.org/abs/2609.24983v1)  
Feed7 permalink: https://feed7.dev/p/2609-24983v1-0wyopxo  
Published: 2026-09-21T17:56:53.000Z  
Trust: Needs Review (needs_review)

## Why Included

onPanda turns the first bad token into an annotation boundary, regenerating from each correction to create on-policy supervision with less manual rewriting.

## Source Summary

onPanda asks an annotator to find the **first inappropriate token**, replace it from candidates or free-form text, discard the remaining output and regenerate from the corrected prefix. A small controlled study reports **52% lower median annotation time** than manual post-editing.

## Practical Implication

For agent tuning, capture corrections at the moment a trajectory first diverges instead of rewriting the final answer wholesale. The resulting positions and paired alternatives can support on-policy SFT and preference data, including tool-connected trajectories.

## Agent-Ready Context

onPanda asks an annotator to find the **first inappropriate token**, replace it from candidates or free-form text, discard the remaining output and regenerate from the corrected prefix. A small controlled study reports **52% lower median annotation time** than manual post-editing.

For agent tuning, capture corrections at the moment a trajectory first diverges instead of rewriting the final answer wholesale. The resulting positions and paired alternatives can support on-policy SFT and preference data, including tool-connected trajectories.

The time result comes from a small study, and the supplied material gives no breakdown by task or annotator. Whether the workflow scales to long coding trajectories without adding review overhead remains open.

## Connected Context

Feed7 judgment across 843 accumulated Signals:

This turns the prior call to mine real agent trajectories into a specific annotation primitive: intervene at the first bad token and preserve the corrected prefix as on-policy training evidence. It may make expert review cheaper than full rewrites, but the small timing study does not establish that first-token correction remains efficient or sufficient for long, branching tool-use failures.

- [Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain](https://feed7.dev/p/improving-agents-is-a-data-mining-problem-vivek-trivedy-langchain-1cgibr8) — Trace mining identifies recurring production failures; onPanda supplies a concrete way to annotate the first divergence in those traces for subsequent training.
- [State of Data — Sean Cai, Independent / State of Data](https://feed7.dev/p/state-of-data-sean-cai-independent-state-of-data-0v9fy69) — It operationalizes the preference for real workflow trajectories by extracting correction data from generated paths rather than relying only on manufactured tasks or saved final answers.
- [Trading Desks to Clinical Trials: Parallels in Applied Vertical AI — Ayush Bhardwaj, Allos AI](https://feed7.dev/p/trading-desks-to-clinical-trials-parallels-in-applied-vertical-ai-ayush-1hwvsg3) — Token-level intervention could reduce the cost of expert labeling in vertical workflows, but it does not remove the candidate’s requirement for domain experts when usefulness lacks an objective answer key.
- [TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development](https://feed7.dev/p/2608-26086v1-0k35b2d) — TraceML’s long-horizon planning failures suggest an important boundary: correcting the first inappropriate token may capture a local divergence without representing broader deficiencies in effort allocation or plan revision.

## Context Map

- Layer: benchmark
- Domains: data
- Topics: agent-evals, harness-engineering

## Uncertainty

- The time result comes from a small study, and the supplied material gives no breakdown by task or annotator. Whether the workflow scales to long coding trajectories without adding review overhead remains open.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.