DecisionRank: On-Policy Distillation for Decision-Model Rerankers
DecisionRank transfers a generative ranker's preferences into a score-based reranker through on-policy distillation. The paper reports competitive retrieval results with substantially faster ranking at 8B parameters.
DecisionRank sorts a decision model's document scores into a ranking, then uses a generative teacher's conditional feedback to supervise those scores. The paper reports that **1.7B** DecisionRank beats its **27B** teacher on TREC DL19–23; its 8B model runs **24.5×** faster on TREC DL and **16.7×** faster on BRIGHT than a generative listwise counterpart.
Builders using retrieval in coding agents could evaluate a score-based reranker when ranking latency matters. The method combines item-space on-policy distillation with offline ranking supervision, and the paper reports results on TREC DL, BEIR and BRIGHT.
DecisionRank sorts a decision model's document scores into a ranking, then uses a generative teacher's conditional feedback to supervise those scores. The paper reports that **1.7B** DecisionRank beats its **27B** teacher on TREC DL19–23; its 8B model runs **24.5×** faster on TREC DL and **16.7×** faster on BRIGHT than a generative listwise counterpart. Builders using retrieval in coding agents could evaluate a score-based reranker when ranking latency matters. The method combines item-space on-policy distillation with offline ranking supervision, and the paper reports results on TREC DL, BEIR and BRIGHT. These are benchmark results, not evidence about end-to-end agent quality or latency in a specific retrieval stack. The speed comparisons apply to the named benchmarks and generative listwise counterpart.
DecisionRank offers a fast score-based reranker trained from generative feedback, giving retrieval systems a way to reduce ranking latency without replacing their candidate-generation stage. DecisionRAG explores the opposite architectural move by navigating a corpus without a first-stage retriever. The benchmark results motivate comparing these routes, but do not establish which improves an end-to-end agent.