Not All Reasoning Generalizes: A Case Study on Learning Generalizable Reasoning Strategies in RLVR
In a controlled RLVR study, models learned structured reasoning in-distribution but switched to ineffective brute force under harder shifts. Co-training with inversion tasks restored some generalization.
In an equation-solving setup, RLVR models learned step-by-step derivations on familiar tasks but often abandoned them under **out-of-distribution shifts**, such as longer outputs or more solutions. Co-training with bijective inversion tasks helped restore the strategy and accuracy.
When evaluating reasoning agents, test whether they keep using the intended strategy as task conditions change. In this study, outcome-only rewards could shape reasoning without explicit process supervision.
In an equation-solving setup, RLVR models learned step-by-step derivations on familiar tasks but often abandoned them under **out-of-distribution shifts**, such as longer outputs or more solutions. Co-training with bijective inversion tasks helped restore the strategy and accuracy. When evaluating reasoning agents, test whether they keep using the intended strategy as task conditions change. In this study, outcome-only rewards could shape reasoning without explicit process supervision. The results come from a controllable equation-solving framework. The supplied material does not establish whether the intervention transfers to other task types or real-world coding agents.
This adds a robustness dimension to reasoning-agent evaluation: success on familiar tasks does not show that a model keeps using its intended strategy when output length or the number of solutions changes. Skill²-Bench complements this by measuring failures when reasoning skills must switch across dependent steps; the study’s equation-solving setup leaves transfer to coding agents unestablished.