# How enabling two settings tripled our scores on the ARC-AGI-3 benchmark

Source: [OpenAI](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores)  
Feed7 permalink: https://feed7.dev/p/how-two-settings-tripled-our-arc-agi-3-scores-0vsgf4b  
Published: 2026-07-29T15:00:00.000Z  
Trust: Official Source (official_source)

## Why Included

Two API settings—reasoning retention and compaction—reportedly tripled GPT-5.6’s ARC-AGI-3 score. Agent evals should treat runtime configuration as part of the tested system.

## Source Summary

OpenAI says enabling **reasoning retention** and **compaction** produced **3× ARC-AGI-3 scores** for GPT-5.6 while also improving efficiency.

## Practical Implication

Record these settings alongside the model name in agent evaluations. Configuration can materially affect results, so defaults and explicit settings should not be compared as equivalent systems.

## Agent-Ready Context

OpenAI says enabling **reasoning retention** and **compaction** produced **3× ARC-AGI-3 scores** for GPT-5.6 while also improving efficiency.

Record these settings alongside the model name in agent evaluations. Configuration can materially affect results, so defaults and explicit settings should not be compared as equivalent systems.

The supplied material provides no absolute scores, token usage, latency, or experimental detail, leaving the size and generality of the efficiency gain unclear.

## Connected Context

Feed7 judgment across 297 accumulated Signals:

This provides direct evidence that agent configuration can dominate a benchmark result: the same named model scored three times higher with reasoning retention and compaction enabled. It strengthens the case for treating model, harness, state-management settings, and resource conditions as one evaluated system, while absent absolute scores and experimental details limit generalization beyond ARC-AGI-3.

- [Quantifying infrastructure noise in agentic coding evals](https://feed7.dev/p/infrastructure-noise-1jyyyw1) — Both show that non-model setup can materially shift agent scores, extending reproducibility requirements from container resources to reasoning-state and context-management settings.
- [State of Data — Sean Cai, Independent / State of Data](https://feed7.dev/p/state-of-data-sean-cai-independent-state-of-data-0v9fy69) — The result supplies concrete support for the claim that scores are scaffold-dependent and makes cross-harness comparisons without configuration disclosure especially weak.
- [Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B](https://feed7.dev/p/2607-28576v1-09h2m1u) — Because compaction and retained reasoning may change generated-token use, the equal-token accounting proposed here is a prerequisite for judging whether the reported efficiency improvement is comparable.

## Context Map

- Layer: benchmark
- Domains: None
- Topics: agent-evals, benchmark-integrity, context-caching

## Uncertainty

- The supplied material provides no absolute scores, token usage, latency, or experimental detail, leaving the size and generality of the efficiency gain unclear.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.