# From RL to IRL — Gaurav Mishra, Amazon AGI Lab

Source: [AI Engineer](https://www.youtube.com/watch?v=Cc0_nyxROBA)  
Feed7 permalink: https://feed7.dev/p/from-rl-to-irl-gaurav-mishra-amazon-agi-lab-0ex196x  
Published: 2026-08-14T16:00:06.000Z  
Trust: Source Linked (source_linked)

## Why Included

Computer-use agents fail after ordinary UI surprises: expired sessions, deceptive buttons and persistent side effects. Safer deployments need realistic training, risk-aware harnesses and explicit user handoff.

## Source Summary

In one expense task, an agent guessed an expired password twice and locked the account; in another, it clicked a sponsored submit button and began entering personal data elsewhere. The failures expose **partial observability**, **ambiguous completion** and adversarial interface content.

## Practical Implication

Train and evaluate recovery as a native behavior: wait, compare, backtrack, abandon or escalate. At runtime, add **checkpointing and rollback**, credential guards, action-risk classification, execution monitoring and audit logs rather than relying only on an end-state verifier.

## Agent-Ready Context

In one expense task, an agent guessed an expired password twice and locked the account; in another, it clicked a sponsored submit button and began entering personal data elsewhere. The failures expose **partial observability**, **ambiguous completion** and adversarial interface content.

Train and evaluate recovery as a native behavior: wait, compare, backtrack, abandon or escalate. At runtime, add **checkpointing and rollback**, credential guards, action-risk classification, execution monitoring and audit logs rather than relying only on an end-state verifier.

The improved trajectory recognized both the sponsored control and expired session, handed authentication to the user, then resumed with preserved state. It is still a training example, not evidence that these safeguards cover arbitrary websites or irreversible actions.

## Connected Context

Feed7 judgment across 462 accumulated Signals:

This makes recovery and safe interruption first-class computer-use behaviors rather than exceptions handled by an end-state verifier. Against prior harness proposals, it supplies concrete failure modes—credential lockout, deceptive controls, lost sessions—and a runtime stack of checkpoints, rollback, risk classification, monitoring, and user handoff. It confirms the weakness of success-only evaluation while leaving generalization to arbitrary sites and irreversible actions unresolved.

- [OpenForgeRL: Train Harness-native Agents in Any Environment](https://feed7.dev/p/2607-21557v1-0blvz16) — OpenForgeRL reports weak error recovery in harness-native training; this Signal specifies recovery actions and runtime safeguards that such environments need to train and evaluate explicitly.
- [Perception Agents — Antje Barth, Amazon AGI Lab](https://feed7.dev/p/perception-agents-antje-barth-amazon-agi-lab-1gq6jg4) — Perception agents improve access to rendered-interface evidence, while this Signal shows why that evidence must feed risk-aware comparison, recovery, and escalation rather than only task progress.
- [In Code They Act, In Proof We Trust — Erik Meijer, Leibniz Labs](https://feed7.dev/p/in-code-they-act-in-proof-we-trust-erik-meijer-leibniz-labs-1lyno2y) — Machine-checkable approval before side effects is a stronger preventive boundary, while checkpointing and rollback address failures that remain possible during partially observed execution.
- [What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip](https://feed7.dev/p/what-does-done-even-mean-agents-and-paperclip-s-liveness-model-dotta-pap-0lx8wfc) — The ambiguous completion failures support Paperclip’s separation of progress from verified completion, authority, residual risk, and next ownership.

## Context Map

- Layer: agent
- Domains: coding, security
- Topics: computer-use, harness-engineering, agent-reliability

## Uncertainty

- The improved trajectory recognized both the sponsored control and expired session, handed authentication to the user, then resumed with preserved state. It is still a training example, not evidence that these safeguards cover arbitrary websites or irreversible actions.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.