# Perception Agents — Antje Barth, Amazon AGI Lab

Source: [AI Engineer](https://www.youtube.com/watch?v=2JX6JYyQG4Y)  
Feed7 permalink: https://feed7.dev/p/perception-agents-antje-barth-amazon-agi-lab-1gq6jg4  
Published: 2026-07-23T16:00:06.000Z  
Trust: Source Linked (source_linked)

## Why Included

Perception agents share the rendered interface with users, turning visual annotations and design rules into agent input and checks. The open-source harness is an early attempt to close computer-use feedback loops.

## Source Summary

Amazon AGI Lab released **two open-source harness pieces**: annotation captures marked screen regions and style details, while verification checks rendered output against design rules and walks user flows.

## Practical Implication

For UI-building agents, give visual intent as shared context instead of translating every change into prose. Add **rendered-screen verification** after implementation so the agent evaluates what users see, not only underlying code or tool responses.

## Agent-Ready Context

Amazon AGI Lab released **two open-source harness pieces**: annotation captures marked screen regions and style details, while verification checks rendered output against design rules and walks user flows.

For UI-building agents, give visual intent as shared context instead of translating every change into prose. Add **rendered-screen verification** after implementation so the agent evaluates what users see, not only underlying code or tool responses.

These components are described as a first step, not a complete perception agent. Visual and flow checks still produce a report for human judgment, and the broader problem of reliable end-to-end work across applications remains open.

## Context Map

- Layer: agent
- Domains: coding, image
- Topics: computer-use, tool-use, agent-reliability

## Uncertainty

- These components are described as a first step, not a complete perception agent. Visual and flow checks still produce a report for human judgment, and the broader problem of reliable end-to-end work across applications remains open.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.