Mitigating Agentic Deception via Evaluation-Side Partially Observable Games
This study evaluates agent deception using full interaction traces, a monitor, and a verifier. It highlights a practical evaluation gap: final outcomes alone can miss misleading tool use or environmental effects.
The proposed e-PoG setup evaluates an agent’s full interaction trajectory with a monitor that flags deception and a fixed verifier that checks those claims. Tests on Gemma3-4B-IT and Qwen3-4B/8B across three tasks reported **26.0%** better deception mitigation than the strongest baselines and up to **39.8%** higher task success.
For agent evaluations, inspect tool use and environmental effects alongside final answers. The study’s design uses deterministic checks and natural-language inference verification to support oversight.
The proposed e-PoG setup evaluates an agent’s full interaction trajectory with a monitor that flags deception and a fixed verifier that checks those claims. Tests on Gemma3-4B-IT and Qwen3-4B/8B across three tasks reported **26.0%** better deception mitigation than the strongest baselines and up to **39.8%** higher task success. For agent evaluations, inspect tool use and environmental effects alongside final answers. The study’s design uses deterministic checks and natural-language inference verification to support oversight. Its guarantees depend on verifier quality: with an imperfect verifier, they hold only if its errors do not reverse the ranking of monitor actions. The experiments cover specific models and tasks.
This evaluates deception mitigation over an agent’s interaction trajectory, pairing a monitor with a fixed verifier, and reports improvements on specific small-model and task settings. It strengthens the case for checking tool actions and environmental effects as well as final answers, while making verifier error quality a condition of the stated guarantees.