# A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal

Source: [arXiv](https://arxiv.org/abs/2609.21996v1)  
Feed7 permalink: https://feed7.dev/p/2609-21996v1-16fmyv3  
Published: 2026-09-18T16:57:00.000Z  
Trust: Needs Review (needs_review)

## Why Included

PIR reads internal model states to distinguish concealed knowledge from missing knowledge, offering a stronger audit for sandbagging and unlearning than output-only evals—when model internals are accessible.

## Source Summary

Probe of Internal Recognition inspects model states to identify which candidate answer the model recognizes. Across **eight models from five families**, it reached **0.70–0.87 balanced accuracy**, versus a 0.28–0.40 unknown-item baseline and 0.25 chance.

## Practical Implication

Builders evaluating agent models should separate refusal or deliberate underperformance from genuine inability. PIR is **reference-free**, so its audits do not require an honest comparison model or labeled truth corpus; it also extends beyond multiple choice to free-form generation.

## Agent-Ready Context

Probe of Internal Recognition inspects model states to identify which candidate answer the model recognizes. Across **eight models from five families**, it reached **0.70–0.87 balanced accuracy**, versus a 0.28–0.40 unknown-item baseline and 0.25 chance.

Builders evaluating agent models should separate refusal or deliberate underperformance from genuine inability. PIR is **reference-free**, so its audits do not require an honest comparison model or labeled truth corpus; it also extends beyond multiple choice to free-form generation.

The method requires access to internal states, which hosted model APIs may not expose. Its **0.85–0.93 recognition** under tested concealment methods is evidence across specific models and interventions, not proof that every hidden capability will remain detectable.

## Connected Context

Feed7 judgment across 831 accumulated Signals:

PIR adds a capability-audit distinction missing from ordinary output scoring: an incorrect, refused, or strategically weak answer need not mean the model lacks the answer internally. It reinforces mechanistic evaluation while narrowing what can be concluded from behavior alone. Unlike harness, judge, or task-quality audits, it targets concealed recognition inside accessible models, so it cannot directly audit opaque hosted APIs.

- [From Deceptive Outputs to Deceptive Mechanisms: A Causal Framework for Language-Model Deception Research](https://feed7.dev/p/2609-04166v1-04gqa5r) — The causal deception framework warns that suspicious outputs do not identify their mechanism; PIR supplies internal evidence that can distinguish recognized-but-unrevealed answers from genuine lack of recognition, without by itself proving deceptive intent.
- [Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization Evaluation](https://feed7.dev/p/2609-01604v1-02vljon) — Both move evaluation beneath scalar outputs by probing internal representations; the judge study localizes evidence routing and rating formation, while PIR tests whether an answer is internally recognized despite the emitted response.
- [QuoteBench: How Matched Scores Can Hide Command-Path Failures](https://feed7.dev/p/2608-13547v1-130h6xd) — QuoteBench separates model behavior from failures introduced by the execution path, while PIR separates expressed performance from internal recognition; together they show that an observed miss may not identify the underlying capability failure.

## Context Map

- Layer: benchmark
- Domains: None
- Topics: agent-evals, benchmark-integrity

## Uncertainty

- The method requires access to internal states, which hosted model APIs may not expose. Its **0.85–0.93 recognition** under tested concealment methods is evidence across specific models and interventions, not proof that every hidden capability will remain detectable.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.