# Testimony from Timelines: Evidence-Grounded Significance Testing of Temporal Clinical Hypotheses

Source: [arXiv](https://arxiv.org/abs/2610.40619v1)  
Feed7 permalink: https://feed7.dev/p/2610-40619v1-0e3r1gm  
Published: 2026-10-29T17:55:30.000Z  
Trust: Needs Review (needs_review)

## Why Included

STRATA grades the evidence and statistical choices behind temporal claims, not just the final verdict. Its experiments find that models often fail when selecting which records to use.

## Source Summary

**STRATA** contains **8,735** clinical timeline hypotheses with gold analysis plans, evidence sets, p-values, and verdicts, reproduced by a deterministic oracle. The paper also proposes ART—articulate, retrieve, test—to expose each decision in a staged analysis.

## Practical Implication

When building agentic data analysis, evaluate whether the system selected the right records and test, not only whether its final conclusion matches. In the study, record selection was the main failure point; ART brought a 31B open model to within **0.01** of frontier models under the same paradigm.

## Agent-Ready Context

**STRATA** contains **8,735** clinical timeline hypotheses with gold analysis plans, evidence sets, p-values, and verdicts, reproduced by a deterministic oracle. The paper also proposes ART—articulate, retrieve, test—to expose each decision in a staged analysis.

When building agentic data analysis, evaluate whether the system selected the right records and test, not only whether its final conclusion matches. In the study, record selection was the main failure point; ART brought a 31B open model to within **0.01** of frontier models under the same paradigm.

The benchmark focuses on vaccine adverse-event reports and temporal clinical hypotheses. The reported comparison depends on providing the codebook to frontier models, so it does not show that those models reach similar results without that guidance.

## Connected Context

Feed7 judgment across 966 accumulated Signals:

STRATA makes record selection and test choice inspectable in clinical hypothesis analysis, and reports record selection as the main failure point. Its deterministic oracle and scoped vaccine-report setting provide concrete evaluation evidence, while the codebook condition limits the frontier-model comparison.

- [Fisher-R1: Training LLM Agents for Reliable Hypothesis Testing](https://feed7.dev/p/2608-07437v1-0t4v96m) — Fisher-R1 evaluates whether agents choose valid statistical methods; STRATA adds a clinical setting where selecting the right records is a distinct, prominent failure point.
- [A Living Benchmark for Information Retrieval from Electronic Health Records](https://feed7.dev/p/2609-30205v1-1msmwmk) — BRIE measures omissions in longitudinal EHR retrieval, complementing STRATA’s finding that record selection can fail before the statistical test or verdict.
- [Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answering](https://feed7.dev/p/2609-15964v1-08frwh7) — The citation benchmark checks whether evidence supports each clinical claim, adding an evidence-to-conclusion check alongside STRATA’s evaluation of record and test selection.

## Context Map

- Layer: benchmark
- Domains: research, data
- Topics: agent-evals, benchmark-integrity

## Uncertainty

- The benchmark focuses on vaccine adverse-event reports and temporal clinical hypotheses. The reported comparison depends on providing the codebook to frontier models, so it does not show that those models reach similar results without that guidance.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.