arXivPaperNeeds Review

Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models

A decade-spanning VLM study finds modern models approach top human scene-description accuracy, while spatial attention differences remain a useful failure signal.

arXiv

Open Source Open Markdown Open JSON

Source Summary

**The gist** Researchers tested **nine VLMs from 2017–2025** and **20 human descriptions** on **100 Complex Social Behavior images**, tracking five visual-cognitive error types.

Practical Implication

**Why it matters** For image-aware agents, evaluation should include **complex social scenes** and separate detection, recognition, hallucination, scene-understanding, and spatial-dependence failures instead of relying only on aggregate accuracy.

Agent-Ready Context

**The gist** Researchers tested **nine VLMs from 2017–2025** and **20 human descriptions** on **100 Complex Social Behavior images**, tracking five visual-cognitive error types.

**Why it matters** For image-aware agents, evaluation should include **complex social scenes** and separate detection, recognition, hallucination, scene-understanding, and spatial-dependence failures instead of relying only on aggregate accuracy.

**Watch out** The conclusions come from **CSB plus an MS-COCO sample**. Modern models still sometimes base descriptions on different image regions than humans, labeled **spatial dependence error**.

Context Map

benchmarkimageresearch#agent-evals#benchmark-integrity#agent-reliability

Uncertainty

The conclusions come from **CSB plus an MS-COCO sample**. Modern models still sometimes base descriptions on different image regions than humans, labeled **spatial dependence error**.