# The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev

Source: [AI Engineer](https://www.youtube.com/watch?v=fZH97QHHYjY)  
Feed7 permalink: https://feed7.dev/p/the-unreasonable-effectiveness-of-bm25-for-agentic-search-jo-kristian-be-10rfoze  
Published: 2026-09-16T13:30:32.000Z  
Trust: Source Linked (source_linked)

## Why Included

Agent search changes the retrieval workload: models issue long, iterative queries and inspect literal matches. BM25 remains useful, but tuning, harness design, and end-to-end evaluation matter.

## Source Summary

Agentic search is retrieval inside an agent loop, combining a tool-capable model, a harness, and a retrieval engine. BrowseComp+ has **830 questions** over roughly **105,000 documents**, with agents repeatedly querying and reformulating rather than issuing one search.

## Practical Implication

Treat BM25 as a tunable primitive, not a fixed baseline. Long model-written queries, literal matching, filesystem workspaces, and progressive disclosure can give coding agents inspectable results they can search further with familiar tools.

## Agent-Ready Context

Agentic search is retrieval inside an agent loop, combining a tool-capable model, a harness, and a retrieval engine. BrowseComp+ has **830 questions** over roughly **105,000 documents**, with agents repeatedly querying and reformulating rather than issuing one search.

Treat BM25 as a tunable primitive, not a fixed baseline. Long model-written queries, literal matching, filesystem workspaces, and progressive disclosure can give coding agents inspectable results they can search further with familiar tools.

The talk does not establish that BM25 always beats embeddings. Implementations and its **two hyperparameters** can materially change results, while single-query ranking metrics miss whether the full agent loop completes its task.

## Connected Context

Feed7 judgment across 812 accumulated Signals:

This narrows retrieval evaluation from one-shot ranking to whether an inspectable agent loop completes its task. It restores tuned BM25, literal matching, and filesystem search as serious primitives for long model-written queries, without establishing that they universally outperform embeddings or other context strategies.

- [Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI](https://feed7.dev/p/context-engineering-in-2026-louis-francois-bouchard-omar-solano-samridhi-1dnlyr0) — Supplies a competing baseline: some workloads may perform better with full cached history, so retrieval must beat that option on the actual constraint.
- [virgiliojr94/book-to-skill](https://feed7.dev/p/book-to-skill-1av16sr) — Implements the same progressive-disclosure pattern for stable documents by exposing an index first and loading referenced chapters on demand.
- [Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked](https://feed7.dev/p/your-agents-lack-context-here-s-how-to-fix-you-re-absolutely-right-brand-0pdoj93) — Extends task-specific retrieval beyond literal document matching to permission-aware organizational context and conflict resolution.
- [How to Generate Mergeable Code with a Context Engine — Peter Werry, Unblocked](https://feed7.dev/p/how-to-generate-mergeable-code-with-a-context-engine-peter-werry-unblock-18x9r65) — Shows an implementation consequence for coding agents: retrieval must include decisions, conventions, and review history, not stop at plausible code matches.

## Context Map

- Layer: context
- Domains: coding, research
- Topics: retrieval, context-engineering, agent-evals

## Uncertainty

- The talk does not establish that BM25 always beats embeddings. Implementations and its **two hyperparameters** can materially change results, while single-query ranking metrics miss whether the full agent loop completes its task.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.