Open Brain
arXivPaperNeeds Review

Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization

The paper estimates confidence in LLM-generated COBOL summaries by comparing outputs from diverse models, then aggregating corroborated agreement and contradictions. It describes a method, not measured deployment results.

arXiv · Nov 6, 2026
Open Source Open MarkdownOpen JSON
Source Summary

The approach generates summaries with diverse models, compares pairs with an LLM evaluator that emits structured judgments, then applies a calibrated aggregation rule. It targets correctness estimation when human-authored reference summaries are unavailable.

Practical Implication

For code agents that summarize legacy code, this offers a possible review signal when reference answers are scarce: compare independent summaries and treat corroborated contradictions as evidence against confidence.

Agent-Ready Context
The approach generates summaries with diverse models, compares pairs with an LLM evaluator that emits structured judgments, then applies a calibrated aggregation rule. It targets correctness estimation when human-authored reference summaries are unavailable.

For code agents that summarize legacy code, this offers a possible review signal when reference answers are scarce: compare independent summaries and treat corroborated contradictions as evidence against confidence.

The material provides no evaluation results or details about calibration quality, so it does not show how reliably the method predicts correctness in practice. Its stated use case is COBOL summaries.
Connected Context · Feed7 Judgment

This proposes comparing diverse model-generated code summaries to estimate correctness when reference summaries are unavailable, but the supplied material gives no evidence that its aggregation is calibrated. It is a possible review signal for COBOL summaries, not a demonstrated correctness guarantee.

Context Map
benchmarkcoding#agent-evals#coding-agents
Uncertainty
The material provides no evaluation results or details about calibration quality, so it does not show how reliably the method predicts correctness in practice. Its stated use case is COBOL summaries.