Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?
SeGaBench suggests coding models can expose C/C++ optimization opportunities that compilers miss, but generated artifacts still need correctness, semantic, and performance validation.
SeGaBench contains 100 synthetic and 20 source-backed C/C++ cases with hidden enabling semantics, oracle artifacts, validators, and reproducible performance protocols. The strongest model produced **94.8% correct artifacts**, while **83.3% of responses reached at least 1.05x speedup**.
Treat agents as speculative optimization partners: give them broader semantic context, then gate proposed transformations with correctness, contract, and performance checks. The strongest model recorded a performance result on **93.3% of cases**.
SeGaBench contains 100 synthetic and 20 source-backed C/C++ cases with hidden enabling semantics, oracle artifacts, validators, and reproducible performance protocols. The strongest model produced **94.8% correct artifacts**, while **83.3% of responses reached at least 1.05x speedup**. Treat agents as speculative optimization partners: give them broader semantic context, then gate proposed transformations with correctness, contract, and performance checks. The strongest model recorded a performance result on **93.3% of cases**. Correct artifacts often recovered only part of the oracle performance gap. The benchmark also centers on controlled cases, so it does not establish how reliably this approach transfers to large production codebases.
SeGaBench confirms a useful but bounded role for coding agents as generators of semantic optimizations that compilers miss, provided every artifact is independently checked for correctness and speed. It complements prior repository-agent benchmarks by isolating optimization capability, but its controlled cases and incomplete recovery of oracle performance prevent extrapolation to dependable production-code improvement.