Diff Risk Score: AI-driven risk-aware software development
Meta's Diff Risk Score, a fine-tuned Llama model, predicts whether a code change will cause a production incident — used to replace blanket code freezes, landing 10,000+ changes during one 2024 freeze period.
**The gist** **Diff Risk Score (DRS)** is a **fine-tuned Llama** model that scores each code change's likelihood of causing a production incident and flags the risky snippets. It powers **19 use cases** at Meta — code unfreeze, test selection, reviewer routing — and during one **2024** partner event let **10,000+** previously frozen changes ship with minimal impact.
**Why it matters** Risk-ranking diffs with an LLM is a copyable pattern as agents raise your change volume: gate merges by predicted blast radius instead of freezing everything or reviewing everything equally. Meta's move from blanket freezes to per-diff risk is the interesting design decision, independent of scale.
**The gist** **Diff Risk Score (DRS)** is a **fine-tuned Llama** model that scores each code change's likelihood of causing a production incident and flags the risky snippets. It powers **19 use cases** at Meta — code unfreeze, test selection, reviewer routing — and during one **2024** partner event let **10,000+** previously frozen changes ship with minimal impact. **Why it matters** Risk-ranking diffs with an LLM is a copyable pattern as agents raise your change volume: gate merges by predicted blast radius instead of freezing everything or reviewing everything equally. Meta's move from blanket freezes to per-diff risk is the interesting design decision, independent of scale. **Watch out** Meta shares **no accuracy numbers** and no training-data detail; explainability is admitted to be **an open research area**, and the config-change variant is still early. Reproducing this without years of incident data is nontrivial.