# LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning

Source: [arXiv](https://arxiv.org/abs/2607.02513v1)  
Feed7 permalink: https://feed7.dev/p/2607-02513v1-0lwaytn  
Published: Unknown  
Trust: Needs Review (needs_review)

## Why Included

First unlearning testbed with ground-truth parameter localization: injects synthetic PII into known weights of OLMo 1B/7B models, showing current methods hide rather than erase and fall to resurfacing attacks.

## Source Summary

**The gist** LACUNA injects synthetic PII into predetermined parameters of **OLMo-based 1B and 7B** models via masked continual pretraining, giving unlearning research its **first ground-truth parameter-level** testbed instead of output-only benchmarks.

## Practical Implication

**Why it matters** If you count on unlearning to strip sensitive data from a model, output-level evals can lie: methods that look clean behaviorally were **highly imprecise** at the weight level and vulnerable to **resurfacing attacks**. When localization is right, even **simple gradient-based erasure** holds up — precision, not the removal algorithm, is the lever.

## Agent-Ready Context

**The gist** LACUNA injects synthetic PII into predetermined parameters of **OLMo-based 1B and 7B** models via masked continual pretraining, giving unlearning research its **first ground-truth parameter-level** testbed instead of output-only benchmarks.

**Why it matters** If you count on unlearning to strip sensitive data from a model, output-level evals can lie: methods that look clean behaviorally were **highly imprecise** at the weight level and vulnerable to **resurfacing attacks**. When localization is right, even **simple gradient-based erasure** holds up — precision, not the removal algorithm, is the lever.

**Watch out** The PII is **synthetic** and planted into known weights, a far cleaner setup than organically memorized data; whether precise localization is achievable in real pretrained models remains the open question.

## Context Map

- Layer: benchmark
- Domains: research, security
- Topics: benchmark-integrity

## Uncertainty

- The PII is **synthetic** and planted into known weights, a far cleaner setup than organically memorized data; whether precise localization is achievable in real pretrained models remains the open question.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.