arXivPaperNeeds Review

Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations

Lower toxicity scores may hide changes in representational harm rather than its removal. Safety evals need topic and demographic analysis alongside surface-form classifiers.

arXiv · Sep 17, 2026

Open Source Open Markdown Open JSON

Source Summary

Across **450,000 completions** from **15 GPT-lineage models**, explicit discriminatory clusters declined while subtler representational disparities remained or emerged. Three classifiers marked a GPT-5 breast-cancer topic framed around men’s rights as non-toxic.

Practical Implication

Do not use a falling toxicity score as the sole release gate for generative features. Compare topic coverage and representation across demographic conditions, then inspect whether safety tuning redistributes who receives positive, negative, or constrained portrayals.

Agent-Ready Context

Across **450,000 completions** from **15 GPT-lineage models**, explicit discriminatory clusters declined while subtler representational disparities remained or emerged. Three classifiers marked a GPT-5 breast-cancer topic framed around men’s rights as non-toxic.

Do not use a falling toxicity score as the sole release gate for generative features. Compare topic coverage and representation across demographic conditions, then inspect whether safety tuning redistributes who receives positive, negative, or constrained portrayals.

Women-directed topic diversity was **36% lower** than men-directed diversity at the GPT-4 alignment boundary. The study covers one model lineage and three demographic conditions, so its proposed detection protocol still needs validation across other models and forms of harm.

Connected Context · Feed7 Judgment

This identifies a release-evaluation failure mode that aggregate toxicity metrics can hide: safety tuning may change harm from explicit language into unequal topic coverage and representation. It therefore broadens benchmark integrity from detecting obvious harmful outputs to comparing distributions across demographic conditions, while limiting the conclusion to one model lineage and three tested conditions.

Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization EvaluationBoth show that a scalar evaluator can conceal the underlying failure; the judge study motivates probing evaluation mechanisms, while this work requires demographic coverage and representation checks beyond toxicity labels.The Low Frequency Trap: Video Language Models Fail at Simple Event BookkeepingThe video study likewise finds that improved aggregate scores need not reflect faithful underlying behavior, reinforcing the need for structured evidence checks rather than answer-level metrics alone.Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge BiasThe mechanistic account of judge bias supports scrutinizing evaluators themselves, which is consequential here because three classifiers failed to flag one reported form of representational harm.

Context Map

benchmark#benchmark-integrity#agent-evals

Uncertainty

Women-directed topic diversity was **36% lower** than men-directed diversity at the GPT-4 alignment boundary. The study covers one model lineage and three demographic conditions, so its proposed detection protocol still needs validation across other models and forms of harm.