CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity
CreativeInstruct adds learned control spans that recover base-model-like diversity after post-training, with reported gains in human creativity ratings and downstream RL training.
CreativeInstruct teaches a model to insert **[StartCreativity] spans** that steer selected generations toward greater variation. The paper also proposes graph-edit distance for structural narrative diversity and reports a human preference for its creativity in **70.3% of cases** versus post-trained models.
For builders working on ideation, synthetic data, or exploratory agent behavior, this suggests creativity can be exposed as a learned generation mode instead of requiring several models at inference. The reported GRPO runs also improved by **about 4% on AMC** and **5 percentage points on MATH** over training from the post-trained checkpoint.
CreativeInstruct teaches a model to insert **[StartCreativity] spans** that steer selected generations toward greater variation. The paper also proposes graph-edit distance for structural narrative diversity and reports a human preference for its creativity in **70.3% of cases** versus post-trained models. For builders working on ideation, synthetic data, or exploratory agent behavior, this suggests creativity can be exposed as a learned generation mode instead of requiring several models at inference. The reported GRPO runs also improved by **about 4% on AMC** and **5 percentage points on MATH** over training from the post-trained checkpoint. The evidence comes from narrative generation and two math benchmarks, not coding-agent tasks. The abstract does not establish how reliably the control spans transfer across domains, prompts, model families, or production constraints.