Preserve to Expand: Source-Aware Adaptation and Model Merging for Multilingual LLM-Based ASR
A study of multilingual speech-model adaptation finds a trade-off between learning new languages and preserving existing ones. Update merging and merge order can change how well the original languages are retained.
The paper studies adapting a trainable speech connector while keeping the language model frozen. In single-target expansion, **SSU-Wanda+TIES** gave the best reported balance of learning a new language and retaining source-language performance; target-only tuning and standard TIES caused severe forgetting.
For speech-model builders adding languages without replaying source-language data, compare both target adaptation and source retention. In multi-target expansion, update merging retained source performance more consistently, while recursive expansion could improve target adaptation depending on language order.
The paper studies adapting a trainable speech connector while keeping the language model frozen. In single-target expansion, **SSU-Wanda+TIES** gave the best reported balance of learning a new language and retaining source-language performance; target-only tuning and standard TIES caused severe forgetting. For speech-model builders adding languages without replaying source-language data, compare both target adaptation and source retention. In multi-target expansion, update merging retained source performance more consistently, while recursive expansion could improve target adaptation depending on language order. The findings concern connector-based Speech LLMs. The material does not provide enough detail to generalize the results to other model architectures or training setups.
This supplies a concrete, connector-specific example of multilingual expansion where adaptation quality depends on retaining source-language ability as well as learning the target. The results favor particular merge strategies in the tested Speech LLM setup, so they sharpen adaptation choices without establishing a general recipe for other architectures.