Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models
Spoken Function Calling reframes speech understanding as structured function selection, giving voice-agent builders a clearer interface for open-domain tool invocation.
Spoken Function Calling replaces ambiguous intent rules with structured function definitions. The work curates spoken functions from existing datasets, uses a **multi-agent system** to synthesize **SFC-Bench**, evaluates text and audio language models, and post-trains audio models for the task.
Voice-agent builders should consider representing speech semantics directly as callable functions instead of maintaining a separate closed-set intent layer. That structure may make in-context examples and tool contracts easier to extend across domains.
Spoken Function Calling replaces ambiguous intent rules with structured function definitions. The work curates spoken functions from existing datasets, uses a **multi-agent system** to synthesize **SFC-Bench**, evaluates text and audio language models, and post-trains audio models for the task. Voice-agent builders should consider representing speech semantics directly as callable functions instead of maintaining a separate closed-set intent layer. That structure may make in-context examples and tool contracts easier to extend across domains. The abstract says SFC improves semantic extraction over traditional spoken-language understanding but provides no dataset size, accuracy figures, or error breakdown. Its robustness to real microphones, accents, noise, and production tool schemas is therefore unresolved.
Spoken Function Calling moves voice understanding from fixed intent classification toward extensible tool contracts, making structured calls part of semantic extraction rather than a downstream translation step. It complements realtime voice infrastructure and structured device tools, while leaving production reliability under noise, accents, and real schemas unevaluated.