AI EngineerVideoSource Linked
While my guitar gently speaks — Todd Fisher, Philo Ventures
A talking guitar is a useful agent-interface case study: connect speech input, a local model, synthesis, pitch detection, and physical control, then expose where latency and segmentation break.
AI Engineer · Aug 18, 2026
Source Summary
Fisher built a Logic Pro plugin that turns generated speech into guitar-controlled output. The system combines a **local language model**, text-to-speech, word segmentation, **YIN pitch detection**, synthesized notes, and a vocoder.
Practical Implication
The project is a compact pattern for embodied agent interfaces: keep language generation separate from signal processing, then map model output onto the instrument’s existing controls. Pre-baked vocal samples can extend the result toward singing.
Agent-Ready Context
Fisher built a Logic Pro plugin that turns generated speech into guitar-controlled output. The system combines a **local language model**, text-to-speech, word segmentation, **YIN pitch detection**, synthesized notes, and a vocoder. The project is a compact pattern for embodied agent interfaces: keep language generation separate from signal processing, then map model output onto the instrument’s existing controls. Pre-baked vocal samples can extend the result toward singing. The live demo remained choppy and hit several failures. Silence-based word splitting is unreliable, and higher-quality pitch-shifted singing was too computationally heavy to generate live.
Context Map
craftaudio#sound-design#tool-useUncertainty
The live demo remained choppy and hit several failures. Silence-based word splitting is unreliable, and higher-quality pitch-shifted singing was too computationally heavy to generate live.