Max Pink Trombone (GitHub) is an open-source port of the amazing Pink Trombone parametric vocal synthesizer for Max/MSP. By closely replicating the acoustics/parameters of the human mouth (tongue + lip position, air pressure, vocal cord tension, etc.), realistic human voices can be synthesized from scratch in real-time.
HandMouth is an experimental application of MPT that uses a Leap Motion hand gesture controller to control the voice(s) using just a single hand. Using your second hand for MIDI input, you can have an entire virtual chorus at your fingertips. The internal processes of HandMouth are based on BodyMouth, a similar instrument controlled using real-time body position.
The applications of HandMouth are still in consideration. Whereas the choreographical and properties of BodyMouth are intrinsically theatrical and expressive, HandMouth is considerably less “visual” of an instrument. Its applications may be more practical, perhaps as an accessibility tool for mute or otherwise nonverbal individuals.
The patch for MPT can be viewed on my GitHub. Feel free to use it however you’d like, but please do leave appropriate credit! The code for HandMouth is currently private.
Tech Used:
- Custom Max/MSP patch based on Pink Trombone (GitHub)
- Leap Motion controller (v1)
- (developmental version) Intel RealSense D415 Depth Camera
About
Max Pink Trombone (MPT), as the name might suggest, is mainly a 1-1 port of Pink Trombone into Max, a popular creative tool for real-time sound processing and synthesis. I was inspired to make this from my previous work on BodyMouth, which introduced me to the lovely world of parametric speech synthesis: a digital emulation of the physical processes of a real human mouth to synthesize realistic human voices in real-time. A fantastic example is Pink Trombone, a web-based, interactive open-source vocal synthesizer. Go try it out!
Bringing Pink Trombone into Max allows you to leverage the amazing vocal synthesizer within the context of Max/MSP and the incredible, low-level control it offers in the creation of live performances involving real-time synthesized speech.
HandMouth was my first application of MPT. It uses a Leap Motion hand tracking camera and API to retrieve precise real-time hand skeleton tracking data, as seen in the corners in the videos above, and use them to “speak” through the synthesized voices. The relevant data for this use case involves the position/orientation of the palm, fingers and wrist. By adjusting these aspects of the hand position precisely over time, you can “sing” a variety of simple single- or multi-syllabic words.