Exploiting passive behaviours for diverse musical playing using the parametric hand

Creativity and style in music playing originates from constraints and imperfect interactions between instruments and players. Digital and robotic systems have so far been unable to capture this naturalistic playing. Whether as an additional tool for musicians, function restoration with prosthetics, or artificial intelligence-powered systems, the physical embodiment and interactions generated are critical for expression and connection with an audience. We introduce the parametric hand, which serves as a platform to explore the generation of diverse interactions for the stylistic playing of both pianos and guitars. The hand’s anatomical design and non-linear actuation are exploitable with simple kinematic modeling and synergistic actuation. This enables the modulation of two degrees of freedom for piano chord playing and guitar strumming with up to 6.6 times the variation in the signal amplitude. When only varying hand stiffness properties, we achieve capabilities similar to the variation exhibited in human strumming. Finally, we demonstrate the exploitability of behaviours with the rapid programming of posture and stiffness for sequential instrument playing, including guitar pick grasping. In summary, we highlight the utility of embodied intelligence in musical instrument playing through interactive behavioural diversity, as well as the ability to exploit behaviours over this diversity through designed behavioural robustness and synergistic actuation.