A night at the opera

Behind the scenes of preparing YuMi’s conducting debut

Conducting is much more than standing up and waving your arms around in front of an orchestra. The conductor’s role is to share a vision of a work of music, to set the tone, and to shape the diverse voices of the musicians into a single expression in service to the composer. Conducting combines scholarship, technique, interpretation and charisma.

Recently I had the fascinating opportunity to prepare ABB’s YuMi, the world’s first truly collaborative dual-arm robot, for its debut as conductor at the Teatro Verdi in Pisa, Italy, for the gala of the First International Festival of Robotics. Working to master the nuanced technique of a human orchestra director with YuMi has surely been one of the most satisfying, albeit challenging, tasks of my professional career. YuMi will accompany Italian tenor Andrea Bocelli in a program of Verdi with the Lucca Philharmonic Orchestra.

YuMi’s sophisticated technology excited me and revealed many possible ways to realize art and music through the robot. Setting up the interaction between the elbow, forearm and wrist of the robot, making use of its versatility in repeated and demanding attempts to break down the upbeats and downbeats, was very successful.

The robot’s performance was developed in two steps. First, my movements were captured with a process called “lead-through programming,” where the robot’s two arms are guided to follow my motions with great attention to detail; these movements are then recorded. The second step involved fine-tuning the movements in ABB’s RobotStudio software, where we made sure the motions were synchronized to the music. Of course this took some technical expertise from ABB, but the lead-through programming let me focus on naturally doing what I do best, bringing the music to life. It is amazing to me that this could be accomplished so smoothly.

I must say, the final outcome is incredible. The gestural nuances of a conductor have been fully reproduced at a level that was previously unthinkable to me. But we did it: YuMi achieves a very high level of fluidity of gesture, with an incredible softness of touch and expressive nuancing. This is an incredible step forward, given the rigidity of gestures by previous robots.

Of course, YuMi is good when it comes to technique but is ultimately not gifted with human sensitivity. The robot uses its arms, but the soul, the spirit, always come from a human. I imagine the robot could serve as an aid, perhaps to execute, in the absence of a conductor, the first rehearsal, before the director steps in to make the adjustments that result in the material and artistic interpretation of a work of music.

Once we understood each other, YuMi and I became good friends and I am very much looking forward to the performance.

______________________________________________________________________________________

At the gala event on September 12, Andrea Bocelli will sing as YuMi directs “La Donna è Mobile,” the famous aria from Verdi’s “Rigoletto.” Soloist Maria Luigia Borsi will sing the classic soprano aria “O mio babbino caro” from “Gianni Schicchi” by Puccini. YuMi will also conduct a passage from Mascagni’s intermezzo from the opera “Cavalleria Rusticana.”