Communications - March 2011

Doi: 10.1145/1897852.1897874

technical Perspective concerto for Violin and Markov Model By Juan Bello, Yann LeCun, and Robert Rowe

in the OPening moments of Jean Sibelius’ Violin Concerto, the young soloist plays delicately, almost languidly. The orchestra responds in kind, muting the repeated string motif to a whisper. As the piece progresses, soloist and orchestra alternatively perform the main motifs in increasing measures of power and virtuosity, which inexorably lead toward the movement’s stirring resolution. The soloist looks relieved as she crosses the stage to shake the conductor’s hand.

This violinist, like most others in music education, can benefit enormously from interacting with large ensembles in honing her performing skills. However, the demand far exceeds the number and capabilities of existing orchestras, ensuring most of these students won’t have access to this experience. Our soloist is no exception. The previous paragraph describes her interaction with Chris Raphael’s Music Plus One system: A machine learning-driven alternative to working with orchestras that retains much of the expressivity and interactivity that makes concerto performance such a rewarding and educational experience. The following paper details the approach, for videos see http://www.music.infor- matics.indiana.edu/papers/icml10/.

Automatic music accompaniment
has been actively researched since the
1980s, starting with the work of such pi-
oneers as Barry Vercoe and Roger Dan-
nenberg. The problem can be broken
into three subparts: 1 tracking the play-
ing of a human soloist, matching it to a
known musical score, and synthesizing
an appropriate accompaniment to that
solo part in real time. Solutions usually
involve ingenious pattern-matching
mechanisms for dealing with expres-
sive, incorrectly played or missing notes
in the soloist performance, while using
the output of the pattern match to drive
the scheduling of accompaniment
events. However, as Raphael notes, it is
impossible to accomplish score follow-
ing by reaction alone. The system must
incorporate a predictive component
that attempts to align upcoming notes
of the accompaniment with imminent
attacks of the human player. Failing to
solve this problem can result in poten-
tially disastrous consequences for the
performance.

Reference

1. Dannenberg, r.b. an on-line algorithm for real-time accompaniment. In Proceedings of the International Computer Music Conference (Paris, France, 1984), 193−198.

Juan Bello is an assistant professor of music technology in the Department of music and Performing arts Professions at nyu’s steinhardt school of Culture, education and human Development.

Yann LeCun is silver Professor of Computer science and neural science at nyu and co-founder of museami, a music technology company.

Robert Rowe is a professor and vice chair in the Department of music and Performing arts Professions at nyu’s steinhardt school of Culture, education and human Development, where he directs the music Composition Program.