Technion researchers in computer science use artificial intelligence to generate musically appealing personalized jazz solos that match human-specific preferences

A paper titled, “BebopNet: Deep Neural Models for Personalized Jazz Improvisation” was recently presented during the 21st International Society for Music Information Retrieval Conference (ISMIR 2020). Authored by M.Sc. students Nadav Bhonker (already graduated) and Shunit Haviv Hakimi, and by their advisor Professor Ran El-Yaniv at the Henry and Marilyn Taub Faculty of Computer Science at the Technion-Israel Institute of Technology, the paper indicates that it is possible to model and optimize personalized jazz preferences.

Click here for audio samples

Learning to generate music is an ongoing AI challenge. An even more difficult task is the creation of musical pieces that match human-specific preferences. In the BebopNet project, Bhonker and Haviv Hakimi, both amateur jazz musicians, focused on personalized, symbol-based, monophonic generation of harmony-constrained jazz improvisations. To tackle this objective, they introduced a pipeline consisting of several steps: supervised learning using a corpus of solos (a language model), high-resolution user preference metric learning, and optimized generation using planning (beam search). The corpus consisted of hundreds of original jazz solos performed by saxophone giants including Charlie Parker, Stan Getz, Sonny Stitt, and Dexter Gordon. They presented an extensive empirical study in which they applied this pipeline to extract individual models as implicitly defined by several human listeners. This approach enables an objective examination of subjective personalized models whose performance is quantifiable.

A plagiarism analysis was also performed to ensure that the generated solos are genuine rather than a concatenation of phrases previously seen in the corpus. “While our computer-generated solos are locally coherent and often interesting or pleasing, they lack the qualities of professional jazz solos related to general structure such as motif development and variations,” said the authors.

Prof. El-Yaniv hopes to overcome this challenge in future research. Preliminary models based on a smaller dataset were substantially weaker, and it is possible that a larger dataset would make a substantially better model. In order to obtain such a large corpus, it might be necessary to abandon the symbolic approach and rely on audio recordings which can be gathered in much larger quantities.

“Perhaps one of the main bottlenecks in AI art generation, including jazz improvisation, is how to evaluate quality meaningfully. Our work emphasizes the need to develop effective methodologies and techniques to extract and distill noisy human feedback that will be required for effective quality evaluation of personalized AI art. Such techniques are key to developing many cool applications,” noted Prof. El-Yaniv.

For more than a century, the Technion – Israel Institute of Technology has pioneered in science and technology education and delivered world-changing impact. Proudly a global university, the Technion has long leveraged boundary-crossing collaborations to advance breakthrough research and technologies. Now with a presence in three countries, the Technion will prepare the next generation of global innovators. Technion people, ideas and inventions make immeasurable contributions to the world, innovating in fields from cancer research and sustainable energy to quantum computing and computer science to do good around the world.

The American Technion Society supports visionary education and world-changing impact through the Technion – Israel Institute of Technology. Based in New York City, we represent thousands of US donors, alumni and stakeholders who invest in the Technion’s growth and innovation to advance critical research and technologies that serve the State of Israel and the global good. Over more than 75 years, our nationwide supporter network has funded new Technion scholarships, research, labs, and facilities that have helped deliver world-changing contributions and extend Technion education to campuses in three countries.