Daniel Johnson's personal fragment of the web

Posts tagged "music"

Summer Research on the HMC Intelligent Music Software Team

This summer, I had the chance to do research at Mudd as part of the Intelligent Music Software team. Every year, under the advisement of Prof. Robert Keller, members of the Intelligent Music Software team work on computational jazz improvisation, usually in connection with the Impro-Visor software tool. Currently, Impro-Visor uses a grammar-based generation approach for most of its generation ability. The goal this summer was to try to integrate neural networks into the generation process.


Composing Music With Recurrent Neural Networks

It’s hard not to be blown away by the surprising power of neural networks these days. With enough training, so called “deep neural networks”, with many nodes and hidden layers, can do impressively well on modeling and predicting all kinds of data. (If you don’t know what I’m talking about, I recommend reading about recurrent character-level language models, Google Deep Dream, and neural Turing machines. Very cool stuff!) Now seems like as good a time as ever to experiment with what a neural network can do.

For a while now, I’ve been floating around vague ideas about writing a program to compose music. My original idea was based on a fractal decomposition of time and some sort of repetition mechanism, but after reading more about neural networks, I decided that they would be a better fit. So a few weeks ago, I got to work designing my network. And after training for a while, I am happy to report remarkable success!

Here’s a taste of things to come:


Trying out Ableton Live

A few weeks ago, I downloaded a trial version of Ableton Live and fiddled around with it. I wrote two short songs, and I thought I may as well put them up here. So, without further ado, here are the two songs I have written:

Street Corners


Speech -> Music

This is a program I wrote to explore the connection between the patterns in poetry and speeches and the patterns in music. Unfortunately, I cannot upload it here (the WebMIDI API integration does not seem to work anymore and also it was built with a very specific soundfont so viewing it elsewhere would not be equivalent), so instead of the actual program you can watch a video of it.

The program, written in Javascript and utilizingthe WebMIDI API, first runs through the poem and extracts phonemes using the excellent Carnegie Mellon Pronouncing Dictionary. It then links together words that share phonemes, assigning each pair a strength depending on the number of matched phonemes (plus a bonus if the words are identical). Finally, it begins to play the poem syllable-by-syllable, assigning each word a note based on its first letter. When the playhead reaches a word that has links to other words, it creates a secondary playhead that plays the matched word simultaneously, producing temporary chords and repetition that die off over time.

In this video, the program is playing its rendition of “The Raven” by Edgar Allan Poe.


© . All rights reserved. | Top