'TransProse' Software Creates Musical Soundtracks from Books

Novels are typically for reading, but a recent study attempted to create an entirely new art form by translating the emotions of the written word into music.

The work could transform the way people interact with literature and spark new ways to visualize information, such as audiovisual e-books that generate music according to the emotions on the page or novel music apps, the researchers said.

"Given a novel in an electronically readable form, our system — called TransProse — generates simple piano pieces whose notes are dependent on the emotion words in the text," said Saif Mohammad, a computer scientist at the National Research Council Canada. [Top 10 Things that Make Humans Special]

Together with Hannah Davis, who created TransProse as her master's thesis at New York University, Mohammad used the software to count the density of words associated with eight basic emotions: anticipation, anger, joy, fear, disgust, sadness, surprise and trust.

Although there have been previous studies that analyzed sentiments or examined the creation of music, TransProse is the first to combine the two, the researchers said. "It is the first system that automatically generates musical pieces based on the emotions in the text, and uses a novel mechanism to determine sequences of notes that capture the emotional activity in text," Mohammad said.

The algorithm uses databases to rate words according to their emotional value, thereby analyzing sentiment in the text and gauging its "emotional temperature."

"For example, whether the key is major or minor is determined by the ratio of positive to negative words across the novel; the main octave is based on the difference between the joy and sadness word densities throughout the text," Davis said.

Conveying the mood

The researchers used TransProse to measure the sentiment changes throughout a book, automatically generating music that reflects specific moods. So far, the scientists have created musical soundtracks for several famous novels, including "Peter Pan" by J.M. Barrie, "A Clockwork Orange" by Anthony Burgess and "The Road" by Cormac McCarthy, among others. The musical pieces are available on the TransProse website.

The algorithm works by splitting the book into four sections: the beginning, early middle, late middle and ending. Then, it generates an emotion profile for each part — a collection of various statistics about the presence of emotional words in the text, Mohammad said. [5 Ways Your Emotions Influence Your World (and Vice Versa)]

But generating music is challenging, the researchers said, because "in composing new music, just as in creating a new story, there are an infinite number of choices and possibilities."

To automate this process, the scientists have come up with a number of rules to determine various elements of music, such as tempo, scale, octave, major or minor key, notes, and the sequence of notes for multiple melodies. These rules are basically "guidelines to make the sequence of notes sound like music as opposed to a cacophonous cascade of sounds," Mohammad said.

The researchers also determined the sequence of notes to be played as they correspond to the changes in density of emotional words in the text.

The resulting compositions consist of a main melody determined by the dominant emotion, accompanied by other melodies generated by other moods throughout the text. "The other two melodies — besides the main one — are based on the top two most prevalent emotions in the novel," Davis said.

Finally, with the help of standard open-source software called JFugue, the sequence of notes is converted into an audio file of piano music.

Musical tweets

The research may improve people's understanding of the elements of music that make it emotional, the scientists said. "Music and literature have an intertwined past. It is believed that they originated together, but in time, the two have developed into separate art forms that continue to influence each other," Mohammad said.

"This work shows that it is possible to automatically translate aspects of literature to music," he added. "We hope that it will lead to further research leading to even more sophisticated systems that automatically generate beautiful and emotionally representative music from text."

Furthermore, there may be other applications beyond the prospect of creating audiovisual e-books that generate music to accentuate the mood on the page, the researchers said.

For instance, pieces of literature could be mapped to musical pieces if the emotions in the text and the piece of music are compatible. Apps could then find and play songs compatible with the mood of the chapter being read.

This type of software could even help filmmakers identify or generate music for movie scripts, or it could be used to transform social media and Web interactivity, the researchers said. "Imagine a tweet stream that is accompanied by music that captures the aggregated sentiment towards an entity, or displaying the world map where clicking on a particular region plays music that captures the emotions of the tweets emanating from there," Mohammad said.

The study is detailed in a paper on the prepublish site Arxiv.

Follow us @livescience, Facebook & Google+. Original article on Live Science. Follow the author on Twitter @SciTech_Cat.