Machine Learning and the Arts

Andreas Refsgaard stands in front of a computer monitor in a t-shirt and baseball cap

Photo: Heidi Erickson

For two weeks in fall 2022, the Lewis Music Library hosted Andreas Refsgaard, a Copenhagen-based creative coder, interaction designer, and educator selected as a 2022–23 Center for Art and Technology (CAST) Visiting Artist. Initiated by Avery Boddie, Lewis Music Library department head, Refsgaard’s residency reflects the library’s focus on digital collections and music technology.

“In a lot of my work, I try to delegate creative decisions to algorithms. I try to make systems where I don’t necessarily know what the result will be,” says Refsgaard. This element of surprise is infused throughout the artist’s work, alongside a playful approach that seeks to make machine learning tools more accessible to all. Two of Refsgaard’s creations—both exemplifying his unique humor—were installed in Lewis throughout the fall term. In the interactive project Doodle Tunes, users draw a piano, drums, bass guitar, or saxophone on a touchscreen, prompting the program to play that instrument’s sound. As more instruments are added, the composition becomes more complex. In Sounds from the Mouth, when the participant faces a webcam and opens their mouth, an autonomous snapshot is created which bounces off a piano, creating a cascading series of notes. Caleb Hall, music technology and digital media librarian, collaborated with Refsgaard to create a custom set of sounds for the installations.

As part of his visit to MIT, Refsgaard also presented a public artist talk and visited classes on music and technology as well as the MIT laptop ensemble. He led two creative workshops, open to novices as well as experts, which drew participants not only from Music and Theater Arts, but Sloan School of Management, the Schwarzman College of Computing, and beyond. “In addition to serving as a space for quiet study and access to physical resources, we want the library to be a place where users congregate, collaborate, and explore together,” says Boddie. “This residency was very successful in that regard.”

To watch a video about Refgaard’s residency or play Doodle Tunes and Sounds from the Mouth, visit