JUNKYARD RAVE (Realtime Audio Variational autoEncoder) is an interactive, mechatronic, welded sculpture that invites participants to musically explore latent space in real-time using hand-fabricated controls parametrically mapped to multiple RAVE* neural network models.
Frequently, AI is used to produce transactional outcomes; however, JUNKYARD RAVE empowers participants to co-perform with AI. The sculpture becomes an augmented musical instrument. Through embodied action–pushing eight foot high levers and switching flips–the audience and neural networks become co-navigators of sonic latent space. As audio is fed into the RAVE models, data is recycled and transformed into a limitless variety of new sonic artifacts. The models enable fast and high quality neural audio synthesis as the participants engage with the unique controls. Vibrant Arduino-programmed LEDs provide additional real-time visual feedback for control manipulation. The sculptural frame was welded and fabricated in-house from recycled metal sourced locally from local junkyards in Everett, MA.
Here is a video from a party at the MIT Media Lab where we connected the Junkyard RAVE to a collection of the DJ’s music and let people manipulate the sonic worlds through the various lever , switch and microphone interactions, generating synthesized audio in real-time as a communal (AI) instrument.
Reference:
RAVE (Caillon, Antoine, and Philippe Esling. “RAVE: A variational autoencoder for fast and high-quality neural audio synthesis.” arXiv preprint arXiv:2111.05011 (2021))
(Detailed Documentation of Collaborators and Demos coming soon.)
Manaswi Mishra 