During the pandemic, I was seeking a way to interact with other musicians to create music. Inspired by the famous Twitch Plays Pokémon, I looked at Twitch's API and discovered that they were using a highly customised version of a very old chat communications platform: internet relay chat (IRC). Having used IRC in the late 1990s, I knew that creating a bot was possible and began to explore using Max to connect people to my eurorack synth. The initial idea was to allow people to simultaneously borrow real world hardware using a text-based interface. Not only would this give people the ability to play with real synthesizers that might otherwise be unaffordable, but the project would allow for collaborative music-making among strangers during a time that people were unable to do safely in person.
Iteration: 1
The first iteration of !tpts which consisted of a physical Mother 32 connected via MIDI and an audio interface.
The first version of Twitch Plays the Synth (!tpts) consisted of a physical Moog Mother 32 mounted in a eurorack case with several other modules. The euroshield 1 is a eurorack module made by 1010 Music that allows one to program a Teensy 3.2 dev board to control other modules using control voltage (CV). It was used to convert MIDI commands from Max to two control voltage outputs to the Mother 32. One output was dedicated to 1 volt per octave for melodies while the other was used to control the filter frequency. For testing, I recruited a friend to assist me in making music on a live twitch stream. They made melodies and changed the filter frequency while I patched and made counter melodies and harmonics to accompany them. Eventually, I upgraded the MIDI module to an Expert Sleepers ES-8 DAC/ADC module that allows one to send and receive CV and audio to and from the eurorack synth. This made it possible to add the ability to adjust the filter resonance, FM amount to the voltage controlled oscillator (VCO), and low frequency oscillator (LFO) frequency remotely. In addition, I added the ability to change the tonal center, tempo, slew rate (glide in between pitches).
My friend and I played for hours at a time on and off for a few weeks; this collaboration was between us as no one would join in during the performance. This was okay as we were just using the time to test the system and improve upon the design. Around this time, I was contacted by a fellow musician/composer James Peck also known as Memorecks. He was interested in creating a simular project and ran into !tpts and wanted to know if I would be willing to collaborate on the same project together.
Iteration: 2
"Bladerunner Esq." from the second version which included an all-digital version of the synth and more controls.
Memorecks liked the idea of the melodic sequencer, but had the idea to make the synth virtual so that all modules of the synthesizer were available and more flexible. Doing this allowed the stream to be live 24 hours a day so that people all over the world could visit and make music. This version of the synth ran atop Ableton Live with two Max for Live devices controlling the melodies and adjusting the parameters. The visual and auditory aspects of the synth were done in Native Instruments React by Memorecks.
The synth contained two oscillators that could be controlled via two separate sequencers as well as other parameters, shown as knobs in the video above. A visual was included in the broadcast that showed the waveform and a piano roll. This allowed the chat to see as well as hear their creations. To help people learn how to use the synth, we made a manual that users could access by typing "!help". This was especially helpful to those that were new to synthesis as it gave a broad overview of the basics while teaching them how to use our specific interface.
To run everything uninterrupted, we purchased a Mac mini server on MacStadium, installed the software, and broadcasted using OBS. Aside from a couple of OBS crashes and macOS update cycles, the server had a good uptime lasting for more than a year.
"Racket Baaaaaal"
Iteration: 3
The third version featured a photo-realistic interface. Here is "Blip Bloop" from that version.
The final version of the stream featured a photo-realistic version of the synth along with more parameters that allowed the chat to add and control ambient sounds along with effects such as tape, chorus, and bit crushing. Some of these features were implemented from requests on our community Discord server where members could add clips of their works, setup meetup times with other musicians, and discuss anything else they desired. One of the members volunteered to moderate the chat and another created a tool to specifically generate melodies using a JavaScript based piano. Based on user feedback, another manual page was created and embedded underneath the video playback.
"Blessd 44"
As interest waned we decided to move onto other projects and disconnected the server. The source code is still available via a Github repository.