Modern Glitching – Auditory Enhancement of Reality with Music

I recently gave a talk at TedXTokyo 2014 about a musical device I built with the aim of giving other people the chance to hear the world as musically as I do. To date, around fifty people have tried my mobile composing inspiration rig with me – mostly with very enthusiastic responses afterwards, and in the more musical/auditory types there’s also a degree of joyful disorientation.

Some of the background to what I think is going on: Around twenty years ago, psychology professor Diana Deutsch discovered what she called the Speech to Song illusion. Essentially, a spoken phrase repeated often enough starts to take on musical qualities. There’s a great Radiolab episode which explains the phenomenon.

For me, I don’t require repetition in order to hear spoken phrases as musical. Speech is intrinsically musical for me, and so is the rest of the world – from cars passing to people typing. I really wanted to share my experience as I find it very beautiful.

 

STORY TIME!

It was the day after winning a prize at MusicTechFest’s Boston Hackathon event, which I took part in and filmed for the BBC. I was sharing a small apartment with a bunch of other music obsessives.  The day before I left, instead of packing neatly as normal I optimistically chucked everything I could see into my case and hoped for the best.

The idea of adjusting auditory experience or adding a ‘Glitch’ to reality – at least in aural terms – is not a new process.  But glitching with modern tech sounded like a great way to reveal the music I hear all the time – plus I wanted to add a more musical classical compositional element to the practice.

Sean Manton and CJ Carr (who was familiar with glitching) were two other music hackers I met at the Hackathon. They were instrumental in my sleep-deprived electronic inspiration.

So, grabbing my iPad and headphone splitters,  I built the first iteration of a device that messed with the ambient sound in the room in real time in a pleasurable manner. Raw audio is changed in real-time and enhanced with sound effects, and crucially, I added basic musical elements and phrases that would play simultaneously. A while later, I got the thing working in a way I liked and emerged from my room, eager to try it on other musical / technical people. My ideal system would allow for me to sing and play melody and harmony but I was nowhere near doing that yet.

So, extra headphones bought, splitter in,  time to try my rudimentary iPad device on CJ and Sean in a quiet teahouse. It was so much fun! The sounds of tea being made, the door opening, teaspoons hitting cups, amplified and enhanced by repetition! Those sounds were unexpected, made musical and wonderfully tingly. I sang along to the notes in the cafe to accentuate them. The staff at the teahouse got interested, all they could hear was us singing and hitting teaspoons and laughing. So we asked if they wanted to try it then wired them in to see their response – they liked it – a lot.

 

Going mobile was more interesting – we were physically connected by our headphone cables, so it took a while to maneuver through the door but together we emerged, wired up out into the wild. And, once our headphones were in, we pretty much stayed ‘glitched in’ for at least 5 hours straight. I could hear the music I normally hear but amplified! Wow! I sang in joyous harmony with the world for my cohorts, who joyously joined in. An ear-opening experience indeed, and I expect we were a strange sight, connected together by cables, singing and swaying – especially as only we could hear the glorious harmonic results of our musical musings.

What followed: glitching around a bookshop, glitching through a delicious dinner at a noodle restaurant until we got chucked out at closing time – and (my favourite) glitching on public transport all over Boston. Some time during the evening, I added a recorded drum loop to the experience – an albeit low-tech but incredibly effective way to turn the world into a very funky soundtrack – rhythm along with harmony generated by reality transcended a run-of-the-mill walk through a city, making it a musical recital!

Now, without our headphones in, the world seemed dry and desolate. And, after trying this on six other people with the persuasive line ‘Hey, you wanna do some digital drugs, guys?’ to gratifying results, it didn’t take long for us to ascertain this was indeed a pleasurable and slightly psychedelic auditory experience – not only as a participant, but also as a listener. The three of us decided to take modern glitching further with a bit more technological clout.

A quick stop on the way back to the hacker apartment meant we now had extra kit. And, by 0100, Sean had plugged his Raspberry Pi computer into the TV – programming on the Pi with PureData. We made some tea and ate bread with the most delicious honey (the honey was in Bb major 6th) and kept working. By then it was 0300 and my taxi was due to arrive at 0615, we only had a few hours left!

We all wanted to add fine-grain control to this strange and wonderful auditory experience. CJ had brought his FM transmitter and binaural microphone/headphones and we plugged everything into my Mac.   I wanted to do more than just sing the city, I wanted to play it too. That meant configuring something that could take multiple inputs – MIDI and Audio at the same time.

Finally at 0400, and full of incredible quantities of tea, bread and honey, we were now running a glitching instance on Ableton Live, with a binaural microphone / headphone setup and my iRig Keys midi controller hooked up. I started building musical stems right then and there.

The latest version has more than just repetition, my new glitching device can harmonise and play with the world in a much deeper way – and I walk around a city first to get what key its in and compose something beautiful that goes with the natural sounds around me. Then I load those sounds up – I can then trigger them when I hear something in the right key, so a motorbike going past in B flat will mean I trigger my ‘B flat, traffic’ piano composition. The main problem is that the laptop gets really hot, also I’m covered in wires so it looks a little strange.

And this is what glitching sounds like – some of these examples have music in, others don’t.

The tech is still very much hacked together, but there’s more documented in the talk.

 

WHAT HAPPENS NEXT?

CJ, Sean and I are all enthusiastic about sharing the joys of glitching – and we’re all working on versions of glitching devices. We’re hoping to create a resource online for anyone interested to play with the idea in their own way.  I’m going to list everything I use in my hacked-together inelegant solution in another post.

I want an app that does this! I want to create a glitchpad! Beautiful musical stems that trigger automatically when friends walk through that city with this app! I want to be invited to perform ‘glitching’ concerts in cities around the world!

(for reference, I’ve reposted the TEDxTokyo video here)

More on this story as it unfolds….

Music Messing with our Heads

I recently devoured a great book called “Quirkology” by Prof Richard Wiseman, whose experiment measuring how fast people walk in different cities around the world showed (unsurprisingly) that the pace of walking has got faster (to find out which cities were faster than others, and other weird and wonderful experiments, buy the book!)

This increase in pace mirrors an increase in musical pitch over the last few centuries – as the pace of life gets faster, Middle C gets higher! In Baroque times, (around 1700) Middle C was a full semitone lower – this is why I have to transpose in my head when I play baroque-pitch harpsichords.

When I’d go dancing, it would distress me musically as the sound systems would regularly play music at about 5-6Hz higher than it should have been – I wondered whether this pitch-shift was intentional, and people would unwittingly dance more / drink more as a consequence of this increase, or whether the sound systems were just rubbish, no-one was experimenting, and only freaks like me would suffer?

My freakiness is Perfect Pitch, a strange affliction/gift that means I can correctly identify notes, chords etc., and tell someone what key they are speaking in – where it starts to get a little strange is that I’ve found that people who speak in, say, F major, appear to be quite persuasive and good at motivating, whereas people who speak in B minor appear to be quite negative in their outlook – I’ll go out on a further limb here, and mention that everyone seems to have a key they normally speak in, and others that they modulate to depending on their situation/company/mood – this is something I’ve done since I was a kid, but last weekend at the Food 2.0 wrap party, I mentioned that someone was speaking in Bb major, which resulted in strange looks and a request to blog about it, hence the post.

…but one digresses (as usual). Continuing the pacing theme, in the 90s, music at 135bpm was considered ridiculously fast, however, in the noughties, we happily imbibe 160bpm without missing a beat (no pun intended) – there’s not that much more room in terms of tempo, (before it becomes pitch) so what happens next?

How fascinating that music affects us so deeply!  During (and after) my music degree, I performed some (very) empirical research. As a lifelong insomniac, I wanted to find out a way to get to sleep easily. The relaxation tapes I purchased were fine in terms of the NLP-type hypnotic language used etc, however, the background music kept me awake!  After reading all kinds of weird and wonderful research that music at 60bpm, the average resting heart rate, can sometimes have a calming effect on the body, I decided to test that out by composing – and engineered music that would relax me by using this tempo and also choosing the keys that I personally found calming.  Well, it worked on me, because I fell asleep writing it, and had to compose some of the stuff in double time (how frustrating).  What was even stranger was that it appeared to work on other people, too…

I’d be very interested to hear any thoughts on music and how it affects humans (or other animals – I remember New Scientist running a piece about chickens listening to Pink Floyd) – and I’ll put some samples  up online soon (will blog with link) so you can have a listen.   LJ x