Virtual Reality Dynamic Positional Audio Experimental Sound Sculpture
11.30.2014 § 3 Comments
This is a little something I whipped up in one day. You will need the Oculus Rift DK2 to experience it.
You will find yourself floating invisibly inside an icosahedron, listening to some strange music. Look at different triangular faces to change the tuning of the instruments. Each of the twenty faces is associated with a different Hexany, one for each combination of three of the first six odd numbers past one, including one as the fourth number ([1,3,5,7], [1,3,5,9], [1,3,5,11], [1,3,5,13], [1,3,7,9], … [1,9,11,13]). Every hexany consists of six pitches, and these are mapped to the six different instrument timbres, which are mapped to the six cardinal directions (above, below, left, right, before, behind).
This project was inspired in part by a demo Total Cinema 360 gave at the first VR Cinema meetup. In this demo, you found yourself inside three virtual realities at once (arranged as trienspheres): look ahead to be flying over a volcano, look behind and to the right to be in a living room hanging out with some folks, and look behind and to the left to be on stage of a rock concert. The kicker is that when you’re looking into any one of these three worlds, the sound surrounding you is of that world. In other words, look into the volcano and you can hear the volcano behind you, even though you know that if you look back there you will see a room or a concert and the sound of the volcano will disappear. Thus an element of sight has been lent to sound: you can only hear things in a line extending from the front of your head, rather than the normal state of being able to hear everything around you no matter where you look. This is what I refer to as Dynamic Positional Audio.
There’s a lot left to improve on this, and hopefully I will be able to get to it soon. I would use more distinct tunings, maybe have a continuum between them, maybe have the sound sources in motion (in response to your motion even maybe), and just make it more beautiful to look at and hear. But hey, I’m a beginner. Ship.
I used Different Methods’s SpatialAudio Unity plugin after much hunting and attempts to do it myself using a MaxMSP+Unity integration.