Virtual Reality: The Future of Spatial Audio
Reality Without Visuals
What is reality without visuals? This seems like a silly question to the average person, because so many of us rely heavily on what we see around us. Visuals can influence how we perceive any object or place and are often given the most attention.
Reality Without Sound
What is reality without sound? Much of our sense of balance, direction, and safety come from sound. Because we hear with two ears, we can sense where sounds are coming from (except underwater, see Music and Sound: an Underwater Experience for more).
Human ears pick up audio in three dimensions. The common issue with a stereo audio track is that we have trouble localizing audio to create the illusion of a 3D space. Just like in real life, each ear should be receiving audio at slightly different speeds so that our brains can localize a sound to a specific direction and distance.
“The premise of VR is to create an alternate reality, but without the right audio cues to match the visuals, the brain doesn't buy into the illusion” - Engadget
Binaural Audio for VR
Binaural recordings are meant to help reproduce sounds in the way that humans hear them. When you listen to a binaural recording, you can perceive 3D space, because it plays through each ear differently. Some people swear by Binaural audio for VR to be the key to immersion.
Take a listen to binaural audio. Please use headphones for best spatial results:
Head-Related Transfer Function (HRTF)
Something I noticed while playing a VR escape room in LA is that the sound was functional but lackluster. When sound comes from in front of you in real life, the sound waves travel into your ear with a high amount of clarity. When the source is behind you, your ears dampen and reflect some of the sound quality, and the brain converts it into directional data and modifications for comprehension. In the VR headphones I have seen so far, you do not get this same directional sense because all of the sounds in the headphones are going directly into the ear.
This is called Head-Related Transfer Function.
Head-Related Transfer Function (HRTF) will be the foundation for high quality personal immersive audio in VR.
Here is a sample in case you are curious to hear what HRTF could sound like. Please use headphones for best spatial results.
The challenge is that HRTF is hard to capture, because it is unique like a fingerprint. Every person has different shaped ears and frequency ranges that they can hear. Binaural recording is a streamlined method to help get us closer to this effect, but binaural audio does not work on everyone, because everyone has different ears and HRTF responses.
Solutions
A few scientists came up with a method to study this problem: placing a microphone in someone’s ear and having it record the same sound coming from speakers surrounding it.
The company Visisonics developed a solution to test this all at once, measuring a person’s unique audio imprint within a few seconds. This could be pivotal in the future of VR audio.
Evolution
Sound can trigger our reflexes. It’s what can make us turn our heads when we hear a stick crack behind us. This innate reflex should be carried through in the VR world to have the greatest emotive impact.
Once the technology matures, techniques will need to be developed to trigger these instinctive reflexes for those experiencing VR.
Speakers vs. Headphones
The current standard in VR is to use headphones. They are compact, lightweight, and easier to control. Why not use speakers? It could be costly and occupy a great deal of physical space.
IOSONO is a company experimenting with speakers for 3D spaces. This type of implementation in a VR experience could pose some challenges, but it might be something to consider for the future.
Steemians! Please let me know what you thought of this post in the comments below, and feel free to up-vote, follow and repost if you enjoyed it!
More information:
VR Needs 3D Audio
https://www.engadget.com/2016/01/22/vr-needs-3d-audio/
What is Binaural Audio?
http://www.kallbinauralaudio.com/what-is-binaural-audio/
Very cool, our senses are amazing and we should be deeply appreciative of them... they are the original interface! (-:
I have been really into VR this year because I got the HTC Vive. It questions reality: What is a video game? WHat is an interface? I have not had time to delve into development but I saw that great phones and surround sound were immediate needs. I am interested in a Virtual Steemfest too. Do not be afraid to get in touch!