Speaking as a passionate ambassador for the power of audio, one very positive side-effect of the rise in VR is that it’s finally got people talking about sound again. All too often in the past I have heard even those directly involved in audio every day utter the words: “There’s nothing new in sound.”
Well, whether you believed that or not, now, with the advent of 3D sound and binaural, it’s firmly on the agenda and the creative industry is finally listening.
However, this new opportunity has, inevitably, fallen – to some extent – into buzzword territory. People like the sound (to excuse the pun) of binaural, and they want a bit of it, but they don’t really know what it is. For example, I recently heard about a client who was looking to create a 3D sound effect to simulate an interior of a car. A car that had been designed to be as near to silent as possible. Can you spot the problem?
Binaural recordings are almost exact copies of the sound waves as they go into your ears. The best binaural recordings – and there are a lot of rubbish ones – give a startling sense of reality and immersion. However there is never a clearly discernible frontal sound field (i.e. because without visual clues we can only perceive binaural sounds as being behind us – more of that later).
So what is this shiny new thing all about? To explain that I will go beyond the latest advances in technology and give you a short lesson in the anatomy of the human brain.
We as human beings are fantastic survivalists, and our eyes and ears are essential parts of that survivalist capability. In the past, when we thought about watching a film, we imagined a little guy in our heads, sitting in an auditorium, beholding the vast spectacle in front of him. Now, the assumption has become that VR is that very same guy, walking around a planetarium and experiencing the spectacle in glorious 3D vision and sound. Sadly, both of these examples are nonsense.
Every person on this planet today is the result of all the people who died before them and a living example of Darwinian evolution. Our brains – and their use of hearing and vision – have become optimised for survival, using aural and visual ‘clues’.
Imagine that a person from 1,000 years ago stepped into a time machine and was transported to Trafalgar Square in 2016. At first they would be scared because every sight and sound would be alien to them and it would take a while for the brain to rationalise and localize the sounds. But then their cognition of the reality they found themselves in would adapt rather quickly – because the brain is an adaptive machine.
When talking about sound in 3D, people ignore the fact that it needs to be considered as part of this fantastic system. If you look at how the brain actually works, you can get interesting insights. Vision and sound are two separate systems, integrated in the brain, with a hierarchy that’s been perfected over time.
The conundrum for the human survivalist is that you can’t always see the things you can hear. Vision is where you are looking and sound is everywhere around you. It’s the good, old, scary story of “It’s behind you!”
Every second we are processing information through our ears. Our brains take this information and evaluate the probability of the sound source being in front of, or behind, us. If we receive visual confirmation (for example, a person standing in front of us with their mouth moving) then our brain tells us the sound is in front of us. It’s not purely visual though. The sound is presented to the decision center and thinks: “What’s my best guess? If I make a bad guess it could all go horribly wrong.” This is high-stakes decision-making going on inside our heads. The location and direction of Audio and Visual cues are being taken and matched together, using probabilities and previous experiences for reference.
In this way, there is no such thing as 3D sound – there is a 3D sound field but we take that apart and reassemble the parts on a ‘best guess’ basis. Our brains can perceive distance and height of sounds but aren’t able to determine instantly and absolutely whether they come from in front or behind. If they don’t receive a visual clue that the sound source is in front, the brain invariably makes us perceive the sound as behind (again, it’s a case of best – or safest – guess).
So what’s the future for the new sound technologies such as binaural, ambisonics and wavefield recreation within VR? Well my guess is that these emerging new technologies will filter across into the current ‘stereo style’ and other workflows, giving enhanced immersive sound for you in your front room, from your TV, and a greatly enriched sound experience on headphones.
Perhaps, as with my father – who could once only appreciate sounds in mono, but eventually learned to ‘hear’ stereo, rather than just two mono sounds – we will continue to train our brains to adapt to hearing in a completely different way. If that becomes a reality then the impact of these new technologies could be a great deal more monumental than anyone anticipated.
Sound is exciting again!
