Feature: The rise of immersive audio
Virtual reality and mobile technology are leading the way for immersive sound experiences, but a lack of standard might be holding widespread adoption back, writes Erica Basnicki.
Experiments in immersive audio have been around since the early 1930s, when Alan Blumlein figured out that listening to two audio channels was much better than one. Yet it has taken the better part of a century for it to capture the imagination, and the ears, of the masses.
Before we continue, a point of order: defining immersive audio. For now, and for the sake of this article, ‘immersive’ audio includes object-based, channel-based and scene-based audio. In other words, sound in space – regardless of how that’s achieved. The definition itself hasn’t slowed its adoption. A lack of a technical standard, however, has (but more on that later).
The confusion is partly born of a heated competition for surround sound supremacy in the cinemas between Dolby Atmos, Auro 3D and the recently released DTS-X. At least, the cinema experience is probably the one most associated with surround sound technology. Where the actual demand for immersive audio is increasing is, according to many, from another entertainment sector entirely.
Both Google and Facebook have plunged headfirst into the world of virtual reality (VR). The former with Google Cardboard, the latter having acquired Oculus in 2014. With two media giants heavily invested in its success, an explosion in VR entertainment isn’t far off.
“A lot has changed since virtual reality became a ‘thing’, all thanks to Oculus for that,” explains Varun Nair (pictured above, right), founder and VP of products at Two Big Ears. “All of a sudden immersive audio stopped being a technology that existed for the sake of it existing, or adding a bit of something extra to an experience. It became something that is ultimately very crucial: you’ve got great 3D visuals creating a sense of realism, and the audio needs to match up to it.”
Two Big Ears was founded in 2013 by Nair and CEO Abesh Thakur (pictured above, left). The company designs immersive and interactive audio tools, among them the real-time 3D audio and environmental modelling engine 3Dception.
Designed with VR and game audio applications in mind, 3Dception recently powered the audio for Björk’s groundbreaking video for Stonemilker. It’s the first official single from her most recent release, Vulnicura, shot on a desolate beach in Iceland using a 360º camera. In order to see the video, you would have had to attend one of the Rough Trade launches held in New York and London, or the exhibition at the MoMa PS1 in Long Island City.
3Dception is also in place for the ongoing retrospective of Björk’s (mid-) career held at the Museum of Modern Art in Manhattan, Songlines. As part of the exhibition, Songlines presents an avant-guard audio tour of sorts; a location-based augmented audio experience that runs off an iPod touch.
“It’s a great opportunity to show off what our technology can do,” says Nair. “We created all this spacialised audio off an iPod Touch, which isn’t exactly the most powerful device out there.”
Back on this side of the Atlantic, Heaven 17’s Martyn Ware has been in the business of immersive audio for the past 15 years. In 2000 he and former Erasure keyboardist Vince Clarke formed Illustrious, and along with it developed their own three dimensional sound technology, the 3D AudioScape surround-sound system.
As Ware explains, demand for their work remained steady until the hugely successful Tales of the Bridge in 2012, which transformed London’s Millennium Bridge into one giant 3D sound installation.
Since then, interest in Illustrious’ work has risen, and Ware is particularly excited about an upcoming project in Liverpool. From 15 May until 5 July, the Liverpool ONE shopping centre on South John Street will surround visitors with the sounds that recall the special relationship between Liverpool and New York, in celebration of the 175th anniversary of the first Cunard transatlantic sailing.
It’s a world first for outdoor shopping centres. As Ware explains: “This is a commercial opportunity, which is for the general public. This is what turns me on more than anything else; it’s not done for an ‘art’ audience or a ‘music’ audience. The exciting thing for me is that this is commissioned by a commercial entity. Now that we have this as an exemplar, it takes away the risk for a lot of other commercial organisations who might wonder ‘Will this work for the general public?’ Proof: it does work for the general public.”
As both a successful music producer, and a 3D sound enthusiast, Ware is in a somewhat unique position to assess the potential of immersive audio to cross over to the wider music industry. Björk’s VR experiments aside, can we expect more immersion from our music?
“I believe in 3D sound, and it wouldn’t be terribly difficult for us to release something in a binaural format. But you’ve got to look at it from the point of view of what is the actual benefit to Heaven 17. Yes, people might say ‘yeah, that’s cool’ but really it needs a whole swathe of people, or somebody much more popular than us to do it. Otherwise it’s just a curio,” says Ware.
“Even when Pink Floyd released stuff in the ‘70s in quadraphonic, nobody bought that. And that’s when people were buying stuff! From an artistic point of view, definitely interested. With Illustrious, we had two albums over a decade ago that were released in binaural on Mute records. I don’t remember getting any emails saying ‘You’ve done those two albums, please put some more out in that format’. It’s about creating a snowball effect. It would need somebody with a lot of marketing resources or huge popularity worldwide to make this happen, I think.”
Tool of an emerging trade
As early adopters of the new medium, both Two Big Ears and Illustrious have had to develop the immersive audio technology in-house. This is becoming less of an issue for content creators as major DAW manufacturers recognise the need for such tools.
Last year, Fairlight launched 3DAW, the company’s 3D audio workspace. Designed for film and television post-production markets, 3DAW supports object-orientated audio such as Dolby Atmos and DTS MDA, as well as fixed bus formats from 5.1, Auro-3D all the way up to NHK’s 22.2 format.
It’s a large number of formats to embrace. Surely we’d be better off with just one?
“Complicated question,” says Fairlight CTO Tino Fibaek (pictured, above). “Fixed bussing is possibly easier to get to grips with and to deliver, but object-oriented has clear advantages in speaker flexibility and being somewhat future-proofed in that it will support speaker formats that are yet to be defined.”
Fibaek is focused on the silver lining of so many competing formats – demand for immersive audio is strong, and growing stronger: “We expect the next segment to grow will be home cinema, with at least four strong delivery formats in the running. Following that we believe demand will come from live broadcast. This will be a combination of object-oriented and immersive features, offering the viewer a better sound experience with more control. Finally, we think that the 3D technology created for the above segments will begin to find home in performing arts; opera, theatre and concerts.”
Live immersive audio is an area global technology company Barco is keen to develop further. As the drivers of Auro 11.1 technology, the company upped its immersive game with the acquisition of 3D audio experts Iosono in late October 2014. Prior to the acquisition, Iosono’s spatial audio processor had already been central to the world’s first 3D opera, Neither, as well as Kraftwerk’s Catalogue 12345678 2013 concert series in Tate Modern’s Turbine Hall.
The company also had a hand in MoMA’s ‘Björk’, where another video – Black Lake – used the Iosono CORE audio processor and 49 speakers by Bowers & Wilkins to engulf a specially built room with sound.
Despite its success with live immersive sound, Brian Claypool, Barco’s senior director of strategic business development, sees momentum building in mobile technology.
“How many times have you watched movies, TV shows or played games on your smartphone or tablet in the past month? Is this different than five years ago? Of course! Consumers are more demanding of having their content where and when they want it. This presents a great opportunity to take certain innovations in audio technology and create more value to the way consumers receive and view their content.”
The Standard Issue
While companies such as Fairlight see competing formats as a sign of healthy demand, others see a lack of an open standardised format as an anchor holding back a widespread adoption of immersive audio.
“We are actually expecting the introduction of such a standard by the Society of Motion Picture & Television Engineers (SMPTE) this year and Barco is ready for it,” says Claypool. “This will put an increased focus on quality of content, so that audiences understand the full potential of what immersive sound can be. Also, content creators will be able to focus on one workflow rather than making sacrifices to create multiple versions of one mix. The arrival of a true standard in the marketplace will help achieve this.”
In terms of mobile entertainment, that standard has already arrived. In March of this year, the Audio Engineering Society established the AES69-2015 standard which, according to the website, “provides an important framework for the growing binaural and 3D personal audio industries”.
The question of a cinema standard was also addressed at the 57th AES Conference on the future of audio entertainment technology, which included a discussion chaired by SMPTE’s Brian Vessa.
“While there are many commercial solutions being offered to movie theatres, the loudspeaker layouts differ and it is believed that a common interchange format will enable cinemas to invest in immersive audio more securely, whatever the system they employ, while being able to receive content in one format that can easily be mapped to the system in question,” explains Francis Rumsey, chair of the AES Technical Council. “A lot depends on the specification of suitable renderers that can map audio to the loudspeaker layout in question.”
An August 2014 SMPTE newsletter notes that Peter Ludé, chair of the SMPTE Working Group on Immersive Sound, is “hopeful a standard will be ready or close to ready for publishing before late 2015.
“We don’t know what the actual, final standard will be yet, of course, because the work is still in progress. But the expectation is that it will be a relatively minor change from [the technology] that is in the field today. I think, for the most part, people are envisioning relatively minor software upgrades to get them from their current status to the new standard when the time comes. Everyone feels it is worth the time and expense to get to a single interoperable immersive format; that is the main reason this process is moving forward.”
Look out for a report on last month’s BBC Sound: Now and Next event, where immersive audio was a major talking point, in the June issue of Audio Media International.
http://www.bbc.co.uk/rd/events/sound2015
http://www.illustriouscompany.co.uk
Main picture: Björk’s Black Lake Room at the Museum of Modern Art in New York City, Credit: The Living
