A new interactive experience from BBC R&D and the BBC Philharmonic aims to give classical music fans the taste of what it might be like to conduct their own Prom.

 Using innovative Pick A Part technology on the BBC Taster platform, it’s possible to connect laptops, smartphones and tablets together to create a synchronised ‘orchestra’. Users can listen to a selection of pieces performed by the BBC Philharmonic, either isolating instruments on each device, or play them together to create a virtual orchestra.

Pick A Part is the latest in a series of orchestrated trial productions that started with audio drama The Vostok-K Incident in 2018 and recently saw the release of an orchestrated episode of 1927’s Decameron Nights. 

The novel concept exploits changes to the recording paradigm during lockdown. Usually orchestral music is recorded with musicians playing at the same time, but with the musicians recording their parts separately from their own homes to be edited together later, BBC R&D were able to put them back together in a way that now enables the audience to interact with those individual recordings.

  “We’ve been discussing audio orchestration with the BBC Philharmonic for a while now – it’s something that we’ve been interested in the potential of, but it’s been hard to actually put together,” Jon Francombe, senior research & development engineer, BBC R&D.

“One of the challenges is that orchestral music is normally recorded with all of the musicians playing together at the same time, while audio orchestration required object-based audio – that’s having access to all of the individual parts that make up the piece.”

 The opportunity came when the country went into lockdown earlier this year, and musicians at the BBC Philharmonic began making music from their homes. 

“Suddenly, we had unprecedented access to recordings of individual musicians playing their parts to be edited together later. So we jumped on the opportunity to see if our idea of applying device orchestration to orchestral music would work.”

 “There are so many potential applications for Pick-a-Part; this is the BBC at its very best, collaborating, innovating and sharing,” says Simon Webb, director of the BBC Philharmonic. “Whether in an educational setting exploring the inner workings of a string quartet, exploring different audio mixes of a recording, or playing along with professional musicians in your own home, this is a brilliant piece of kit.”

Pick A Part has been made using the BBC’s production tool, Audio Orchestrator. With Audio Orchestrator, it’s possible to import audio files, set up rules that determine how they’re assigned to connected devices, and export a prototype application.

To make Pick A Part, the development team modified the template application, to add some new features, including showing a picture of the selected instrument on the connected devices, automatically assigning an instrument once a device has been connected; and displaying a representation of the instruments assigned to each connected device.

Once the dev team had access to the individual instruments, precious synchronisation was required.  “The framework that we use is accurate to within around 10–20 milliseconds, but there’s sometimes additional delay added by devices, which can reach up to 200 milliseconds,” explains Francombe. “That’s OK for the types of sounds in the drama productions that we’d previously released, but not good enough for music.”

To get around this, the R&D team developed a manual calibration stage so that listeners can easily correct the time-alignment for their own devices. Danial Haddadi, an industrial placement student with BBC R&D, ran an experiment to determine how well listeners can align two devices, and what method is easiest for doing this. Based on the experimental results, he implemented the calibration tool that we used in Pick A Part. The same speech content is played from the main device and an aux device, and the listener is asked to adjust the time delay on the aux device until the two devices are in sync.

The research team responsible say they have more trial productions made with Audio Orchestrator in the pipeline, including a Hallowe’en drama with the BBC Writersroom. It’s also involved in research collaborations at the universities of Surrey and York, looking at technical and creative aspects of device orchestration.