Best Tech News Site: Clubhouse is adding spatial audio effects to make users feel like they’re really in the room

Sunday, August 29, 2021

Clubhouse is adding spatial audio effects to make users feel like they’re really in the room

It’s been a busy summer for Clubhouse. The hit social audio app rolled out new messaging features and an Android app over the last few months and now the company is turning its attention to enhancing its core audio experience. Clubhouse announced Sunday that its rooms will now be infused with spatial audio to give the app’s listeners a richer sense of hanging out live with a group of other people.

TechCrunch spoke with Clubhouse’s Justin Uberti about the decision to add spatial audio, which has the effect of making different speakers sound like they’re coming from different physical locations instead of just one spot.

Uberti joined Clubhouse in May as its head of streaming technology after more than a decade at Google where he created Google Duo, led the Hangouts team and most recently worked on Google’s cloud gaming platform Stadia. Uberti also created the WebRTC standard that Clubhouse was built on top of.

“One of the things you realize in these group audio settings is that you don’t get quite the same experience as being in a physical space,” Uberti said.

While Clubhouse and other voice chat apps bring people together in virtual social settings, the audio generally sounds relatively flat, like it’s emanating from a single central location. But at the in-person gatherings Clubhouse is meant to simulate, you’d be hearing audio from all around the room, from the left and right of a stage to the various locations in the audience where speakers might ask their questions.

To pull off the new audio tricks, Clubhouse is integrating an API from Second Life creator Philip Rosedale’s spatial audio company High Fidelity and blending it with the company’s own custom audio processing, tuned for the chat app.

High Fidelity’s HRTF technology, which stands for “Head Related Transfer Function,” maps speech to different virtual locations by subtly adding a time delay between stereo channels and replicating the way that high and low frequencies would sound entering the ear depending on a sound’s origin.

The result, long used in social VR, gives virtual social experiences a sense of physical presence that good records have been pulling off for ages. Think listening to Pink Floyd’s Dark Side of the Moon in stereo with good headphones but instead of sound effects and instruments playing around your head, you’re hearing the people you’re hanging out with arrayed in virtual space.

Skittish is what you’d get if you crossed Animal Crossing with Clubhouse

According to Uberti, Clubhouse’s implementation will be subtle, but noticeable. While the audio processing will “gently steer conversation” to put most speakers in front of the listener, Clubhouse users should have a new sense that people are speaking from different physical locations.

The new audio features will roll out Sunday to the majority of iOS users, reaching the rest of Clubhouse’s iOS and Android users within the next few weeks. The experience will be available to everyone in time, but users will also have the ability to toggle spatial audio off.

Clubhouse will use the same virtual soundstage techniques to give large rooms a sense of sounding large while making more intimate rooms sound like they’re actually happening in a smaller physical space. And because most people use headphones to participate on Clubhouse, most of the app’s users can benefit from the effects possible through two-channel stereo sound.

“You have this notion of people [being] in a space, in a room… We try to mimic the feel of how it would be in a circle with people standing around talking.”

Uberti also notes that spatial audio could give regular Clubhouse users a less obvious benefit. It’s possible that regular, non-spatialized audio in social apps contributes to the pandemic-era phenomenon of Zoom fatigue. As the human brain processes virtual audio like a phone call or group audio room, it differentiates between speakers in a different way than it would in a natural in-person setting.

“Your mind has to figure out who’s talking. Without spatial cues you have to use timbre… that requires more cognitive effort,” Uberti said. “This could actually make for a more enjoyable experience aside from more immersion.”

It’s too early to know how Clubhouse’s many subcommunities will take to the spatial audio effects, but it could enhance experiences like comedy, music and even ASMR on the app quite a bit.

“Someone tells a joke and it often feels really flat,” Uberti said. “But on Clubhouse, when you feel the laughter come from all around you, it feels a lot like a comedy club experience.”