Proposal Discussion Binaural HRTF implemented in the Cobra engine

Hi there. I wanted to make this proposal to your extremely competent audio guru at Frontier Developments.

He knows what I'm talking about.

There's a duo who have developed their own binaural audio engine for Unity, so it's not a hardware challenge, but a software challenge.

It is really a game changer, and I think it's going to be mandatory for decent VR experiences of the future.. so might as well build some form of future support for it into the Cobra engine.

For those who don't know what Binaural HRTF sounds like, try listening to these two demos. Notice not only the clarity of direction, but also how things sound far away, and close by.

https://www.youtube.com/watch?v=2dzsVjn8hNc

Here, notice how the sound changes from up, down, and all the directions.
https://www.youtube.com/watch?v=lsXh5EtSXDs


Binaural audio was making an entry on the scene in 1998, but Creative Labs ruined it for all of us, for the next 15+ years they helped halt progress, and instead milked (in conjunction with affiliates) the inferior audio formats such as the fake "surround" standards known as 7.1 , True Dolby Surround, and THX.

Here is the backstory of how that happened:

http://en.wikipedia.org/wiki/A3D
Creative Labs sued Aureal for patent infringement in March 1998, and Aureal countersued for patent infringement and deceptive trade practices. Aureal won the lawsuit brought by Creative in December 1999. However, the cost of the legal battle caused Aureal's investors to cease funding operations, forcing Aureal into bankruptcy. Creative then acquired Aureal's assets in September 2000 through the bankruptcy court with the specific provision that Creative Labs would be released from all claims of past infringement by Creative Labs upon Aureal's A3D technology. Creative Labs has not chosen to support the A3D API.

In short: A hostile takeover.
 
Last edited:
Sadly this one still doesn't work for me. I get a really good experience in the rear hemisphere (side to side, top to bottom, and middle of head to behind), but absolutely nothing when the sound is coming from the front hemisphere. It just sounds flat, or if anything, like it's coming from behind.
 
Sadly this one still doesn't work for me. I get a really good experience in the rear hemisphere (side to side, top to bottom, and middle of head to behind), but absolutely nothing when the sound is coming from the front hemisphere. It just sounds flat, or if anything, like it's coming from behind.

It could be that your ears aren't shaped for this configuration. It's amazing how our different ear shapes affect the sound waves coming in, and the brain has to teach itself how to interpret those sound waves. I wouldn't lose hope that you won't get something that works though. They just have to include a method to calibrate for your individual head and ears.

There are research papers on having individual HRTF's implemented. That means you could "calibrate" it to work for your own head shape, cranium volume, and ear shape by telling a program which direction you hear the sound in different iterations.

http://www.tav.net/sonido3d/spacial_audio.pdf


Here's a document about the HRTF - Head Related Transfer Function
Check page 10 on this document for different ear shapes
•If ears are different => properties of scattered waves from them will be different.
•HRTFs will have to be individual.

http://www.umiacs.umd.edu/~ramani/cmsc828d_audio/HRTF_INTRO.pdf
 
LOL! I was giggling as a girl...

Those sounds were... tickling... my body. I kid you not.

I could feel it around the neck and down my spine! :D
It was some of te funniest feeling ever.

LOL! I love it!


EDIT: OMG! ASMR! OMGOMGOMGOMGOMGOMG!
 
Last edited:
LOL! I was giggling as a girl...

Those sounds were... tickling... my body. I kid you not.

I could feel it around the neck and down my spine! :D
It was some of te funniest feeling ever.

LOL! I love it!

Aha, then you're like me and are lucky enough to experience the lovely sensation known as ASMR. There are plenty of youtube videos made that can maximize this feeling. Search for "Binaural ASMR" on youtube. I can recommend GentleWhispers and Massage ASMR. I use them to relax and go to sleep oftentimes, if and when I've got insomnia. Such as now.
 
Last edited:
Aha, then you're like me and are lucky enough to experience the lovely sensation known as ASMR. There are plenty of youtube videos made that can maximize this feeling. Search for "Binaural ASMR" on youtube. I can recommend GentleWhispers and Massage ASMR. I use them to relax and go to sleep oftentimes, if and when I've got insomnia. Such as now.

o_O Those you just mentioned were added some 15 minutes ago to my youtube subscription.

Whoa.

The only sad thing is, after the first time, I'm slightly less sensitive...
It's... it's... so... intense... And yet still totally relaxing and not ... you know...:eek:
 
o_O Those you just mentioned were added some 15 minutes ago to my youtube subscription.

Whoa.

The only sad thing is, after the first time, I'm slightly less sensitive...
It's... it's... so... intense... And yet still totally relaxing and not ... you know...:eek:

Yeah, you do grow "immune" to ASMR after a while.. that's why I limit myself to 2 or 3 times a week. You also have to change it up, find different ASMRtists. I limit it to when I really need some help sleeping, or after a stressful day.

But to get back on topic.. I'm not looking for ASMR in Elite Dangerous :D Just insane immersion.

Imagine landing on a strange new planet, and experiencing the things with binaural audio. Or in a space station. Or.. in your cockpit. You hear a spark flying off into your ear, and you can see it in your peripheral vision - and you jump out of your chair to avoid the flying spark...
 
Last edited:
I fully approve this thread! Binaural audio will be SO important for ED (up down) especially compared to FPS games and VR generally.

I've stumbled across binaura headphone sound a few weeks ago when I was looking for "surround headphones" and all the 5.1 and 7.1 solutions are basically total crap for VR. Just get good stereo headphones. You need an audio engine that mixes the sounds in the game, or else you loose true 3D positioning and for VR it's especially important that you can tilt your head to locate audio "out of plane". For a space flight sim, up and down are very important.

Luckily there is OpenAL-Soft that supports binaural HRTF right now. OpenAL is a audio api that is similar to OpenGL and prevalent on OSX and linux games. For example you can plug in OpenAL-Soft for minecraft and enjoy HRTF audio without the game needing to change anything. I really hope and pray that FD went with OpenAL and NOT Fmod for ED :eek:
You can simply "plug in" OpenAL-Soft and configure it with different profiles. The config file support is rather nasty but you can convert different HRTF profiles that might fit your own head and ear shape better.

I'm currently working on a small config utility and a demo to easily select different HRTF files. I've converted about 50 profiles from the HRTF database for ease of use. But it's not quite finished yet :D


Sadly this one still doesn't work for me. I get a really good experience in the rear hemisphere (side to side, top to bottom, and middle of head to behind), but absolutely nothing when the sound is coming from the front hemisphere. It just sounds flat, or if anything, like it's coming from behind.

It's possible that you need good headphones or have an abnormally shaped head. Just kidding ;) But every head is different, and sound waves get dampened and phase shifted not only according to your ear shape, but also brows, nose, forehead and hair. And your skull transmits sound waves as well.


EDIT:
It could be that your ears aren't shaped for this configuration. It's amazing how our different ear shapes affect the sound waves coming in, and the brain has to teach itself how to interpret those sound waves. I wouldn't lose hope that you won't get something that works though. They just have to include a method to calibrate for your individual head and ears.

There are research papers on having individual HRTF's implemented.

Interesting! There was also a product called "MyEars" that had a calibration utility to personalize the HRTF and install a "virtual surround sound card". But it seems out of business :/ Hope it wasn't creative.

It's also possible that Perhaps one HRTF is sufficient (or even ideal) for everyone. Some guys in the OR forum tested a VR game where you would measure the response time to locate a sound and found out that he would become much better at locating sound. It's logical that your brain would be able to train itself to locate audio when your hair gets longer or your ear shape change.

Some more youtube demos:
Virtual Barber Shop
The Interrogation Chamber - Amazing Binaural 3D Sound Play (Uhm actually kind of grisly lol)

PPS: BTW Web Audio has HRTF build into chrome and mozilla browsers already. See the demos for some amazing stuff.
 
Last edited:
Wow. You seem to know far more than me about it Dejay. You're actually working on some tools! Very impressive! I'm glad to see there are more people really excited about this technology. It is fully doable!
I just hope Frontier Developments will at least consider this for the future of ED.. imagine hearing binaurally where inside your ship the shot came from.. and just knowing instinctively where the enemy is. Insane immersion. Please mention it to your sound guru Jim Croft, and let us know what you think. Maybe a possible addition for the future?

It's also possible that Perhaps one HRTF is sufficient (or even ideal) for everyone. Some guys in the OR forum tested a VR game where you would measure the response time to locate a sound and found out that he would become much better at locating sound. It's logical that your brain would be able to train itself to locate audio when your hair gets longer or your ear shape change.

I think it makes sense too! The plasticity of the human brain is more remarkable than people think. But we have to be careful not to impose a synthetic HRTF setup on a person, who will "train" his brain to function in VR-space, but then when you take off the VR and head phones, your brain is still trained to the VR HRTF.. and not your real head HRTF.. so that's why I think it's safest and wisest to have a system to adapt the HRTF to your own head, by some trial and error HRTF calibration. Not even sure if it's doable..

DeJay, do you know if that OpenAL binaural standard has a calibration system for people with different heads and ears?
 
Last edited:
Is this the same as AMD TrueAudio?

yT

Well, it's a poor implementation of it... but it seems to be based on the same principles, sure. I'm much more in favor of open standards, such as OpenAL, that's free for everyone rather than proprietary licensed systems. And I don't respect people who try to re-label old technology with their own Fancy© sounding names. It's Binaural audio, and it's based on the Head-Related-Transfer Function. Let's all use the same language, and call it "Simple Approximation of Binaural HRTF", lest we end up confusing consumers even more.

I understand that this is a part of AMD's new policy for surviving the competition with Nvidia. They have started focusing more on consoles, and they have revolutionized how drivers are written with Mantle, increasing the usage of all your resources (CPU, GPU, RAM). This seems to be in the same line of thinking as Mantle. And I'm all for getting all your resources used.

But I'm not that impressed with this demo to be honest. The demos that I posted in the OP were more impressive to my ears. Not to mention, the famous "Barbershop demo" that Dejay posted, which I'm sure everyone have heard.
 
Well I really just a script kiddie who wrote a primitive configuration utility consisting of a combo box, very simple really :eek: I just want to see how well the hrtf implementation in OpenAL-Soft works, but didn't find any demos out there. ATM my main problem is overcoming my laziness :p

I just hope Frontier Developments will at least consider this for the future of ED.. imagine hearing binaurally where inside your ship the shot came from.. and just knowing instinctively where the enemy is. Insane immersion. Please mention it to your sound guru Jim Croft, and let us know what you think. Maybe a possible addition for the future?

Yeah I've asked in the "meet the team" thread with Jim Croft. OpenAL is really very easy to use and a standard library for cross platform audio (besides commercial fmod). It's possible though that they are using fmod for other more advanced audio features like blending engine sounds or interactive music.

I think it makes sense too! The plasticity of the human brain is more remarkable than people think. But we have to be careful not to impose a synthetic HRTF setup on a person, who will "train" his brain to function in VR-space, but then when you take off the VR and head phones, your brain is still trained to the VR HRTF.. and not your real head HRTF.. so that's why I think it's safest and wisest to have a system to adapt the HRTF to your own head, by some trial and error HRTF calibration. Not even sure if it's doable..

Hehe possible! But I doubt that would happen. It would probably be more like wearing some kind of helmet and learning the sound cues for that, then taking it off and "switching". One idea was to have a kind of "exaggerated" idealized HRTF function.

DeJay, do you know if that OpenAL binaural standard has a calibration system for people with different heads and ears?

No it hasn't. I started looking into HRTF stuff because I wanted a 7.1 virtual surround headset that allows for configuration, but of course nothing :p It seems the myears software actually managed to do that though.
I initially had the idea too for a "configuration game" and solve/optimize a equation with unknown variables from those results. But from what I know, a HRTF profile consists of "impuse responses" for different elevations and azimuth angles.

It's actually a recording from in-ear microphones of white noise at different directions around the test subject. This recording is turned into a "convolution filter". I didn't know about these convolution filters, but web audio actually allows downloading a recording of an effect and turning it into a live version of that effect. This kind of stuff isn't even new apparently, but Creative rather wants to push proprietary effect DSP to the user...

There are a lot of variables in a HRTF, so I think it won't be easy to "configure" that. Hmmm... maybe you could actually combine a HRTF from different impulse responses, e.g. if the users things that the sound came from direction X/Y you take the impulse response for that direction? Or blending them.. I'm gonna ask in the OpenAL mailing list.

My plan is to let the user simple choose and try out different HRTF functions. Or "play a configuration game" and measure with which the user performs best / most accurate / fastest to locate sounds. Maybe something like fruit ninja but blindfolded ;)

There was an idea in the OR thread to actually 3D scan the users head with a kinect like setup and generate a HRTF from that :D But I don't know if that's really feasible.


Is this the same as AMD TrueAudio?

I haven't read anything about TrueAudio yet, but my guess it's more about pushing gpu's ;) I'm sure it contains binaural audio though, it's really well researched for many years. But I'm not sure if it's really necessary to accelerate sound processing anymore since CPUs have gotten so much faster. If they come up with something to raytrace sound waves to simulate occlusion dampening, reflection and reverb that could be very interesting. GSound is doing that.
 
If they come up with something to raytrace sound waves to simulate occlusion dampening, reflection and reverb that could be very interesting. GSound is doing that.

That would be insanely awesome. The Looking Glass Studios actually did a very primitive approximation of that for Thief: The Dark Project. The original Thief back in 1999 and Thief 2. None of the latter 2 Thief games have it. Thief 4 released just now had dreadful audio, horrible production at every level.
For the original Thief, they approximated the walls and corridors, so it was not the exact polygons that were propagating the sounds, but like simple "collision boxes" set up in corridors and rooms, just for the sound propagation. It was remarkable for its time.

This GSound is really intriguing to me... :D

Now, if only GSound + binaural HRTF could happen.. and be implemented in an open standard!

But.. it seems the world of corporations HATES open standards these days.
 
Last edited:
Well I think VR will put much more focus on binaural HRTF. Also any attention AMD or others give binaural audio helps too. It's kind of insane that HRTF functions are "hidden" like that in OpenAL-soft.
 
I bow to the passion people can put in these projects.
Much respect.

Plus... Shigawire and Dejay: I bow to you too for making me discover this new world.

Thank you so much.
 
Back
Top Bottom