Apple’s AR headset or “Apple Glass” smart glasses can deliver realistic sound effects to accompany its digital imagery over a real-world view, accurately reproducing sound to match where the sound is coming from in the world. 3D space.
Gaming and virtual reality systems all provide some level of audio feedback to the user to accompany the visuals. In the case of augmented reality, the experience of seeing a digitally created element in the real world view could easily be accentuated by a sound effect when it interacts in some way.
While audio can help sell the video illusion, there are some limitations. A typical stereo headphone setup can provide limited left and right differentiation at a basic level, while virtual sound systems are able to simulate 3D positions by changing the properties of the sound effect to mimic distortions based on sound. positioning.
For example, a sound played behind the user’s head might be muffled, while that in the front might be much clearer. However, while it can be useful, the effect is not really believable as differences in physiological elements such as the ears, head and body can alter what a person expects to hear from a patient. noise in a specific position.
In a patent granted to Apple on Tuesday titled “Selecting the head-bound transfer function for binaural sound reproduction,” Apple believes it can enhance the effect of virtual 3D sound positioning by having what it calls the transfer function. head related (HRTF). It is actually a pair of acoustic filters for each ear that characterize the transmission of sound from one place in an environment to the ear.
Each HRTF varies on a number of factors, so as to match the differences between people in terms of spatial hearing expectations. Although it is relatively time-consuming or impractical to create an HRTF for an individual user using consumer electronics with an actual level of accuracy, it is possible to either use a generic “medium” HRTF or select an HRTF. which is closest to the user’s needs.
An example of a scoring matrix for a group of users knowing different HRTF parameters.
The HRTF is then used as part of a binaural simulation digital signal processing algorithm to reproduce an audio recording as binaural sound, which in turn is played through headphones.
The patent centers on the idea of selecting the HRTF closest to what is needed, reducing the database to a select few. The pruning process can include removing options that are far from basic user characteristics such as height, gender, race, and age.
Reducing the database to a smaller number increases the chances that the most correct HRTF will be used, the patent advises. After the initial settings, the system could then restrict its choice to the most generic from the reduced list, to select the most appropriate HRTF to use.
To further assist the system, it is suggested that a binaural measurement can be taken by the audio device for the user. If the device is used by a single user, the measurement should only be done once, but it would potentially offer other informationon about which HRTF would be best.
The system would look for the best match for an auditor’s HRTF, not its specific settings.
It is also suggested that by grouping users together, the massive use of different HRTFs can be controlled to help improve selection. For example, knowing that users between the ages of 20 and 25 prefer HRTFs with certain properties will mean that the system will err to those particular filters if it encounters someone matching those settings in the future.
The repository lists its inventors as Darius A. Satongar, Jonathan D. Sheaffer, Martin E. Johnson, and Peter V. Jupin.
Apple files numerous patent applications every week. While the existence of a record indicates areas of interest for the company’s research and development efforts, it does not guarantee that the concept will be used in a future product or service.
Apple’s audio processing has resulted in products like the HomePod, along with its pass-through noise cancellation feature in the AirPods Pro, but it continues to develop more ideas related to sound.
In December 2019, he was granted a patent for a “system for removing virtual audio from a listener using a crosstalk suppressor,” which performs a similar spatial audio trick by giving the impression that sound is emitted from a different location from the unit’s speakers. .
A suitable place where the patent could be applied is in “Apple Glass,” Apple’s long-talked about AR glasses. Since it is believed to use AR and would apply digital elements in a real environment for the user to see and interact with, perfecting audio playback could help make the nonexistent element more believable to the wearer. .
A related patent called “Display devices with multimodal audio” suggests that a headset could have headphones that adjust to suit the type of audio presented.