[go: up one dir, main page]

WO2022043906A1 - Assistive listening system and method - Google Patents

Assistive listening system and method Download PDF

Info

Publication number
WO2022043906A1
WO2022043906A1 PCT/IB2021/057815 IB2021057815W WO2022043906A1 WO 2022043906 A1 WO2022043906 A1 WO 2022043906A1 IB 2021057815 W IB2021057815 W IB 2021057815W WO 2022043906 A1 WO2022043906 A1 WO 2022043906A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
audio signal
individual
user
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2021/057815
Other languages
French (fr)
Inventor
Andries Louis ENSLIN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of WO2022043906A1 publication Critical patent/WO2022043906A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0272—Voice signal separating
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2205/00—Details of stereophonic arrangements covered by H04R5/00 but not provided for in any of its subgroups
    • H04R2205/041—Adaptation of stereophonic signal reproduction for the hearing impaired
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/13—Acoustic transducers and sound field adaptation in vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/15—Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/13—Aspects of volume control, not necessarily automatic, in stereophonic sound systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/11—Application of ambisonics in stereophonic audio systems

Definitions

  • This invention relates to an assistive listening system and method. More particularly, but not exclusively, this invention relates to a system and a method for adjusting the perceptual properties of individual audio components from an audio source to conform to the listener’s personal preference and requirements.
  • an audio device that may produce good results for one individual can produce poor results for another individual.
  • expensive Hi-Fi stereo headphones can appear to produce good audio quality for one individual, while the same audio effect may not be perceived well by another individual.
  • the quality of original audio input signals also significantly affects the audio output as the quality is seldomly, if ever improved by known devices and methods which results in poor quality audio output.
  • Some audio devices include an audio equalizer to enhance sound quality by equalizing the overall audio content played back through the playback devices’ speakers.
  • the equalizer affects the entire audio signal as a whole and is not able to equalize individual, mixed audio components, limiting the effectiveness of such audio equalizer. Each individual user will still perceive the audio effect differently.
  • Other audio devices such as headsets, headphones, or mobile devices, utilize volume control in which a user can adjust the volume if the output sound level is not optimal for the user.
  • Changing the volume can change the overall signal level accordingly. For example, increasing the volume can increase the overall signal level.
  • increasing the volume can increase the overall signal level.
  • the user may find that an individual audio component that is already audible becomes louder, but the individual audio component that cannot be perceived remains unperceivable.
  • Increasing the sound level further can lead to signal saturation, clipping or distortion due to the limited dynamic range of all such audio devices.
  • Programmable hearing aids include a small, low-power digital audio processing device, or digital signal processor (DSP), which locally receives an audio input from an on-board microphone, processes the audio input and outputs the audio directly to the wearer through a small speaker.
  • DSP digital signal processor
  • a DSP is specifically designed to perform the audio signal analysis and computation required to deliver the clearest sound to the user. This analysis and computation involve reshaping the audio signals using mathematical equations (algorithms).
  • a further development to increase an individual’s music perception includes assistive listening technology which involves an additional device that connects to the hearing aid.
  • the primary purpose of an assistive listening device is to improve the signal-to-noise ratio (SNR) in various acoustic environments, in order to enhance the required sound.
  • SNR signal-to-noise ratio
  • Numerous types of technologies have been introduced to help overcome some of the shortcomings of a hearing aid, usually in the form of a microphone placed closer to the sound source, which then transmits the signal wirelessly to the hearing aid or an intermediate device that is connected to the hearing aid.
  • Some of these devices include FM transmitters, Infrared, Bluetooth and induction loops, with the latter being one of the most widely adopted options.
  • a further disadvantage, particularly to individuals requiring hearing aids, is that there is no provision to isolate audio components or adjust one or more perceptual property of these components emanating from surroundings, such as various sounds in a large room or sounds emanating from outside a vehicle when travelling, for example.
  • a user is often unable to perceive sounds external of the vehicle as a result of existing audio components inside the vehicle, such as music or passenger sounds and due to the limitations of hearing aids to isolate the internal and external sounds adequately.
  • a multitrack assistive listening system which include:
  • a processor configured to: o receive the composite audio signal or the individual audio components from the audio signal source; o if a composite audio signal is received, process the received composite audio signal to isolate the one or more individual audio components; o adjust one or more perceptual property of at least one of the individual audio components based on input of a user’s preference; and o mix the at least one adjusted individual audio component to produce a remixed audio signal based on the input of the user’s preference;
  • a transmitter in communication with the processor, configured to stream the remixed audio signal to at least one output audio device associated with the user.
  • the processor may be configured to adjust the one or more perceptual property of the at least one individual audio component in real-time.
  • the processor may be configured to isolate the one or more individual audio components using a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like.
  • a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like.
  • the processor may be configured to isolate the one or more individual audio components based on a type of composite audio signal received.
  • the type of composite audio signal may be in the form of audio signals such as pre-recorded music listened to in a home environment, live music or a live theatre environment; or audio-visual signals such as a television program, music video or a movie watched in a home environment or the like with the individual audio components including any one or more of drums, bass, instruments, vocals, band, dialogue, crowd ambience, effects, room ambience, ambisonic audio and the like.
  • the composite audio signal source from which the composite audio signal is receivable may be any one or more of the following including: a media player, a television or radio broadcast, one or more microphones, one or more instruments, and a database such as an online database.
  • the one or more perceptual property which the processor is configured to adjust may include any one or more of tone (treble and bass), loudness/volume, localisation and timbre of any one or more of the individual audio components.
  • the multitrack assistive listening system to further include an input device in communication with the processor such that the input device is configured to operatively provide the user’s preference to the processor in order for the processor to adjust the one or more perceptual property of the at least one of the individual audio components.
  • the input device may be integral with the processor or be provided as a separate mobile device in communication with the processor.
  • the integral or separate input device may include a visual display and input means which allows a user to select any one of the individual audio components and adjust any one of the perceptual properties of the selected individual audio components.
  • the input device may be in the form of a mobile device such as a smartphone, tablet, laptop or the like.
  • the input device may also be embedded into a media streaming device with a user interface embedded on the media streaming device and displayed via a connected visual display.
  • the invention provides for the transmitter to either transmit I stream the remixed audio signal to at least one output audio device associated with the user either directly or via the input device to the at least one output audio device.
  • the at least one output audio device may include any one or more selected from the group comprising headphones, speakers, earphones, hearing aids and the like.
  • the hearing aid may be in the form of a cochlear implant, Behind-the-ear (BTE) hearing aid, In-the-ear (ITE) hearing aid, Bone-anchored hearing aid (BAHA), or a hybrid cochlear implant.
  • BTE Behind-the-ear
  • ITE In-the-ear
  • BAHA Bone-anchored hearing aid
  • the audio signal source from which the composite audio signals may be received is a plurality of audio input devices such as microphones positioned on a vehicle with the composite audio signal in the form of an ambisonic audio mix which replicates surround sound outside of a vehicle and at least one audio output device located inside the vehicle.
  • the processor may be configured to process the received ambisonic audio mix and isolate the individual audio components each representing the location of the input devices relative to the vehicle.
  • the processor may adjust the perceptual property of at least one of the individual audio components as per the user’s preference and stream the remixed audio signal to the output device associated with the user.
  • the output device associated with the user may be in the form of headphones, earphones, hearing aids and the like which is wearable by the user.
  • an assistive listening method comprising the steps of:
  • the step of receiving a composite audio signal may comprise receiving the composite audio signal from any one or more of the following including a media player, a television or radio broadcast, a microphone, one or more instruments, and a database including an online database.
  • the composite audio signal may include audio signals including pre-recorded or live audio; and/or audio-visual signals including a television program, music video or movie.
  • the individual audio components may include any one or more of drums, bass, instruments, vocals, band, dialogue, crowd ambience, effects, room ambience and ambisonic audio.
  • the step of isolating individual audio components from the received composite audio signal may be performed using a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like.
  • a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like.
  • Receiving the user’s input regarding a desired perceptual level of each of the individual audio components may include the step of providing an input device through which the user operatively provides the input to the processor.
  • the input device may be integral with the processor, or in the form of a mobile device, including a smartphone, tablet or laptop, in communication with the processor.
  • the remixed audio signal may be streamed to at least one output device either directly or via the input the device to the output device.
  • the invention further provides receiving the composite audio signal from a plurality of microphones positioned on a vehicle with the composite audio signal being in the form of an ambisonic audio mix which replicates surround sound outside of a vehicle.
  • the at least one output audio device is located inside the vehicle; and the individual audio components each represents the location of the input devices relative to the vehicle such that the perceptual property of at least one of the individual audio components is adjustable as per the user’s input.
  • FIG. 1 is a diagrammatic representation of a broad overview of the invention.
  • Figure 2 is a schematic illustration of the invention where it is used in a vehicle.
  • a multitrack assistive listening system in accordance with the invention, is designated generally be refence numeral 10 in Figure 1.
  • the system 10 includes an audio signal source which, through an audio input device 12 provides a composite audio signal 14.
  • the composite audio signal 14 comprises mixed/summed, or individual audio components. These individual audio components are often referred to as multitrack audio as it comprises multitrack or individual components, for example vocals, instruments and percussion.
  • the system 10 further includes a processor 16 which is in communication with the audio input device 12 and configured to receive the composite audio signal 14.
  • the processor 16 is configured to process the received composite audio signal 14 and separate I isolate the composite audio signal 14 into individual audio components 18.
  • the processor 16 uses any one or more of the known separation techniques such as computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like. Any one or more of these techniques is used to isolate the individual audio components 18 from the composite audio signal 14.
  • the individual audio components 18 which are isolated by the processer 16 can include drums, bass, instruments, vocals, band, dialogue, crowd ambience, sound effects, room ambience and the like.
  • the system 10 further includes an input device 20 which is in communication with the processor 16.
  • the input device 20 has a suitable user interface having visual display and input means through which the user enters his/her desired perceptual properties 22 of at least one of the individual audio components 18, which the user would like the processor to adjust and remix.
  • the desired perceptual properties 22 is then transmitted to the processor 16 for processing.
  • the processor 16 is configured to subsequently adjust the one or more perceptual properties of at least one of the individual audio components to match the desired perceptual properties received from the input device 20 as selected by the user.
  • the one or more perceptual properties which the processor is configured to adjust includes the tone (treble/bass), loudness/volume, localisation and/or timbre of any one of the individual audio components.
  • the input device 20 is in the form of any mobile device, such as a smart phone (not shown).
  • the smart phone is provided with appropriate software, such as an application (“app”), which allows the user to select any one of the individual audio components 18 and operatively adjust any one of the perceptual properties (for example, tone, volume, localisation, timber) of the selected individual audio components.
  • the input device 20 may be integral with the processor 16 in which instance, the processor 16 will also include a visual display and input means to allow the user to adjust the individual audio components according to preference.
  • the input device 20 may also be embedded into a media streaming device (not shown), similar to a television streamer or other media streaming devices, with the user interface embedded on the media streaming device and displayed via a connected television set or other appropriate screen.
  • the processor 16 adjusts and mixes the individual components 24 to produce a remixed audio signal 26 based on the desired perceptual properties 22 and stream the remixed audio signal 26 to an output audio device 28 via a transmitter 30.
  • the output audio device 28 can include headphones, speakers, earphones, hearing aids and the like.
  • the remixed audio signal 26 may be streamed directly to the output audio device 28 or via the input device 20 (such as the smartphone) to the output audio device 28 (in this instance the headphones/earphones of the smartphone).
  • the invention enables the user to adjust by amplifying or reducing the tone (treble/bass), loudness/volume, localisation and/or timbre of any one or more of the individual audio components 18, in real-time.
  • This allows the system 10 to provide audio output to the user which comprises adjusted individual audio components configured according to the user’s specific preference (i.e. desired perceptual properties) and provides the user with an experience tailor-made by himself/herself according to taste and/or preference.
  • the multitrack assistive listening system 10 can be used at a live concert or live music show.
  • the processor 16 is in the form of a multitrack audio device (not shown) which is operable to receive the composite audio signal, or separate predefined audio signals from various audio input devices such as a singer’s microphone, instruments, crowd microphones and the like.
  • the multitrack audio device is configured to isolate the composite audio signal from the various audio sources received via a wireless signal, such as Wi-Fi, into individual audio components such as crowd ambience, vocals, instruments, bass, drums and the like.
  • the input device 20 is also a mobile device, such as a smartphone associated with the user through which the user is operable to indicate their desired perceptual properties 22 of each individual audio component 18 which is transmitted to the multitrack audio device at the concert.
  • the multitrack audio device is configured to adjust the perceptual properties in real-time of at least one of the individual audio components and mix the adjusted individual audio components to produce a remixed audio signal 26 based on user’s desired perceptual properties 22.
  • the transmitter associated with the multitrack audio device is in the form of a Wi-Fi and/or Bluetooth transmitter (not shown) which is operable to stream the remixed audio signal 26 to the output audio device 28, such as a headphone or hearing aid associated with the user.
  • the user is thereby able to listen to a preferred edition of the live concert through their headphone or hearing aid.
  • the user is able to set the audio output which is more focussed on a specific individual audio component for example, a specific instrument or vocals, or more focussed on crowd ambience and the like; or for example to drown out the crowd for example, if the user prefers vocals, instruments and the like.
  • the user’s experience is also tailor-made according to his/her preference, but without affecting the listening experience of any other user.
  • the assistive listening system 10 can be used with a home treater or home audio system (not shown).
  • the audio input device 12 is a user’s media player, Hi-Fi or television (through means of a radio and television broadcast respectively) and an external microphone (not shown) which is in communication with the processor 16.
  • the external microphone is operable to create an audio signal which represents a room’s ambience in which the audio device is placed.
  • the room ambience audio signal and the composite audio signal from the media player, Hi-Fi or television is sent to the processor which is operable to process and isolate the individual audio components.
  • the individual audio components in this example includes drums, bass, instruments, vocals, room ambience, dialogue and the like.
  • the user operates the input device 20 to indicate their desired perceptual properties
  • the processor 16 is configured to adjust the perceptual properties of the individual audio components in real-time and mix the adjusted individual audio components to produce a remixed audio signal 26 based on the user’s desired perceptual properties 22.
  • the transmitter associated with the processor 16 is in the form of a Wi-Fi and/or Bluetooth transmitter which is operable to stream the remixed audio signal 26 to the output audio device 28, such as a headphone or hearing aid associated with the user.
  • the processor 16 is able to transmit the remixed audio signal to the media player, Hi-Fi or television allowing the remixed audio signal to be played.
  • the assistive listening system 100 can be used in a vehicle 102 where immersive binaural audio is created to replicate surround sound outside a vehicle.
  • the audio input device is in the form of four microphones, not shown, positioned at a front-left (FL) 104.1 , front-right (FR) 104.2, rear-left (RL) 104.3 and rear-right (RR) 104.4 part of the vehicle 102.
  • the composite audio signal is in the form of an ambisonic audio mix which replicates surround sound outside of the vehicle 102.
  • the processor is configured to process the received ambisonic audio mix down the binaural audio tracks as received from the four microphones into individual audio components each representing a FL, FR, RL and RR channel.
  • the user is able to select the desired perceptual property of any one of the audio components where after the processor adjusts the individual audio component to produce a remixed audio signal.
  • the user can adjust the different audio components separately, to ensure that while listening to the radio or having a conversation, they are aware simultaneously of the surroundings and any imminent danger outside of the vehicle - such as a vehicle approaching from behind or a cyclist right next to them.
  • the composite audio signal may also include an audio-visual signal.
  • the audio signal is in the form of a movie, television programme, music video, video game and the like. This enables the user to adjust the perceptual properties of any audio associated with a visual work.
  • the composite audio signal source is derived from a database, such as an online music or media store (not shown).
  • a database such as an online music or media store (not shown).
  • the audio or audio-visual content which is obtained from the music store is already isolated into the one or more individual audio components and the user is then able to adjust the individual audio components based on the user’s preference and stream these to the audio output device as described above.
  • This provides a music or media store in which a user is able to purchase music and audio-visual content with customisable audio.
  • the audio or audio-visual content may also be streamed in real-time, for example by watching a movie from the online media store.
  • the input device as defined above comprises a user interface displayed on the audio output device, such as the television set, which allows the user to provide the input to adjust the individual audio components according to preference.
  • the music and/or audio-visual content is already isolated into the individual contents and the user may then “remix” the audio according to his/her preference.
  • the distinction of this embodiment here is that the individual audio components received from the composite audio and audio-visual content is already isolated before reaching the user such that only the individual audio components reach the user.
  • the individual audio components have already been isolated by the processor at an audio production house for example, which in turn provides the separated synced audio files of the audio or audio-visual content (for example in the form of multitrack .mov, mpeg-2 or mpeg-4 files or through streaming, for example in an interleaved audio file comprising the synced audio “sterns”) to the user through the online music or media store which allows the user to then adjust the individual audio components according to preference through the suitable user interface (input device) as described above.
  • the processor at an audio production house for example, which in turn provides the separated synced audio files of the audio or audio-visual content (for example in the form of multitrack .mov, mpeg-2 or mpeg-4 files or through streaming, for example in an interleaved audio file comprising the synced audio “sterns”) to the user through the online music or media store which allows the user to then adjust the individual audio components according to preference through the suitable user interface (input device) as described above.
  • the individual audio components may be adjusted (“remixed”) by a third party and uploaded to the database prior to the user obtaining the music/audio-visual content from the music or media store such that the user receives the remixed audio signals directly from the music or media store without having to adjust the individual audio components.
  • This allows the user to receive remixed audio signals which have been adjusted by a third party based on their preference, where third parties can include famous DJ’s where users may be able to associate levels of individual audio components with different DJs.
  • the invention extends to a method of receiving composite audio signals, isolating and adjusting individual audio components and providing output of the remixed audio signals. It is accordingly asserted that the limitations, shortcoming and/or disadvantages of the prior art assistive listening systems and methods are at least partially alleviated or overcome.
  • the invention allows a user to adjust the perceptual property of at least one individual audio component in a composite audio signal and to provide output of the adjusted individual audio components based on the user’s preference and provide this to an audio output device for their enjoyment.
  • This fulfills the need of individuals to create custom and personalized auditory experiences through real-time manual control and manipulation of audio input signals to provide the desired audio output, regardless of the quality of the audio input source.
  • the invention further provides an alternative to known hearing aid devices and could be especially useful for hearing-impaired people who are using lower technology devices that might not have proper noise reduction algorithms or good sound scene classifiers.
  • the invention provides a system and a method to adjust the perceptual properties of individual audio components based on a user’s own preference with any known mobile device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

This invention relates to an assistive listening system and method. More particularly, but not exclusively, this invention relates to a system and a method for adjusting the perceptual properties of one or more individual audio components of composite audio signals, in real-time, in order to provide and stream remixed audio signals to a user based on the user's elected preference.

Description

ASSISTIVE LISTENING SYSTEM AND METHOD
INTRODUCTION TO THE INVENTION
This invention relates to an assistive listening system and method. More particularly, but not exclusively, this invention relates to a system and a method for adjusting the perceptual properties of individual audio components from an audio source to conform to the listener’s personal preference and requirements.
BACKGROUND TO THE INVENTION
The appreciation and perception of art is subjective, and its value is determined, to a certain degree, by the person looking at it. The same is relevant when different individuals listen to audio content and in particular music, with each person having a different perception of the music based on their auditory response and hearing ability. One example of this is younger people who are able to hear audio frequencies at higher frequencies than older people. As people age, or as a result of hearing damage due to exposure to loud sounds, their hearing tends to deteriorate and their auditory response and sensitivity across a range of frequencies changes.
Accordingly, an audio device that may produce good results for one individual can produce poor results for another individual. For example, expensive Hi-Fi stereo headphones can appear to produce good audio quality for one individual, while the same audio effect may not be perceived well by another individual. The quality of original audio input signals also significantly affects the audio output as the quality is seldomly, if ever improved by known devices and methods which results in poor quality audio output. Some audio devices include an audio equalizer to enhance sound quality by equalizing the overall audio content played back through the playback devices’ speakers. However, the equalizer affects the entire audio signal as a whole and is not able to equalize individual, mixed audio components, limiting the effectiveness of such audio equalizer. Each individual user will still perceive the audio effect differently. Other audio devices, such as headsets, headphones, or mobile devices, utilize volume control in which a user can adjust the volume if the output sound level is not optimal for the user. Changing the volume can change the overall signal level accordingly. For example, increasing the volume can increase the overall signal level. However, the user may find that an individual audio component that is already audible becomes louder, but the individual audio component that cannot be perceived remains unperceivable. Increasing the sound level further can lead to signal saturation, clipping or distortion due to the limited dynamic range of all such audio devices.
Another example of this is live concerts where different individuals, based on their auditory response and physical location within the venue of the live music concert or show experience the music in different ways and not according to their own personal preference. Loud or soft concert playback levels, less than optimal room acoustics and preferred musical balance all impact the appreciation and perception of the show or concert for an individual
This is especially true for individuals who suffer from hearing loss and who generally wear a hearing aid to assist their hearing. Current hearing aids are explicitly designed and optimised for speech intelligibility which results in people’s music perception being negatively affected. In an attempt to overcome this, programmable hearing aids have been used. Programmable hearing aids include a small, low-power digital audio processing device, or digital signal processor (DSP), which locally receives an audio input from an on-board microphone, processes the audio input and outputs the audio directly to the wearer through a small speaker. A DSP is specifically designed to perform the audio signal analysis and computation required to deliver the clearest sound to the user. This analysis and computation involve reshaping the audio signals using mathematical equations (algorithms). Because of the size of a typical at-ear hearing aid, audio processing power is limited and thus functionality is typically limited to just one audio processing algorithm and often a single hearing profile. Modifications to the hearing profile typically require a trip to an audiologist to connect the hearing aid to a special interface to make adjustments. An audiologist can change the variables for the fixed set of calculations, but cannot change the calculations which are built into the hardware of the DSP.
A further development to increase an individual’s music perception includes assistive listening technology which involves an additional device that connects to the hearing aid. The primary purpose of an assistive listening device is to improve the signal-to-noise ratio (SNR) in various acoustic environments, in order to enhance the required sound. Numerous types of technologies have been introduced to help overcome some of the shortcomings of a hearing aid, usually in the form of a microphone placed closer to the sound source, which then transmits the signal wirelessly to the hearing aid or an intermediate device that is connected to the hearing aid. Some of these devices include FM transmitters, Infrared, Bluetooth and induction loops, with the latter being one of the most widely adopted options. There are however no contributions made in the field of multitrack assistive listening technology which would allow for both people with normal hearing and those whose hearing is impaired to obtain an improved music perception by adjusting the individual audio components of a mixed/composite audio signal to their own preference in real-time.
A further disadvantage, particularly to individuals requiring hearing aids, is that there is no provision to isolate audio components or adjust one or more perceptual property of these components emanating from surroundings, such as various sounds in a large room or sounds emanating from outside a vehicle when travelling, for example. In respect of the latter, a user is often unable to perceive sounds external of the vehicle as a result of existing audio components inside the vehicle, such as music or passenger sounds and due to the limitations of hearing aids to isolate the internal and external sounds adequately. This invariably results in the user not perceiving crucial audio information emanating from outside the vehicle, such as a warning sound from another vehicle, which is necessary to avoid a vehicle accident.
There is accordingly a need to not only rectify audio signals, but also to allow for customisation and personalisation of audio matter, to suit an individual’s personal taste, regardless of the quality or nature of the original input audio signal.
OBJECT OF THE INVENTION
It is accordingly an object of the present invention to provide an assistive listening system and method which the applicant believes the aforementioned limitations and disadvantages may at least partially be alleviated, and/or which would provide a useful alternative for known method and systems. SUMMARY OF THE INVENTION
According to a first aspect of the invention there is provided a multitrack assistive listening system, which include:
- an audio signal source from which either a composite audio signal in which the composite audio signal has one or more individual audio components, or individual audio components is receivable;
- a processor configured to: o receive the composite audio signal or the individual audio components from the audio signal source; o if a composite audio signal is received, process the received composite audio signal to isolate the one or more individual audio components; o adjust one or more perceptual property of at least one of the individual audio components based on input of a user’s preference; and o mix the at least one adjusted individual audio component to produce a remixed audio signal based on the input of the user’s preference;
- a transmitter in communication with the processor, configured to stream the remixed audio signal to at least one output audio device associated with the user.
The processor may be configured to adjust the one or more perceptual property of the at least one individual audio component in real-time.
The processor may be configured to isolate the one or more individual audio components using a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like.
The processor may be configured to isolate the one or more individual audio components based on a type of composite audio signal received. The type of composite audio signal may be in the form of audio signals such as pre-recorded music listened to in a home environment, live music or a live theatre environment; or audio-visual signals such as a television program, music video or a movie watched in a home environment or the like with the individual audio components including any one or more of drums, bass, instruments, vocals, band, dialogue, crowd ambience, effects, room ambience, ambisonic audio and the like.
The composite audio signal source from which the composite audio signal is receivable may be any one or more of the following including: a media player, a television or radio broadcast, one or more microphones, one or more instruments, and a database such as an online database.
The one or more perceptual property which the processor is configured to adjust may include any one or more of tone (treble and bass), loudness/volume, localisation and timbre of any one or more of the individual audio components.
There is further provided for the multitrack assistive listening system to further include an input device in communication with the processor such that the input device is configured to operatively provide the user’s preference to the processor in order for the processor to adjust the one or more perceptual property of the at least one of the individual audio components. The input device may be integral with the processor or be provided as a separate mobile device in communication with the processor. The integral or separate input device may include a visual display and input means which allows a user to select any one of the individual audio components and adjust any one of the perceptual properties of the selected individual audio components. In a preferred embodiment the input device may be in the form of a mobile device such as a smartphone, tablet, laptop or the like.
The input device may also be embedded into a media streaming device with a user interface embedded on the media streaming device and displayed via a connected visual display.
The invention provides for the transmitter to either transmit I stream the remixed audio signal to at least one output audio device associated with the user either directly or via the input device to the at least one output audio device.
The at least one output audio device may include any one or more selected from the group comprising headphones, speakers, earphones, hearing aids and the like.
The hearing aid may be in the form of a cochlear implant, Behind-the-ear (BTE) hearing aid, In-the-ear (ITE) hearing aid, Bone-anchored hearing aid (BAHA), or a hybrid cochlear implant.
In an alternative embodiment, the audio signal source from which the composite audio signals may be received is a plurality of audio input devices such as microphones positioned on a vehicle with the composite audio signal in the form of an ambisonic audio mix which replicates surround sound outside of a vehicle and at least one audio output device located inside the vehicle. The processor may be configured to process the received ambisonic audio mix and isolate the individual audio components each representing the location of the input devices relative to the vehicle. There is further provided for the processor to adjust the perceptual property of at least one of the individual audio components as per the user’s preference and stream the remixed audio signal to the output device associated with the user. The output device associated with the user may be in the form of headphones, earphones, hearing aids and the like which is wearable by the user.
According to a second aspect of the invention there is provided an assistive listening method comprising the steps of:
- receiving either a composite audio signal, the composite audio signal having one or more individual audio components, or individual audio components;
- isolating individual audio components from the received composite audio signal;
- receiving a user’s input regarding a desired perceptual level of each of the individual audio components;
- adjusting the perceptual level of at least one of the individual audio components based on the user’s input;
- mixing the adjusted individual audio components to produce a remixed audio signal; and
- steaming the remixed audio signal to at least one output device.
The step of receiving a composite audio signal may comprise receiving the composite audio signal from any one or more of the following including a media player, a television or radio broadcast, a microphone, one or more instruments, and a database including an online database.
The composite audio signal may include audio signals including pre-recorded or live audio; and/or audio-visual signals including a television program, music video or movie. The individual audio components may include any one or more of drums, bass, instruments, vocals, band, dialogue, crowd ambience, effects, room ambience and ambisonic audio.
The step of isolating individual audio components from the received composite audio signal may be performed using a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like.
Receiving the user’s input regarding a desired perceptual level of each of the individual audio components may include the step of providing an input device through which the user operatively provides the input to the processor. The input device may be integral with the processor, or in the form of a mobile device, including a smartphone, tablet or laptop, in communication with the processor. The remixed audio signal may be streamed to at least one output device either directly or via the input the device to the output device.
The invention further provides receiving the composite audio signal from a plurality of microphones positioned on a vehicle with the composite audio signal being in the form of an ambisonic audio mix which replicates surround sound outside of a vehicle. The at least one output audio device is located inside the vehicle; and the individual audio components each represents the location of the input devices relative to the vehicle such that the perceptual property of at least one of the individual audio components is adjustable as per the user’s input.
BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS
The invention will now be described by way of example only with reference to the accompanying drawings wherein:
Figure 1 is a diagrammatic representation of a broad overview of the invention; and
Figure 2 is a schematic illustration of the invention where it is used in a vehicle.
The presently disclosed subject matter will now be described more fully hereinafter with reference to the accompanying figures, in which representative embodiments are shown. The presently disclosed subject matter can, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiment to those skilled in the art.
DESCRIPTION OF PREFERRED EMBODIMENT OF THE INVENTION
With reference to the drawings, in which like numerals refers to like features, a multitrack assistive listening system, in accordance with the invention, is designated generally be refence numeral 10 in Figure 1. The system 10 includes an audio signal source which, through an audio input device 12 provides a composite audio signal 14. The composite audio signal 14 comprises mixed/summed, or individual audio components. These individual audio components are often referred to as multitrack audio as it comprises multitrack or individual components, for example vocals, instruments and percussion.
The system 10 further includes a processor 16 which is in communication with the audio input device 12 and configured to receive the composite audio signal 14.
The processor 16 is configured to process the received composite audio signal 14 and separate I isolate the composite audio signal 14 into individual audio components 18. The processor 16 uses any one or more of the known separation techniques such as computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like. Any one or more of these techniques is used to isolate the individual audio components 18 from the composite audio signal 14. The individual audio components 18 which are isolated by the processer 16 can include drums, bass, instruments, vocals, band, dialogue, crowd ambience, sound effects, room ambience and the like.
The system 10 further includes an input device 20 which is in communication with the processor 16. The input device 20 has a suitable user interface having visual display and input means through which the user enters his/her desired perceptual properties 22 of at least one of the individual audio components 18, which the user would like the processor to adjust and remix. The desired perceptual properties 22 is then transmitted to the processor 16 for processing. The processor 16, in turn, is configured to subsequently adjust the one or more perceptual properties of at least one of the individual audio components to match the desired perceptual properties received from the input device 20 as selected by the user. The one or more perceptual properties which the processor is configured to adjust includes the tone (treble/bass), loudness/volume, localisation and/or timbre of any one of the individual audio components.
The input device 20 is in the form of any mobile device, such as a smart phone (not shown). In this embodiment, the smart phone is provided with appropriate software, such as an application (“app”), which allows the user to select any one of the individual audio components 18 and operatively adjust any one of the perceptual properties (for example, tone, volume, localisation, timber) of the selected individual audio components. Alternatively, the input device 20 may be integral with the processor 16 in which instance, the processor 16 will also include a visual display and input means to allow the user to adjust the individual audio components according to preference. The input device 20 may also be embedded into a media streaming device (not shown), similar to a television streamer or other media streaming devices, with the user interface embedded on the media streaming device and displayed via a connected television set or other appropriate screen.
Once the user has provided his/her desired perceptual properties 22 to the processor 16, the processor 16 adjusts and mixes the individual components 24 to produce a remixed audio signal 26 based on the desired perceptual properties 22 and stream the remixed audio signal 26 to an output audio device 28 via a transmitter 30.
The output audio device 28 can include headphones, speakers, earphones, hearing aids and the like. The remixed audio signal 26 may be streamed directly to the output audio device 28 or via the input device 20 (such as the smartphone) to the output audio device 28 (in this instance the headphones/earphones of the smartphone).
Effectively, the invention enables the user to adjust by amplifying or reducing the tone (treble/bass), loudness/volume, localisation and/or timbre of any one or more of the individual audio components 18, in real-time. This allows the system 10 to provide audio output to the user which comprises adjusted individual audio components configured according to the user’s specific preference (i.e. desired perceptual properties) and provides the user with an experience tailor-made by himself/herself according to taste and/or preference.
In one example embodiment, the multitrack assistive listening system 10 can be used at a live concert or live music show. In this example embodiment the processor 16 is in the form of a multitrack audio device (not shown) which is operable to receive the composite audio signal, or separate predefined audio signals from various audio input devices such as a singer’s microphone, instruments, crowd microphones and the like. The multitrack audio device is configured to isolate the composite audio signal from the various audio sources received via a wireless signal, such as Wi-Fi, into individual audio components such as crowd ambience, vocals, instruments, bass, drums and the like.
In this embodiment, the input device 20 is also a mobile device, such as a smartphone associated with the user through which the user is operable to indicate their desired perceptual properties 22 of each individual audio component 18 which is transmitted to the multitrack audio device at the concert. The multitrack audio device is configured to adjust the perceptual properties in real-time of at least one of the individual audio components and mix the adjusted individual audio components to produce a remixed audio signal 26 based on user’s desired perceptual properties 22. The transmitter associated with the multitrack audio device is in the form of a Wi-Fi and/or Bluetooth transmitter (not shown) which is operable to stream the remixed audio signal 26 to the output audio device 28, such as a headphone or hearing aid associated with the user. The user is thereby able to listen to a preferred edition of the live concert through their headphone or hearing aid. In this instance, the user is able to set the audio output which is more focussed on a specific individual audio component for example, a specific instrument or vocals, or more focussed on crowd ambience and the like; or for example to drown out the crowd for example, if the user prefers vocals, instruments and the like. In this respect, the user’s experience is also tailor-made according to his/her preference, but without affecting the listening experience of any other user.
In another example embodiment, the assistive listening system 10 can be used with a home treater or home audio system (not shown). In such an example embodiment the audio input device 12 is a user’s media player, Hi-Fi or television (through means of a radio and television broadcast respectively) and an external microphone (not shown) which is in communication with the processor 16. The external microphone is operable to create an audio signal which represents a room’s ambience in which the audio device is placed. The room ambience audio signal and the composite audio signal from the media player, Hi-Fi or television is sent to the processor which is operable to process and isolate the individual audio components. The individual audio components in this example includes drums, bass, instruments, vocals, room ambience, dialogue and the like.
In use, the user operates the input device 20 to indicate their desired perceptual properties
22 of at least one of the individual audio components. The processor 16 is configured to adjust the perceptual properties of the individual audio components in real-time and mix the adjusted individual audio components to produce a remixed audio signal 26 based on the user’s desired perceptual properties 22. The transmitter associated with the processor 16 is in the form of a Wi-Fi and/or Bluetooth transmitter which is operable to stream the remixed audio signal 26 to the output audio device 28, such as a headphone or hearing aid associated with the user. Alternatively, the processor 16 is able to transmit the remixed audio signal to the media player, Hi-Fi or television allowing the remixed audio signal to be played.
In yet another example embodiment, as shown in Figure 2, the assistive listening system 100 can be used in a vehicle 102 where immersive binaural audio is created to replicate surround sound outside a vehicle. In this example embodiment the audio input device is in the form of four microphones, not shown, positioned at a front-left (FL) 104.1 , front-right (FR) 104.2, rear-left (RL) 104.3 and rear-right (RR) 104.4 part of the vehicle 102. The composite audio signal is in the form of an ambisonic audio mix which replicates surround sound outside of the vehicle 102. The processor is configured to process the received ambisonic audio mix down the binaural audio tracks as received from the four microphones into individual audio components each representing a FL, FR, RL and RR channel. The processor is configured to adjust the perceptual property of at least one of the individual audio components as per the user’s preference and stream the remixed audio signal to the output device associated with the user positioned at a location indicated by reference numeral 106. When the remixed audio is played via the output device such as hearing aids or stereo headphones, the user experiences an immersive audio journey of the vehicle. This also allows for each individual audio component to be adjusted independently, without losing the immersive binaural coherency, as each channel (representing FL, FR, RL and RR) is a stereo binaural mix-down. The system 100 which is used in the vehicle also includes an internal car audio input and radio audio input (not shown) which provides an internal and radio audio component to the processor. The user is able to select the desired perceptual property of any one of the audio components where after the processor adjusts the individual audio component to produce a remixed audio signal. The user can adjust the different audio components separately, to ensure that while listening to the radio or having a conversation, they are aware simultaneously of the surroundings and any imminent danger outside of the vehicle - such as a vehicle approaching from behind or a cyclist right next to them.
It will be understood that the composite audio signal may also include an audio-visual signal. In this instance, the audio signal is in the form of a movie, television programme, music video, video game and the like. This enables the user to adjust the perceptual properties of any audio associated with a visual work.
In accordance with yet another embodiment of the invention, the composite audio signal source is derived from a database, such as an online music or media store (not shown). In this instance, the audio or audio-visual content which is obtained from the music store is already isolated into the one or more individual audio components and the user is then able to adjust the individual audio components based on the user’s preference and stream these to the audio output device as described above. This provides a music or media store in which a user is able to purchase music and audio-visual content with customisable audio. The audio or audio-visual content may also be streamed in real-time, for example by watching a movie from the online media store. In this instance, the input device as defined above comprises a user interface displayed on the audio output device, such as the television set, which allows the user to provide the input to adjust the individual audio components according to preference. In this instance, the music and/or audio-visual content is already isolated into the individual contents and the user may then “remix” the audio according to his/her preference. The distinction of this embodiment here is that the individual audio components received from the composite audio and audio-visual content is already isolated before reaching the user such that only the individual audio components reach the user. Here, the individual audio components have already been isolated by the processor at an audio production house for example, which in turn provides the separated synced audio files of the audio or audio-visual content (for example in the form of multitrack .mov, mpeg-2 or mpeg-4 files or through streaming, for example in an interleaved audio file comprising the synced audio “sterns”) to the user through the online music or media store which allows the user to then adjust the individual audio components according to preference through the suitable user interface (input device) as described above.
Alternatively, the individual audio components may be adjusted (“remixed”) by a third party and uploaded to the database prior to the user obtaining the music/audio-visual content from the music or media store such that the user receives the remixed audio signals directly from the music or media store without having to adjust the individual audio components. This allows the user to receive remixed audio signals which have been adjusted by a third party based on their preference, where third parties can include famous DJ’s where users may be able to associate levels of individual audio components with different DJs.
The invention extends to a method of receiving composite audio signals, isolating and adjusting individual audio components and providing output of the remixed audio signals. It is accordingly asserted that the limitations, shortcoming and/or disadvantages of the prior art assistive listening systems and methods are at least partially alleviated or overcome.
The invention allows a user to adjust the perceptual property of at least one individual audio component in a composite audio signal and to provide output of the adjusted individual audio components based on the user’s preference and provide this to an audio output device for their enjoyment. This fulfills the need of individuals to create custom and personalized auditory experiences through real-time manual control and manipulation of audio input signals to provide the desired audio output, regardless of the quality of the audio input source.
The invention further provides an alternative to known hearing aid devices and could be especially useful for hearing-impaired people who are using lower technology devices that might not have proper noise reduction algorithms or good sound scene classifiers. The invention provides a system and a method to adjust the perceptual properties of individual audio components based on a user’s own preference with any known mobile device.
The description is presented by way of example only in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention. The words which have been used herein are words of description and illustration, rather than words of limitation.

Claims

1 . A multitrack assistive listening system, including;
- an audio signal source from which a composite audio signal, the composite audio signal having one or more individual audio components, or individual audio components is receivable;
- a processor configured to: o receive the composite audio signal or the individual audio components from the audio signal source; o if a composite audio signal is received, process the received composite audio signal to isolate the one or more individual audio components; o adjust one or more perceptual property of at least one of the individual audio components based on input of a user’s preference; and o mix the at least one adjusted individual audio component to produce a remixed audio signal based on the input of the user’s preference; and
- a transmitter in communication with the processor, configured to stream the remixed audio signal to at least one output audio device associated with the user.
2. The multitrack assistive listening system of claim 1 , wherein the processor is configured to adjust the one or more perceptual property of the at least one individual audio component in real-time.
3. The multitrack assistive listening system of claim 1 , wherein the processor is configured to isolate the one or more individual audio components using a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like. The multitrack assistive listening system of claim 3, wherein the processor is configured to isolate the one or more individual audio components based on a type of composite audio signal received and wherein the type of composite audio signal includes: audio signals including pre-recorded or live audio; and/or audio-visual signals including a television program, music video, movie or video game; and wherein the individual audio components include any one or more of drums, bass, instruments, vocals, band, dialogue, crowd ambience, effects, room ambience and ambisonic audio. The multitrack assistive listening system of claim 4, wherein the audio signal source from which the composite audio signal or individual audio components is receivable is any one or more of the following including:
- a media player,
- a television or radio broadcast;
- a microphone,
- one or more instruments, and
- a database including an online database. The multitrack assistive listening system of claim 1 , wherein the one or more perceptual property which the processor is configured to adjust includes any one or more of the tone including treble and bass, loudness or volume, localisation and timbre of any one or more of the individual audio components.
7. The multitrack assistive listening system of claim 1 , wherein the system further includes an input device in communication with the processor and wherein the input device is configured to provide the input of the user’s preference to operatively permit the user to provide the input to the processor to adjust the one or more perceptual property of the at least one of the individual audio components.
8. The multitrack assistive listening system of claim 7, wherein the input device is integral with the processor, or wherein the input device is a mobile device in communication with the processor both having visual display and input means to provide the input of the user’s preference to the processor.
9. The multitrack assistive listening system of claim 8, wherein the mobile device includes a smartphone, tablet, or a laptop.
10. The multitrack assistive listening system of claim 7, wherein the input device is embedded into a media streaming device with a user interface embedded on the media streaming device and displayed via a connected visual display.
11. The multitrack assistive listening system of claim 10, wherein the transmitter is configured to transmit the remixed audio signal to the output audio device via the input device.
12. The multitrack assistive listening system of claim 1 , wherein the output audio device includes any one or more selected from the group comprising headphones, speakers, earphones, and hearing aids.
13. The multitrack assistive listening system of claim 12, wherein the hearing aid is in the form of a cochlear implant, Behind-the-ear (BTE) hearing aid, In-the-ear (ITE) hearing aid, Bone-anchored hearing aid (BAHA), or a hybrid cochlear implant. The multitrack assistive listening system of claim 1 , wherein the audio signal source from which the composite audio signal is received is a plurality of microphones positioned on a vehicle with the composite audio signal being in the form of an ambisonic audio mix which replicates surround sound outside of a vehicle and the at least one output audio device located inside the vehicle; and wherein the processor is configured to process the received ambisonic audio mix and isolate the individual audio components each representing the location of the input devices relative to the vehicle such that the processor is configured to adjust the perceptual property of at least one of the individual audio components as per the user’s preference and stream the remixed audio signal to the output audio device. An assistive listening method, comprising the steps of:
- receiving a composite audio signal, the composite audio signal having one or more individual audio components, or individual audio components;
- if a composite audio signal is received, isolating individual audio components from the received composite audio signal;
- receiving a user’s input regarding a desired perceptual level of each of the individual audio components;
- adjusting the perceptual level of at least one of the individual audio components based on the user’s input;
- mixing the adjusted individual audio components to produce a remixed audio signal; and
- steaming the remixed audio signal to at least one output device. The multitrack assistive listening system of claim 15, wherein the step of receiving a composite audio signal comprises receiving the composite audio signal from any one or more of the following including: a media player,
22 a television or radio broadcast;
- a microphone,
- one or more instruments, and
- a database including an online database. The assistive listening method as claimed in claim 16, wherein the composite audio signal includes: audio signals including pre-recorded or live audio; and/or audio-visual signals including a television program, music video or movie; and wherein the individual audio components include any one or more of drums, bass, instruments, vocals, band, dialogue, crowd ambience, effects, room ambience and ambisonic audio. The assistive listening method as claimed in claim 15, wherein the step of isolating individual audio components from the received composite audio signal is performed using a separation technique selected from the group comprising: computer assisted scene analysis (CASA), blind source separation (BSS), independent component analysis (ICA), frequency analysis, temporal analysis, neural network analysis and the like. The assistive listening method as claimed in claim 15, wherein the step of receiving a user’s input regarding a desired perceptual level of each of the individual audio components includes the step of providing an input device through which the user operatively provides the input to the processor. The assistive listening method as claimed in claim 19, wherein the input device is integral with the processor, or in the form of a mobile device, including a smartphone, tablet or laptop, in communication with the processor.
23 The assistive listening method as claimed in claim 19, wherein the input device is embedded into a media streaming device and wherein the step of receiving a user’s input includes the step of displaying a user interface on a visual display. The assistive listening method as claimed in claim 19, wherein the step of steaming the remixed audio signal to at least one output device includes streaming the remixed audio signal via the input the device to the output device. The assistive listening method as claimed in claim 15, wherein:
- the step of receiving the composite audio signal, includes the step of receiving the composite audio signal from a plurality of microphones positioned on a vehicle with the composite audio signal being in the form of an ambisonic audio mix which replicates surround sound outside of a vehicle; and
- wherein the at least one output audio device is located inside the vehicle; and
- wherein the individual audio components each represents the location of the input devices relative to the vehicle such that the perceptual property of at least one of the individual audio components is adjustable as per the user’s input.
24
PCT/IB2021/057815 2020-08-27 2021-08-26 Assistive listening system and method Ceased WO2022043906A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ZA2020/05330 2020-08-27
ZA202005330 2020-08-27

Publications (1)

Publication Number Publication Date
WO2022043906A1 true WO2022043906A1 (en) 2022-03-03

Family

ID=80352850

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2021/057815 Ceased WO2022043906A1 (en) 2020-08-27 2021-08-26 Assistive listening system and method

Country Status (1)

Country Link
WO (1) WO2022043906A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220129237A1 (en) * 2020-10-22 2022-04-28 Gaudio Lab, Inc. Audio signal processing method and apparatus

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070127734A1 (en) * 2003-06-30 2007-06-07 Christian Brulle-Drews Configurable information distribution system for a vehicle
US20080130924A1 (en) * 1998-04-14 2008-06-05 Vaudrey Michael A Use of voice-to-remaining audio (vra) in consumer applications
US20190385622A1 (en) * 2014-10-10 2019-12-19 Qualcomm Incorporated Signaling layers for scalable coding of higher order ambisonic audio data
WO2020128087A1 (en) * 2018-12-21 2020-06-25 Gn Hearing A/S Source separation in hearing devices and related methods

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080130924A1 (en) * 1998-04-14 2008-06-05 Vaudrey Michael A Use of voice-to-remaining audio (vra) in consumer applications
US20070127734A1 (en) * 2003-06-30 2007-06-07 Christian Brulle-Drews Configurable information distribution system for a vehicle
US20190385622A1 (en) * 2014-10-10 2019-12-19 Qualcomm Incorporated Signaling layers for scalable coding of higher order ambisonic audio data
WO2020128087A1 (en) * 2018-12-21 2020-06-25 Gn Hearing A/S Source separation in hearing devices and related methods

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220129237A1 (en) * 2020-10-22 2022-04-28 Gaudio Lab, Inc. Audio signal processing method and apparatus
US11714596B2 (en) * 2020-10-22 2023-08-01 Gaudio Lab, Inc. Audio signal processing method and apparatus

Similar Documents

Publication Publication Date Title
US10869142B2 (en) Hearing aid with spatial signal enhancement
JP5325988B2 (en) Method for rendering binaural stereo in a hearing aid system and hearing aid system
US10880659B2 (en) Providing and transmitting audio signal
US9747367B2 (en) Communication system for establishing and providing preferred audio
US9924283B2 (en) Enhanced dynamics processing of streaming audio by source separation and remixing
CN105409243B (en) Preprocessing of channelized music signals
US11457319B2 (en) Hearing device incorporating dynamic microphone attenuation during streaming
WO2010148169A1 (en) Spatial audio object coding (saoc) decoder and postprocessor for hearing aids
US9393412B2 (en) Multi-channel object-oriented audio bitstream processor for cochlear implants
EP2806661B1 (en) A hearing aid with spatial signal enhancement
CN110996143B (en) Digital television signal processing method, television, device and storage medium
CN114846817A (en) Control device, signal processing method, and speaker device
WO2022043906A1 (en) Assistive listening system and method
US11297454B2 (en) Method for live public address, in a helmet, taking into account the auditory perception characteristics of the listener
WO2024238643A1 (en) Audio processing using hearing loss data
CA3033282A1 (en) A method and a system for presenting media
Sigismondi Personal monitor systems
KR20060004529A (en) Apparatus and method for generating stereo sound
DK201370280A1 (en) A hearing aid with spatial signal enhancement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21860693

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21860693

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: OTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15/05/2024)

122 Ep: pct application non-entry in european phase

Ref document number: 21860693

Country of ref document: EP

Kind code of ref document: A1