CN118870277A - Hearing aid method, head-mounted device and computer program product with active noise reduction - Google Patents
Hearing aid method, head-mounted device and computer program product with active noise reduction Download PDFInfo
- Publication number
- CN118870277A CN118870277A CN202411137922.7A CN202411137922A CN118870277A CN 118870277 A CN118870277 A CN 118870277A CN 202411137922 A CN202411137922 A CN 202411137922A CN 118870277 A CN118870277 A CN 118870277A
- Authority
- CN
- China
- Prior art keywords
- noise reduction
- hearing aid
- unit
- sound source
- target sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 196
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004590 computer program Methods 0.000 title claims abstract description 11
- 230000005236 sound signal Effects 0.000 claims abstract description 106
- 230000007613 environmental effect Effects 0.000 claims abstract description 40
- 238000004458 analytical method Methods 0.000 claims abstract description 22
- 210000005069 ears Anatomy 0.000 claims abstract description 12
- 230000002708 enhancing effect Effects 0.000 claims abstract description 8
- 238000005070 sampling Methods 0.000 claims description 27
- 230000009466 transformation Effects 0.000 claims description 18
- 230000015572 biosynthetic process Effects 0.000 claims description 11
- 238000003786 synthesis reaction Methods 0.000 claims description 11
- 230000001965 increasing effect Effects 0.000 claims description 9
- 230000000694 effects Effects 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 5
- 230000003321 amplification Effects 0.000 claims description 4
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 4
- 208000009205 Tinnitus Diseases 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 230000002194 synthesizing effect Effects 0.000 claims description 2
- 230000000977 initiatory effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000001771 impaired effect Effects 0.000 description 5
- 208000032041 Hearing impaired Diseases 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 208000016354 hearing loss disease Diseases 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000004984 smart glass Substances 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 206010011878 Deafness Diseases 0.000 description 1
- 108010007100 Pulmonary Surfactant-Associated Protein A Proteins 0.000 description 1
- 102100027773 Pulmonary surfactant-associated protein A2 Human genes 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 210000000613 ear canal Anatomy 0.000 description 1
- 230000010370 hearing loss Effects 0.000 description 1
- 231100000888 hearing loss Toxicity 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
- H04R25/502—Customised settings for obtaining desired overall acoustical characteristics using analog signal processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/48—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using constructional means for obtaining a desired frequency response
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
- H04R2225/49—Reducing the effects of electromagnetic noise on the functioning of hearing aids, by, e.g. shielding, signal processing adaptation, selective (de)activation of electronic parts in hearing aid
Landscapes
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
The application provides a hearing aid method with active noise reduction, a head-mounted device and a computer program product. The hearing aid method with active noise reduction comprises the following steps: the active noise reduction unit reduces environmental sounds in the ears of the wearer; the hearing aid unit collects an environment image and determines whether a target sound source exists or not according to analysis of the environment image; when the target sound source exists, determining the direction of the target sound source; when a target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel; and the loudspeaker unit plays a second audio signal which is formed after being processed by the active noise reduction unit and the hearing aid unit. According to the embodiment of the application, the audio signal of the environmental sound in the ear of a wearer can be reduced, and the audio signal corresponding to the sound source signal is enhanced, so that a more silent hearing-aid environment is provided in the auditory canal, and the hearing aid can play sound in a specific direction more clearly.
Description
Technical Field
The present application relates to the field of signal processing technology, and in particular, to a hearing assistance method with active noise reduction, a head-mounted device, and a computer program product.
Background
With social progress and an increase in the living standard of people, the use of head-mounted devices such as hearing aids has become more and more popular. For example, the hearing aid (Personal Sound Amplification Product, PS AP) is a small-sized loudspeaker that is effective to compensate for hearing loss in an impaired person, amplify sounds that would otherwise not be heard by the impaired person, and then utilize the residual hearing of the impaired person to allow the sounds to be delivered to the brain-auditory center of the impaired person, which in turn allows the impaired person to perceive the sounds.
However, since the hearing-impaired person is generally insensitive to the high-frequency audio signal and has more middle-low frequency noise in the noisy environment, when the hearing aid is used in the noisy environment, the signal-to-noise ratio when the hearing aid is worn is lowered due to the fact that the middle-low frequency audio is too much heard by the hearing-impaired person, so that the speech intelligibility of the hearing aid is reduced when the hearing aid is used in the noisy environment. Meanwhile, in the scene, the hearing-aid environment of the user is not quiet enough, and higher hearing-aid gain is needed, so that the hearing of the hearing-impaired person is accelerated to be damaged, and the hearing protection of the hearing-impaired person is not facilitated. For some people with normal hearing, when wearing other head-mounted devices, it is sometimes necessary to use hearing aid functions to receive external audio. Accordingly, there is a need to provide a hearing aid method, a head-mounted device and a computer program product with active noise reduction in order to solve the above-mentioned problems.
Disclosure of Invention
The present application has been made keeping in mind at least one of the above problems occurring in the prior art. According to an aspect of the present application, there is provided a hearing aid method with active noise reduction, applied to a head-mounted device including an active noise reduction unit, a hearing aid unit, and a speaker unit, the method including:
the active noise reduction unit reduces environmental sounds in the ears of the wearer;
The hearing aid unit collects an environment image and determines whether a target sound source exists or not according to analysis of the environment image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel;
and the loudspeaker unit plays a second audio signal which is formed after being processed by the active noise reduction unit and the hearing aid unit.
In some embodiments, the start-up condition includes any one of:
the direction of the line of sight of the wearer coincides with the direction of the target sound source;
The direction of the target sound source is such that a speaker facing the wearer is present;
The wearer is in a speaking state, and a speaker exists in the direction of the target sound source;
The wearer is in a speaking state, and the direction of the line of sight of the wearer coincides with the direction of the target sound source.
In some embodiments, the active noise reduction unit reduces ambient sounds in the wearer's ear, comprising:
Determining or generating a feedforward noise reduction signal according to a noise signal acquired by the external microphone; and/or
A feedback noise reduction signal is determined or generated from the noise signal collected by the in-ear microphone.
In some embodiments, the active noise reduction unit includes a feed-forward noise reduction channel and a feedback noise reduction channel; when the environmental noise is greater than a preset threshold, increasing at least one of the following:
a first gain of the hearing aid unit;
A second gain of the feedforward noise reduction channel;
and a third gain of the feedback noise reduction channel.
In some embodiments, the hearing assistance unit enhances the first audio signal of the target sound source direction through a hearing assistance channel, comprising:
the hearing aid unit reduces the sampling rate of the target sound source acquired by the microphone;
the hearing aid unit divides the target sound source into a plurality of frequency bands to form a plurality of sub-bands;
The hearing aid unit carries out gain adjustment on each sub-band;
The hearing aid unit performs dynamic range control on each sub-band;
the hearing aid unit synthesizes a plurality of sub-bands into one path to output the second audio signal.
In some embodiments, the hearing assistance unit enhances the first audio signal of the target sound source direction through a hearing assistance channel, comprising:
the hearing aid unit reduces the sampling rate of the target sound source acquired by the microphone;
The hearing aid unit performs frequency domain transformation on the target sound source;
The hearing aid unit correspondingly attenuates the target sound source of each frequency based on the signal-to-noise ratio of the target sound source of each frequency after frequency domain transformation, and then outputs the second audio signal.
The embodiment of the application further provides a head-mounted device, which comprises an active noise reduction unit, a hearing aid unit and a loudspeaker unit; the active noise reduction unit and the hearing aid unit are connected in parallel, and are connected in series with the loudspeaker unit; wherein,
The active noise reduction unit is used for reducing environmental sounds in ears of a wearer;
the hearing aid unit is used for collecting an environment image and determining whether a target sound source exists or not according to analysis of the environment image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel;
the loudspeaker unit is used for playing the second audio signal which is formed after being processed by the active noise reduction unit and the hearing aid unit.
In some embodiments, the start-up condition includes any one of:
the direction of the line of sight of the wearer coincides with the direction of the target sound source;
The direction of the target sound source is such that a speaker facing the wearer is present;
The wearer is in a speaking state, and a speaker exists in the direction of the target sound source;
The wearer is in a speaking state, and the direction of the line of sight of the wearer coincides with the direction of the target sound source.
In some embodiments, the hearing aid unit comprises a first feedforward microphone, a first analog gain module, a first analog-to-digital converter, a first low-pass and downsampling filter, and a hearing aid module connected in sequence; the first feedforward microphone module is used for collecting environmental sounds outside ears; the first analog gain module, the first analog-to-digital converter, the first low-pass and down-sampling filter and the hearing aid module are used for noise reduction processing of the noise signal and amplification processing of the target sound signal.
In some embodiments, the hearing aid module comprises a second low-pass and downsampling filter, an analysis filter bank, a subband gain adjustment module, a subband dynamic range control module, a synthesis filter bank, and a first upsampling and low-pass filter connected in sequence; wherein,
The second low-pass and downsampling filter and the first upsampling and lowpass filter for reducing the sampling rate of the hearing aid module;
the analysis filter bank is used for dividing an input audio signal into a plurality of frequency bands to form a plurality of sub-bands;
the subband gain adjusting module is used for carrying out gain adjustment on each subband so as to achieve the effect of noise reduction on the input audio signal;
the sub-band dynamic range control module is used for carrying out dynamic range control on each sub-band;
the synthesis filter bank is used for synthesizing a plurality of sub-bands into one output audio signal.
In some embodiments, the hearing aid module comprises a third low-pass and downsampling filter, a frequency domain transform module, a noise reduction processing module, a frequency domain inverse transform module, and a second upsampling and lowpass filter connected in sequence; wherein,
The third low pass and downsampling filter and the second upsampling and lowpass filter for reducing the sampling rate of the hearing aid module;
The frequency domain transformation module and the frequency domain inverse transformation module are used for changing the frequency representation of the input audio signal;
The noise reduction processing module is used for correspondingly attenuating the audio signals of all frequencies based on the signal-to-noise ratio of the input audio signals of all frequencies after the frequency domain transformation.
In some embodiments, the active noise reduction unit includes a feed-forward active noise reduction branch and a feedback active noise reduction branch:
The feedforward active noise reduction branch is used for determining or generating a feedforward noise reduction signal according to the external noise signal;
the feedback active noise reduction branch is used for determining or generating a feedback noise reduction signal according to the in-ear noise signal.
In some embodiments, the feedforward active noise reduction branch includes a feedforward microphone module, a second analog gain module, a second analog-to-digital converter, a fourth low-pass and downsampling filter, and a feedforward active noise reduction filter connected in sequence; the feedforward microphone module is used for collecting environmental sounds outside ears; the second analog gain module, the second analog-to-digital converter, the fourth low-pass and downsampling filter and the feedforward active noise reduction filter are used for picking up sound outside the earphone.
In some embodiments, the feedback active noise reduction branch comprises a feedback microphone module, a third analog gain module, a third analog-to-digital converter, a fifth low-pass and downsampling filter and a feedback active noise reduction filter which are connected in sequence; the feedback microphone module is used for collecting environmental sounds in the auditory canal; the third analog gain module, the third analog-to-digital converter, the fifth low-pass and downsampling filter and the feedback active noise reduction filter are used for picking up environmental sounds in the earphone.
In some embodiments, the speaker unit includes a digital-to-analog converter.
In some embodiments, the active noise reduction unit and/or the hearing aid unit is connected to the speaker unit via a limiter.
In some embodiments, the active noise reduction unit and/or the hearing aid unit is connected to the speaker unit through an upsampling and filtering module.
In some embodiments, the head-mounted device further comprises an audio echo module; the audio echo module is used for sending the audio to be broadcast to the loudspeaker unit, and sending the audio to be broadcast to a feedback active noise reduction branch of the active noise reduction unit at the same time so as to offset out the audio component to be broadcast in the feedback active noise reduction branch.
In some embodiments, the hearing unit has a first delay and the active noise reduction unit has a second delay; the first delay is greater than 10 times the second delay.
In yet another aspect, embodiments of the present application provide a computer program product comprising a computer program/instruction which, when executed by a processor, implements the steps of the hearing aid method with active noise reduction described above.
According to the hearing aid method with active noise reduction, the existence of the target sound source and the direction of the target sound source are determined according to the environment image, when the starting condition is met, the environment sound audio signal in the ear of a wearer is reduced, the audio signal corresponding to the sound source signal is enhanced, and then the enhanced audio signal is sent to the loudspeaker unit for playing, so that a more silent hearing aid environment is provided in the auditory canal, and the hearing aid can play sound in a specific direction more clearly.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 shows a schematic flow chart of a hearing assistance method with active noise reduction according to an embodiment of the application;
fig. 2 shows a schematic flow chart of step S101 according to an embodiment of the application;
FIG. 3 shows a schematic flow chart of step S102 according to one embodiment of the application;
fig. 4 shows a schematic flow chart of step S102 according to another embodiment of the application;
FIG. 5 shows a schematic block diagram of a head mounted device according to an embodiment of the application;
Fig. 6 shows a schematic block diagram of a hearing aid module according to an embodiment of the application;
fig. 7 shows a schematic block diagram of a hearing aid module according to another embodiment of the application.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the embodiments of the present application, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
For head-mounted devices such as augmented Reality (Augmented Reality, AR)/Virtual Reality (VR) smart glasses, users cannot easily hear audio of the in-ear interests because the ears are blocked by the head-mounted device. For another example, for people with hearing impairment, the listening effect is not ideal because of the relatively large external noise. However, when the user turns on the active noise reduction function, the sound outside the ear including the sound of the surrounding person speaking into the user is also suppressed, and particularly when the user is playing audio such as music, the user is more likely to miss a useful speech signal such as an external useful call.
Based on at least one technical problem described above, the present application provides a hearing aid method with active noise reduction. The method is applied to a head-mounted device comprising an active noise reduction unit, a hearing aid unit and a loudspeaker unit. The method comprises the following steps: the active noise reduction unit reduces environmental sounds in the ears of the wearer; the hearing aid unit collects an environment image and determines whether a target sound source exists or not according to analysis of the environment image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel; and the loudspeaker unit plays a second audio signal which is formed after being processed by the active noise reduction unit and the hearing aid unit. According to the hearing aid method with active noise reduction, the existence of the sound source signal and the direction of the sound source signal are determined according to the environment image, when the starting condition is met, the environment sound audio signal in the ear of a wearer is reduced, the audio signal corresponding to the sound source signal is enhanced, and then the enhanced audio signal is sent to the loudspeaker unit for playing, so that a more silent hearing aid environment is provided in the auditory canal, and the hearing aid can play sound in a specific direction more clearly.
The hearing aid method with the active noise reduction function is applied to the head-mounted equipment. The headset may be glasses-like devices such as augmented Reality (Augmented Reality, AR)/Virtual Reality (VR) smart glasses, or other headsets such as hearing aids, e.g., headphones, open headphones, semi-in-ear headphones, real wireless headphones, etc.
FIG. 1 shows a schematic flow chart of a hearing assistance method with active noise reduction according to an embodiment of the application; as shown in fig. 1, a hearing assistance method 100 with active noise reduction according to an embodiment of the application may include steps S101, S102, and S103:
In step S101, the active noise reduction unit reduces ambient sounds in the ear of the wearer.
Referring to fig. 5, the head-mounted device according to the embodiment of the present application includes an active noise reduction unit, a hearing aid unit, and a speaker unit.
The hearing aid unit can comprise a first feedforward microphone, a first analog gain module, a first analog-to-digital converter, a first low-pass and downsampling filter and a hearing aid module which are connected in sequence; for example, the first feedforward microphone may include a number of left microphones and a number of right microphones. The hearing aid unit may further comprise at least one camera module (not shown) for capturing an ambient image, from which analysis it is determined whether a target sound source is present. The active noise reduction unit may include an active noise reduction unit including a feed-forward active noise reduction branch and a feedback active noise reduction branch; the feedforward active noise reduction branch comprises a feedforward microphone module, a second analog gain module, a second analog-to-digital converter, a fourth low-pass and downsampling filter and a feedforward active noise reduction filter which are connected in sequence; the feedback active noise reduction branch circuit comprises a feedback microphone module, a third analog gain module, a third analog-to-digital converter, a fifth low-pass and downsampling filter and a feedback active noise reduction filter which are connected in sequence. The feedforward microphone module can also comprise a plurality of left feedforward microphones and a plurality of right feedforward microphones; the feedback microphone module may also include a number of left feedback microphones and a number of right feedback microphones. In addition, the active noise reduction unit also comprises a left processor and a right processor. The speaker unit includes a digital-to-analog converter. Wherein the speaker unit further comprises at least one left speaker and at least one right speaker.
In some embodiments, the initial audio signals may be collected based on the left microphone and the right microphone, then the active noise reduction unit performs noise reduction processing on the left earphone and the right earphone, the hearing aid unit performs hearing aid processing on the left earphone and the right earphone, and finally the audio signals are played through the left speaker and the right speaker.
The noise reduction principle of the left ear is described as follows: and a left feedforward microphone (FF MIC) is used for realizing feedforward (FF) active noise reduction on the left ear through a feedforward (FF) active noise reduction filter. Meanwhile, a left Feedback (FB) microphone can be adopted, and Feedback (FB) on the left ear can be actively noise-reduced through a Feedback (FB) active noise-reducing channel. Active noise reduction weakens in-ear environmental sounds heard by the wearer, and improves the hearing experience of the wearer.
The feedforward (FF) active noise reduction filter may be implemented by hardware, where the delay is T2. For example, T2 is 10us or 20us. Notably, the time delay T2 should be less than 50us.
The noise reduction principle of the right ear is similar to that of the left ear, and reference may be made to the above description, and no further description is given here.
In one embodiment of the present application, as shown in fig. 2, the active noise reduction unit of step S101 reduces environmental sounds in the ear of the wearer, including step A1 and/or step A2:
In step A1, a feedforward noise reduction signal is determined or generated according to a noise signal acquired by an external microphone; and/or the number of the groups of groups,
In step A2, a feedback noise reduction signal is determined or generated from the noise signal collected by the in-ear microphone.
In the embodiment of the present application, only the feedforward noise reduction signal or the feedback noise reduction signal may be generated, and the feedforward noise reduction signal and the feedback noise reduction signal may also be generated.
The active noise reduction unit comprises a feedforward noise reduction channel and a feedback noise reduction channel; when the environmental noise is greater than a preset threshold, increasing at least one of the following:
a first gain of the hearing aid unit;
A second gain of the feedforward noise reduction channel;
and a third gain of the feedback noise reduction channel.
In some embodiments, the noise level of the environment may be determined by any of a left microphone, a feedforward microphone, a feedback microphone, increasing the active noise reduction gain (i.e., the second gain and the third gain) and/or increasing the hearing aid unit gain when the noise is greater than a preset threshold.
In some cases, the external noise level may also be determined by any of the left microphone, the right microphone, the feedforward microphone, and the feedback microphone. The environmental noise level is determined by the microphones, and can be set according to actual situations.
In other cases, the ambient noise in the target sound source direction (the direction in which the image or video captured by the camera module is directed) may be removed or reduced first when the noise level of the environment is calculated. When the environmental noise in the direction of the target sound source is removed or weakened, the noise is still larger than the preset threshold value, and the environmental noise of the wearer is noisy, and at the moment, the active noise reduction gain can be increased, so that a stronger noise reduction effect is obtained. For example, the feedforward (FF) active noise reduction gain (i.e., the second gain and the third gain) may be increased, and the Feedback (FB) active noise reduction gain may be increased, thereby eliminating the environmental noise to a greater extent. In other examples, the audio signal of the target sound source direction may also be made easier for the wearer to hear by increasing the first gain of the hearing aid unit.
In step S102, the hearing aid unit collects an environmental image, and determines whether a target sound source exists according to analysis of the environmental image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, the first audio signal of the target sound source direction is enhanced through a hearing aid channel.
Wherein the start-up condition includes any one of:
the direction of the line of sight of the wearer coincides with the direction of the target sound source;
The direction of the target sound source is such that a speaker facing the wearer is present;
The wearer is in a speaking state, and a speaker exists in the direction of the target sound source;
The wearer is in a speaking state, and the direction of the line of sight of the wearer coincides with the direction of the target sound source.
In the embodiment of the application, the hearing aid unit can be provided with the camera module to acquire the environment image and determine whether the target sound source exists according to analysis of the environment image. For example, generally, since the camera module is located in front of the wearer, the view angle of the camera module is also located in front of the wearer, it may be determined that the image or video collected by the camera module includes a target object such as a person, and particularly when the person is facing the wearer, it may be determined that the target object such as the person is a target sound source. In addition, whether the person is a target sound source may be determined based further on the distance between the person and the wearer, whether other persons are present between the person and the wearer, or the like. If the character is a target sound source, its direction relative to the wearer is the target sound source direction. When a person in this direction is facing the wearer, there is a greater likelihood that it will speak with the wearer if it is speaking. In some examples, it may be determined whether a person in that direction is speaking based on their expression recognition and mouth movement characteristics; if the person in this direction is speaking, the hearing aid module channel is opened.
In some examples, an additional set of camera modules may also be provided such that the additional set of camera modules is directed toward the wearer's eyes to determine the direction of the wearer's gaze. That is, another set of camera modules tracks the wearer's eye images to determine the direction of the wearer's gaze. And opening the hearing aid module channel when the sight line direction of the wearer coincides with the target sound source direction. At this time, there is a high possibility that the wearer is interested in the audio signal (such as a voice signal) of the direction of the target sound source.
In some examples, a Microphone (MIC) and inertial sensor or the like is used to detect if the wearer is speaking, and when the target sound source direction has been determined and the wearer is detected to be speaking, the hearing aid module channel is opened. For example, when the wearer is speaking and a person is present in the target sound source direction determined by the image or video captured by the camera module or an audio signal (e.g., a voice signal) is captured, there is a greater likelihood that the wearer is interested in the audio signal (e.g., a voice signal) from the target sound source direction determined by the image or video captured by the camera module, the hearing aid channel is opened and the audio signal enhances the audio signal from the target sound source direction determined by the image or video captured by the camera module, which is beneficial for the wearer to hear the audio signal from the target sound source direction determined by the image or video captured by the camera module.
In some examples, the hearing aid module channel may be opened according to N1 left microphones and N1 right microphones provided on the head-mounted device, and the target sound source may be positioned according to the left and right microphones, and it may be determined that there is a target sound source in the direction of the target sound source determined by the image or video, for example, there is a voice signal from the direction. This further determines that there is a greater likelihood that the user will be interested in the audio signal (such as a speech signal) from direction 1.
In addition, the embodiment of the present application detects whether or not an activation condition is satisfied, for example, whether or not a person is present in a speaking, a direction of a line of sight of the wearer, a direction of a target sound source determined by an image or video, whether or not the person is speaking, and the like. It is noted that the scene detection may also be performed when detecting whether the start-up condition is fulfilled, e.g. for detecting whether the wearer is speaking, based on the frequency and/or duration of the wearer's speaking detected over a certain period of time (e.g. a few seconds, tens of seconds, hundreds of seconds, etc.), etc.
In one embodiment of the present application, as shown in fig. 3, the hearing aid unit of step S102 enhances the first audio signal of the target sound source direction through a hearing aid channel, including steps B1, B2, B3, B4, and B5:
in step B1, the hearing aid unit reduces the sampling rate of the target sound source acquired by the microphone;
In step B2, the hearing aid unit divides the target sound source into a plurality of frequency bands to form a plurality of sub-bands;
in step B3, the hearing aid unit performs gain adjustment on each sub-band;
in step B4, the hearing aid unit performs dynamic range control on each sub-band;
in step B5, the hearing aid unit synthesizes a plurality of sub-bands to output the second audio signal.
In some embodiments, the camera module determines a target sound source direction. With N1 left microphones (i.e. feed-forward microphones of the hearing aid unit channels), the audio signal is acquired based on a lower sampling rate, enhancing part of the audio signal in the direction of the target sound source. Here, referring to the schematic diagram of the hearing aid module shown in fig. 6, the audio signal is decomposed into a plurality of sub-bands by an analysis filter bank, the sub-band gain adjustment module performs gain adjustment on each sub-band audio signal, and the sub-band Dynamic Range Control (DRC) module performs dynamic range Control on each sub-band audio signal to achieve the effect of noise reduction on each sub-band. The synthesis filter bank then re-synthesizes the audio of the plurality of sub-bands into one path (e.g., beam-forming the plurality of sub-bands into one path of signal), i.e., the second audio signal.
Finally, the processed second audio signal is passed through a hearing aid unit, and finally processed by a digital-to-analog converter (Digital to Analog Converter, DAC) and played by a left ear speaker. By means of the hearing aid unit, the wearer can hear the audio signal of the target sound source direction more clearly. In some cases, the hearing aid module implements speech noise reduction processing of the input audio.
In another embodiment of the present application, as shown in fig. 4, the hearing aid unit enhances the first audio signal of the target sound source direction through a hearing aid channel, including steps C1, C2 and C3:
in step C1, the hearing aid unit reduces the sampling rate of the target sound source acquired by the microphone;
in step C2, the hearing aid unit performs frequency domain transformation on the target sound source;
In step C3, the hearing aid unit attenuates the target sound source of each frequency based on the signal-to-noise ratio of the target sound source of each frequency after the frequency domain transformation, and then outputs the second audio signal.
In the embodiment of the application, the audio noise reduction time delay in the hearing aid unit is T1. For example, T1 may be 500us, 1ms, 2ms, 5ms, or the like. T1> T2 (as previously described, T2 is the feed forward F active noise reduction filter delay). In some cases, T1>10 x T2. In some embodiments, audio noise reduction or speech noise reduction of the hearing aid unit may be implemented by the processor.
The specific processing process can refer to fig. 7, and each module between the frequency domain transformation and the frequency domain inverse transformation is completed by a processor through a software program, so that independent hardware resources are saved, the processing of the hearing aid is more flexible, the hearing aid does not depend on a specific hardware module, and a better hearing aid effect can be realized. Meanwhile, as the active noise reduction module is turned on, the environmental sound perceived by a user is greatly reduced, and therefore, although the hearing aid module has larger time delay than the FF active noise reduction module, the audio played by the hearing aid unit channel through the digital-analog converter (Digital to Analog Converter, DAC) is less influenced by the environmental sound, and therefore, the time delay of the hearing aid module has less influence on user experience. On the other hand, the hearing aid module mainly enhances the audio signal of the target sound source direction, so that the superposition of the environmental sound on the earphone, which is different from the time delay of the hearing aid signal, has less influence on the user experience. For the hearing aid function, the same audio signals are transmitted into the ear through a hearing aid module and a loudspeaker, and are transmitted into the ear in a simple sound mode through a physical way, and the sounds with different time delays are overlapped in the ear to influence the hearing experience of a user. Therefore, the application reduces the environmental sound by actively reducing the noise, alleviating the bad hearing experience of the user in this respect.
In step S103, the speaker unit plays the second audio signal formed after the processing of the active noise reduction unit and the hearing aid unit.
According to the hearing aid method with active noise reduction, the existence of the sound source signal and the direction of the sound source signal are determined according to the environment image, when the starting condition is met, the environment sound audio signal in the ear of a wearer is reduced, the audio signal corresponding to the sound source signal is enhanced, and then the enhanced audio signal is sent to the loudspeaker unit for playing, so that a more silent hearing aid environment is provided in the auditory canal, and the hearing aid can play sound in a specific direction more clearly.
The head mounted device of the present application is described below in connection with fig. 5, wherein fig. 5 shows a schematic block diagram of a head mounted device according to an embodiment of the present application. The head mounted device 500 according to an embodiment of the present application may include: an active noise reduction unit 51 (51 a and 51 b), a hearing aid unit 52, and a speaker unit 53; wherein the active noise reduction unit 51 and the hearing aid unit 52 are connected in parallel, and the active noise reduction unit 51 and the hearing aid unit 52 are connected in series with the speaker unit 53. In addition, the hearing aid unit 52 further includes a camera module (not shown) for capturing an environmental image, and determining whether the target sound source exists according to analysis of the environmental image.
Wherein the active noise reduction unit 51 is configured to reduce environmental sounds in the ear of the wearer;
the hearing aid unit 52 is configured to acquire an environmental image, and determine whether a target sound source exists according to analysis of the environmental image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel;
The speaker unit 53 is configured to play the second audio signal that is processed by the active noise reduction unit 51 and the hearing aid unit 52.
Wherein the start-up condition includes any one of:
the direction of the line of sight of the wearer coincides with the direction of the target sound source;
The direction of the target sound source is such that a speaker facing the wearer is present;
The wearer is in a speaking state, and a speaker exists in the direction of the target sound source;
The wearer is in a speaking state, and the direction of the line of sight of the wearer coincides with the direction of the target sound source.
Referring to fig. 5, the hearing unit 52 includes a first feedforward microphone 521, a first analog gain module 522, a first analog-to-digital converter 523, a first low-pass and downsampling filter 524, and a hearing aid module 525, which are sequentially connected; the first feedforward microphone module is used for collecting environmental sounds outside ears; the first analog gain block 522, the first analog to digital converter 523, the first low pass and down sample filter 524 and the hearing aid block 525 are used for noise reduction processing of the noise signal and amplification processing of the target sound signal.
In one embodiment of the present application, as shown in fig. 6, the hearing aid module 525 includes a second low-pass and downsampling filter 601, an analysis filter bank 602, a subband gain adjusting module 603, a subband Dynamic Range Control (DRC) module 604, an synthesis filter bank 605 and a first upsampling and low-pass filter 606, which are connected in sequence.
Wherein the second low-pass and downsampling filter 601 and the first upsampling and lowpass filter 606 are used to reduce the sampling rate of the hearing aid module;
The analysis filter bank 602 is configured to divide an input audio signal into a plurality of frequency bands to form a plurality of subbands;
the subband gain adjusting module 603 is configured to perform gain adjustment on each subband, so as to achieve the effect of noise reduction on the input audio signal;
The sub-band dynamic range control module 604 is configured to perform dynamic range control on each sub-band;
the synthesis filter bank 605 is configured to synthesize a plurality of the subbands into an output audio signal.
In some embodiments, the analysis filter bank 602 in the hearing aid (Personal Sound Amplify Product, PSAP) module 525 divides the input audio signal into a plurality of sub-bands (frequency bands) and the synthesis filter bank 605 synthesizes the audio signals of the plurality of sub-bands (frequency bands) into one audio signal. Since the second low-pass and downsampling filter 601 is further disposed before the analysis filter bank 602 and a corresponding first upsampling and lowpass filter 606 is disposed after the synthesis filter bank 605, each module from the analysis filter bank 602 to the synthesis filter bank 605 operates at a lower sampling rate, and thus the computation load of the hearing aid module is reduced. For example, the sampling rate is f2, f2 is less than f1 (f 1 is the sampling rate of the feedforward active noise reduction filter 515).
Among other things, analysis filter banks, synthesis filter banks may be implemented using filters such as gamma tone filters (Gamma Tone Filter) and/or multiple sets of crossover filters (Crossover Filter).
In another embodiment of the present application, as shown in fig. 7, the hearing aid module 525 includes a third low-pass and down-sampling filter 701, a frequency domain transform module 702, a noise reduction processing module 703, a frequency domain inverse transform module 704, and a second up-sampling and low-pass filter 705, which are connected in sequence; wherein,
The third low pass and downsampling filter 701 and the second upsampling and low pass filter 705 for reducing the sampling rate of the hearing aid module;
The frequency domain transform module 702 and the frequency domain inverse transform module 704 are configured to change a frequency representation of an input audio signal;
the noise reduction processing module 703 is configured to attenuate the audio signal of each frequency correspondingly based on the signal-to-noise ratio of the input audio signal of each frequency after the frequency domain transformation.
In some embodiments, the hearing aid module 525 is provided with a third low-pass and downsampling filter 701 before the frequency domain transforming module 702, and a second upsampling and low-pass filter 705 after the frequency domain inverse transforming module 704. Such that the modules from the frequency domain transform module 702 to the frequency domain inverse transform module 704 operate at lower sampling rates. For example, the sampling rate is f2, and f2 is smaller than f1. The reduced sampling rate reduces the amount of computation by the hearing aid module 525. Here, the noise reduction processing may be obtained by performing different attenuation on each frequency signal based on the signal-to-noise ratio of each frequency after the frequency domain transformation. In other embodiments, other voice noise reduction algorithms may also be employed.
The modules from the frequency domain transformation module 702 to the frequency domain inverse transformation module 704 in the embodiment of the application can be realized by a processor through a software program, so that not only hardware resources are saved, but also the processing of the hearing aid module 525 is more flexible, and the dependence on specific hardware modules is reduced, so that better hearing aid effect is realized.
In some embodiments, the active noise reduction unit 51 includes a feed-forward active noise reduction branch 51a and a feedback active noise reduction branch 51b.
The feedforward active noise reduction branch 51a is configured to determine or generate a feedforward noise reduction signal according to the external noise signal. The feedforward active noise reduction branch 51a in fig. 5 is used for feedforward active noise reduction (ANC). The feedforward microphone (FF MIC) 511 collects audio signals, and the audio signals are processed by the second analog gain module 512, the second analog-to-digital converter (ADC) 513, the fourth low-pass and downsampling filter module 514, the feedforward (FF) active noise reduction (ANC) filter 515, and the limiter to form audio signals, which are used as one of inputs of the digital-to-analog converter (DAC) 531 in the speaker unit 53.
For example, the sampling rate of Feed Forward (FF) active noise reduction (ANC) filter 515 may be set to f1.
Referring to fig. 5, the feedback active noise reduction branch 51b is configured to determine or generate a feedback noise reduction signal according to an in-ear noise signal. The feedback microphone (FB MIC) 516 of the feedback active noise reduction branch 51b collects an audio signal, and the audio signal is processed by the third analog gain module 517, the third analog-to-digital converter (ADC) module, the fifth low-pass and downsampling filter module 519, the Feedback (FB) active noise reduction (ANC) filter 520, and the second limiter 5112 to form an audio signal, and the audio signal is used as one of inputs of the digital-to-analog converter (DAC) 531 of the speaker unit 53.
The feedforward active noise reduction branch 51a includes a feedforward microphone module 511, a second analog gain module 512, a second analog-to-digital converter 513, a fourth low-pass and downsampling filter 514, and a feedforward active noise reduction filter 515, which are sequentially connected. Wherein, the feedforward microphone module 511 is used for collecting environmental sounds outside the ear; the second analog gain block 512, the second analog-to-digital converter 513, the fourth low-pass and downsampling filter 514 and the feedforward active noise reduction filter 515 are used to pick up sounds outside of the headset.
The feedback active noise reduction branch includes a feedback microphone module 516, a third analog gain module 517, a third analog-to-digital converter 518, a fifth low-pass and downsampling filter 519, and a feedback active noise reduction filter 520, which are sequentially connected. The feedback microphone module 516 is configured to collect environmental sounds in the ear canal; the third analog gain module 517, the third analog-to-digital converter 518, the fifth low-pass and downsampling filter 519, and the feedback active noise reduction filter 520 are used to pick up ambient sounds within the headset.
Wherein the Feed Forward (FF) active noise reduction (ANC) filter 515, the Feedback (FB) active noise reduction (ANC) filter 520 may be an adaptive filter or a fixed filter. For example, an infinite impulse response filter (InfiniteImpulse Response Filter, IIR) structure; the filter structure can also be a finite length unit impulse response filter (Finite Impulse Response, FIR) structure or a filter structure mixed by IIR and FIR.
It is noted that the Feed Forward (FF) active noise reduction branch 51a and the Feedback (FB) active noise reduction branch 51b may operate simultaneously or may operate separately. A feedforward (FF) Microphone (MIC) is located outside the earphone, a Feedback (FB) Microphone (MIC) is located inside the earphone, and is close to the auditory canal when worn.
In some embodiments, the Microphone (MIC) may be a digital Microphone (MIC). In the case where the microphone is a digital microphone, the first, second, and third analog gain modules 522, 512, and 517, and the first, second, and third analog-to-digital converters 523, 513, and 518 in fig. 5 may not be provided.
In some embodiments, the speaker unit includes a digital-to-analog converter (DAC) 531.
In some embodiments, the feedforward active noise reduction branch 51a, the feedback active noise reduction branch 51b, and the hearing aid unit 52 are connected to the speaker unit 53 through a first limiter 5111, a second limiter 5112, and a third limiter 526, respectively. In some embodiments, the limiters used to connect the active noise reduction unit 51 and/or the hearing aid unit 52 may be the same type or model, or may be different types or models of limiters, which are not limited herein.
In some embodiments, the active noise reduction unit 51 and/or the hearing aid unit 52 is connected to the speaker unit 53 through an up-sampling and filtering module. That is, an up-sampling and filtering module (not shown) may be further disposed before the digital-to-analog converter (DAC) 531, and after the signals are superimposed, the signals are processed by the up-sampling and filtering module and then used as input of the digital-to-analog converter (DAC) 531.
In some embodiments, the head-mounted device further comprises an audio echo module 54; the audio echo module 54 is configured to send a to-be-broadcast audio to the speaker unit 53, and send the to-be-broadcast audio to a Feedback (FB) active noise reduction branch 51b of the active noise reduction unit 51, so as to cancel a to-be-broadcast audio component in the Feedback (FB) active noise reduction branch 51 b. When playing audio, the audio signal to be played is transferred to a digital-to-analog converter (DAC) 531 in the speaker unit 53 for playing on the one hand. On the other hand, when the Feedback (FB) active noise reduction branch 51b is turned on, the audio signal to be broadcast passes through the audio echo module 54 to cancel out the audio component to be broadcast in the Feedback (FB) active noise reduction branch 51b, so that the audio signal collected in the Feedback (FB) active noise reduction branch 51b is not affected by the audio to be broadcast. Specifically, the audio signal to be broadcast is added or subtracted by the output of the audio echo module 54 and the fifth low-pass and downsampling filter 519 in the Feedback (FB) noise reduction branch 51b, after the influence of the audio signal to be broadcast on the noise reduction path is eliminated, the remaining environmental noise is sent to the Feedback (FB) active noise reduction filter 520 and the second limiter 5112 for processing, and the audio signal formed after processing is input as one of digital-to-analog converters (DACs) 531 in the speaker unit 53.
In some embodiments, when the Feedback (FB) active noise reduction leg 51b is on, the hearing aid unit 52 outputs an audio signal to the audio echo module 54, and the signal component of the hearing aid module 525 in the Feedback (FB) noise reduction leg 51b is cancelled by the audio echo module 54, so that the audio signal collected in the Feedback (FB) noise reduction leg 51b is not affected by the hearing aid module 525 signal. In some examples, the audio signal output by the hearing aid unit 52 is superimposed with the audio to be played 55 as input to the audio echo module 54.
With continued reference to fig. 5, in the hearing aid unit 52, the first feedforward microphone 521 collects an audio signal, and the audio signal is processed by the hearing aid module 525 and the limiter through the first analog gain module 522, the first analog-to-digital converter (ADC) module 523, the first low-pass and downsampling filter module 524, and is used as one of inputs of the digital-to-analog converter (DAC) 531 in the speaker unit 53.
In some embodiments, the hearing aid unit 52 has a first delay T1 and the active noise reduction unit 51 has a second delay; the first delay is greater than 10 times the second delay T2.
The four inputs are finally integrated into one audio signal, which is sent to a digital-to-analog converter (DAC) 531 in the speaker unit 53 for playback.
In addition, each analog gain module, analog-to-digital converter, low-pass and downsampling filter, etc. in fig. 5 may be different types of devices, which are not limited herein.
According to the head-mounted device provided by the embodiment of the application, the existence of the sound source signal and the direction of the sound source signal are determined according to the environment image, when the starting condition is met, the environment sound audio signal in the ear of a wearer is reduced, the audio signal corresponding to the sound source signal is enhanced, and then the enhanced audio signal is sent to the loudspeaker unit for playing, so that a more silent hearing-aid environment is provided in the auditory canal, and the hearing aid can play sound in a specific direction more clearly.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the above illustrative embodiments are merely illustrative and are not intended to limit the scope of the present application thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the application. All such changes and modifications are intended to be included within the scope of the present application as set forth in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, e.g., the division of the elements is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another device, or some features may be omitted or not performed.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in order to streamline the application and aid in understanding one or more of the various inventive aspects, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the application. However, the method of the present application should not be construed as reflecting the following intent: i.e., the claimed application requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be combined in any combination, except combinations where the features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
Various component embodiments of the application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some of the modules according to embodiments of the present application may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present application can also be implemented as an apparatus program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present application may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
The foregoing description is merely illustrative of specific embodiments of the present application and the scope of the present application is not limited thereto, and any person skilled in the art can easily think about variations or substitutions within the scope of the present application. The protection scope of the application is subject to the protection scope of the claims.
Claims (20)
1. A hearing aid method with active noise reduction is applied to a head-mounted device, and the head-mounted device comprises an active noise reduction unit, a hearing aid unit and a loudspeaker unit; characterized in that the method comprises:
the active noise reduction unit reduces environmental sounds in the ears of the wearer;
The hearing aid unit collects an environment image and determines whether a target sound source exists or not according to analysis of the environment image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel;
and the loudspeaker unit plays a second audio signal which is formed after being processed by the active noise reduction unit and the hearing aid unit.
2. The method of claim 1, wherein the initiation condition comprises any one of:
the direction of the line of sight of the wearer coincides with the direction of the target sound source;
The direction of the target sound source is such that a speaker facing the wearer is present;
The wearer is in a speaking state, and a speaker exists in the direction of the target sound source;
The wearer is in a speaking state, and the direction of the line of sight of the wearer coincides with the direction of the target sound source.
3. The method of claim 1, wherein the active noise reduction unit reduces environmental sounds in the wearer's ear, comprising:
Determining or generating a feedforward noise reduction signal according to a noise signal acquired by the external microphone; and/or
A feedback noise reduction signal is determined or generated from the noise signal collected by the in-ear microphone.
4. The method of claim 1, wherein the active noise reduction unit comprises a feed-forward noise reduction channel and a feedback noise reduction channel; when the environmental noise is greater than a preset threshold, increasing at least one of the following:
a first gain of the hearing aid unit;
A second gain of the feedforward noise reduction channel;
and a third gain of the feedback noise reduction channel.
5. The method of claim 1, wherein the hearing assistance unit enhances the first audio signal of the target sound source direction through a hearing assistance channel, comprising:
the hearing aid unit reduces the sampling rate of the target sound source acquired by the microphone;
the hearing aid unit divides the target sound source into a plurality of frequency bands to form a plurality of sub-bands;
The hearing aid unit carries out gain adjustment on each sub-band;
The hearing aid unit performs dynamic range control on each sub-band;
the hearing aid unit synthesizes a plurality of sub-bands into one path to output the second audio signal.
6. The method of claim 1, wherein the hearing assistance unit enhances the first audio signal of the target sound source direction through a hearing assistance channel, comprising:
the hearing aid unit reduces the sampling rate of the target sound source acquired by the microphone;
The hearing aid unit performs frequency domain transformation on the target sound source;
The hearing aid unit correspondingly attenuates the target sound source of each frequency based on the signal-to-noise ratio of the target sound source of each frequency after frequency domain transformation, and then outputs the second audio signal.
7. A head-mounted device, characterized in that the head-mounted device comprises an active noise reduction unit, a hearing aid unit and a loudspeaker unit; the active noise reduction unit and the hearing aid unit are connected in parallel, and are connected in series with the loudspeaker unit; wherein,
The active noise reduction unit is used for reducing environmental sounds in ears of a wearer;
the hearing aid unit is used for collecting an environment image and determining whether a target sound source exists or not according to analysis of the environment image; when a target sound source exists, determining the direction of the target sound source; when the target sound source exists and the starting condition is met, enhancing a first audio signal of the target sound source direction through a hearing aid channel;
the loudspeaker unit is used for playing the second audio signal which is formed after being processed by the active noise reduction unit and the hearing aid unit.
8. The head-mounted device of claim 7, wherein the activation condition comprises any one of:
the direction of the line of sight of the wearer coincides with the direction of the target sound source;
The direction of the target sound source is such that a speaker facing the wearer is present;
The wearer is in a speaking state, and a speaker exists in the direction of the target sound source;
The wearer is in a speaking state, and the direction of the line of sight of the wearer coincides with the direction of the target sound source.
9. The head-mounted device of claim 7, wherein the hearing aid unit comprises a first feedforward microphone, a first analog gain block, a first analog-to-digital converter, a first low-pass and downsampling filter, and a hearing aid block connected in sequence; the first feedforward microphone module is used for collecting environmental sounds outside ears; the first analog gain module, the first analog-to-digital converter, the first low-pass and down-sampling filter and the hearing aid module are used for noise reduction processing of the noise signal and amplification processing of the target sound signal.
10. The head-mounted device of claim 7, wherein the hearing aid module comprises a second low-pass and downsampling filter, an analysis filter bank, a subband gain adjustment module, a subband dynamic range control module, a synthesis filter bank, and a first upsampling and low-pass filter connected in sequence; wherein,
The second low-pass and downsampling filter and the first upsampling and lowpass filter for reducing the sampling rate of the hearing aid module;
the analysis filter bank is used for dividing an input audio signal into a plurality of frequency bands to form a plurality of sub-bands;
the subband gain adjusting module is used for carrying out gain adjustment on each subband so as to achieve the effect of noise reduction on the input audio signal;
the sub-band dynamic range control module is used for carrying out dynamic range control on each sub-band;
the synthesis filter bank is used for synthesizing a plurality of sub-bands into one output audio signal.
11. The head-mounted device according to claim 7, wherein the hearing aid module comprises a third low-pass and downsampling filter, a frequency domain transform module, a noise reduction processing module, an inverse frequency domain transform module, and a second upsampling and low-pass filter, connected in sequence; wherein,
The third low pass and downsampling filter and the second upsampling and lowpass filter for reducing the sampling rate of the hearing aid module;
The frequency domain transformation module and the frequency domain inverse transformation module are used for changing the frequency representation of the input audio signal;
The noise reduction processing module is used for correspondingly attenuating the audio signals of all frequencies based on the signal-to-noise ratio of the input audio signals of all frequencies after the frequency domain transformation.
12. The head-mounted device of claim 7, wherein the active noise reduction unit comprises a feed-forward active noise reduction branch and a feedback active noise reduction branch:
The feedforward active noise reduction branch is used for determining or generating a feedforward noise reduction signal according to the external noise signal;
the feedback active noise reduction branch is used for determining or generating a feedback noise reduction signal according to the in-ear noise signal.
13. The head-mounted device of claim 12, wherein the feedforward active noise reduction leg comprises a feedforward microphone block, a second analog gain block, a second analog-to-digital converter, a fourth low-pass and downsampling filter, and a feedforward active noise reduction filter connected in sequence; the feedforward microphone module is used for collecting environmental sounds outside ears; the second analog gain module, the second analog-to-digital converter, the fourth low-pass and downsampling filter and the feedforward active noise reduction filter are used for picking up sound outside the earphone.
14. The head-mounted device of claim 12, wherein the feedback active noise reduction branch comprises a feedback microphone module, a third analog gain module, a third analog-to-digital converter, a fifth low-pass and downsampling filter, and a feedback active noise reduction filter connected in sequence; the feedback microphone module is used for collecting environmental sounds in the auditory canal; the third analog gain module, the third analog-to-digital converter, the fifth low-pass and downsampling filter and the feedback active noise reduction filter are used for picking up environmental sounds in the earphone.
15. The head-mounted device of claim 7, wherein the device comprises a plurality of buttons,
The speaker unit includes a digital-to-analog converter.
16. The head mounted device according to claim 7, wherein the active noise reduction unit and/or the hearing aid unit is connected to the speaker unit via a limiter.
17. The head-mounted device according to claim 7 or 16, wherein the active noise reduction unit and/or the hearing aid unit is connected to the speaker unit by an upsampling and filtering module.
18. The head-mounted device of claim 7, wherein the head-mounted device further comprises an audio echo module; the audio echo module is used for sending the audio to be broadcast to the loudspeaker unit, and sending the audio to be broadcast to a feedback active noise reduction branch of the active noise reduction unit at the same time so as to offset out the audio component to be broadcast in the feedback active noise reduction branch.
19. The head-mounted device of claim 18, wherein the device comprises a plurality of buttons,
The hearing aid unit has a first delay, and the active noise reduction unit has a second delay; the first delay is greater than 10 times the second delay.
20. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411137922.7A CN118870277A (en) | 2024-08-19 | 2024-08-19 | Hearing aid method, head-mounted device and computer program product with active noise reduction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411137922.7A CN118870277A (en) | 2024-08-19 | 2024-08-19 | Hearing aid method, head-mounted device and computer program product with active noise reduction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118870277A true CN118870277A (en) | 2024-10-29 |
Family
ID=93167396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411137922.7A Pending CN118870277A (en) | 2024-08-19 | 2024-08-19 | Hearing aid method, head-mounted device and computer program product with active noise reduction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118870277A (en) |
-
2024
- 2024-08-19 CN CN202411137922.7A patent/CN118870277A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6017825B2 (en) | A microphone and earphone combination audio headset with means for denoising proximity audio signals, especially for "hands-free" telephone systems | |
| JP3210494B2 (en) | Hearing assistance device, noise suppression device, and feedback suppression device having convergent adaptive filter function | |
| DK2899996T3 (en) | Signal enhancement using wireless streaming | |
| US8611554B2 (en) | Hearing assistance apparatus | |
| EP3593349B1 (en) | System and method for relative enhancement of vocal utterances in an acoustically cluttered environment | |
| CN113630684B (en) | Earphone with active noise reduction function and noise reduction method thereof | |
| JP2003520469A (en) | Noise reduction apparatus and method | |
| CN111432318B (en) | Hearing device comprising direct sound compensation | |
| CN111683319A (en) | Call pickup noise reduction method, earphone and storage medium | |
| JP2020512710A (en) | Speech intelligibility improvement system | |
| US10204637B2 (en) | Noise reduction methodology for wearable devices employing multitude of sensors | |
| CN111629313B (en) | Hearing device comprising loop gain limiter | |
| CN114501211A (en) | Active noise reduction circuit with liyin permeability, method, equipment and storage medium | |
| CN113038315A (en) | Voice signal processing method and device | |
| CN113038318A (en) | Voice signal processing method and device | |
| CN118870277A (en) | Hearing aid method, head-mounted device and computer program product with active noise reduction | |
| JP5249431B2 (en) | Method for separating signal paths and methods for using the larynx to improve speech | |
| CN115866474A (en) | Transparent transmission noise reduction control method and system of wireless earphone and wireless earphone | |
| TWI740374B (en) | Method for eliminating specific object voice and ear-wearing audio device using same | |
| CN115398934A (en) | Method, device, earphone and computer program for actively suppressing occlusion effect when reproducing audio signals | |
| US20240334125A1 (en) | Audio processing based on target signal-to-noise ratio | |
| CN115668370A (en) | Voice detector of hearing device | |
| WO2024127986A1 (en) | Speech processing system, speech processing method, and program | |
| WO2024205944A1 (en) | Audio processing based on target signal-to-noise ratio | |
| CN113347519B (en) | Method for eliminating specific object voice and ear-wearing type sound signal device using same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |