EP1006511A1 - Sound processing method and device for adapting a hearing aid for hearing impaired - Google Patents

Abstract

The method consists in extracting (5) the parameters characterizing the pitch, the voicing, the energy and the spectrum of the speech signal, in modifying (6) the parameters to make speech intelligible to a hearing impaired person, and in reconstructing (7) a speech signal perceptible to the hearing impaired from the modified parameters. Application: hearing aids. <IMAGE>

Description

The present invention relates to a method and a device for correction of hearing impaired sounds. It applies equally to the production of hearing aids, as software executable on personal computers or answering machines and so general to all devices intended to improve listening comfort and understanding the speech of people with deafness.

The problem with hearing impaired people comes mainly from the specific and degraded nature of their auditory perception.

In his need to communicate, man has since the dawn of time builds a mode of oral communication, speech, which is based on average production (voice) and perception characteristics (the ear) of the sound signal. Current language is therefore that of the greatest number. Conversely, the hearing of the hearing impaired is very far from average and everyday language is hardly, if at all, accessible.

Understanding everyday language is a must for integration of the hearing impaired into their community. In what can be considered a reflex of social survival, all hearing impaired are brought naturally to form a language of their own and to implement processes, techniques, and a communication strategy enabling it to transpose the common language to its particular language. An example known and spectacular is that of lip reading, which provides access to normal speech through a visual alphabet of lip position.

The twentieth century saw a continuous effort in the design of machines to relieve and assist the hearing impaired.

Two classes of machines have been developed.

A first class is aimed at "mild" deafness and aims to correct the hearing and make it normal as much as possible. That's what make the usual prostheses widely available on the market.

A second class is intended for heavier deafness and aims to transform speech into synthetic speech accessible to the hearing impaired person. In this category most achievements are aimed at the "deep deaf". A remarkable example is that of the cochlear implant which acts by means of electrodes by stimulation direct from the auditory nerve.

The present invention aims to propose a solution for so-called "intermediate" deaf people. These people currently no suitable technical assistance. They are too affected to be served by standard prostheses, but their hearing is sufficient to be able to do without devices for the deaf.

The usual prostheses generally use a method of selective amplification of speech as a function of frequency. In its implementation an automatic regulation of the sound level acts on the gain of amplification, the aim being to give the best listening comfort and protection against instantaneous power peaks.

For reasons of commercial strategy and in response to the patient requests, these prostheses are miniaturized to be worn behind the ear or as an insert, which leads to performance relatively poor can only satisfy hearing corrections very coarse. Typically it is defined only three bands of frequencies for frequency correction. These prostheses are intended without ambiguity with the most frequent "light" deafness. More deafness can be relieved, but at the cost of painful inconvenience caused in particular by the amplification of background noise, and the phenomenon of Larsen. On the other hand there is no possibility of correction in the zones frequencies for which there is no hearing.

On the history of prostheses for the deaf we can see usefully refer to the work of M.J.M.TATO professor of ORL and MM VIGNERON and LAMOTTE cited in the article M J C LAFON having for title "Transposition and modulation", published in the audiophonology bulletin annales scientists of Franche Comté, Volume XII, N ° 3 & 4, monograph 164, 1996. These prostheses exploit the fact that deaf people are rarely completely deaf, and that a very weak balance of perception persists, often in the bass, which it has often been tried to take advantage of.

This is how it is possible to give back in a very rustic way, perception of the deaf sound by so-called “transposition” processes from treble to bass. Unfortunately the understanding of language requires more than just perception and it turns out that the transmission of intelligibility is inseparable from a necessary "richness" of sound. Giving back this “wealth” has become one of the main subjects of concern. It was thus that it was envisaged to create a word of synthesis in order to restore the structural elements that form the support for the intelligibility of everyday language.

The technique implemented in 1952 by M J.M. TATO, consists to record the speech spoken very quickly and to restore it at half speed. This allows transposition of an octave in the bass, while retaining the original speech structure. Tests have shown a some benefit for the deaf.

However, the method has the disadvantage that it cannot be used that in deferred time. The technique developed in 1971 by MM C.VIGNERON and M.LAMOTTE allows adaptation in "time real "by cutting time into 1/100 second intervals in removing every second interval, and applying the method of M J.M.TATO on the remaining intervals. But this system presents unfortunately a significant background noise.

The idea of building "natural" sounds is also present in a prosthesis also cited under the name GALAXIE in the article by Mr. JC LAFON. This prosthesis implements a battery of filters and mixers distributed over six sub-bands and transposes in the bass usable for the deaf.

Unfortunately, these processes which occur at the level of signal have too much distortion and too much listening discomfort to be used by people with deafness intermediate.

1- Il parait important de pouvoir transposer la globalité de la structure acoustique c'est à dire de ramener les éléments structurels de la parole porteurs de l'intelligibilité dans la zone de perception du malentendant.

2- Il parait aussi important de produire des sons « naturels »c'est à dire de reproduire une parole synthétique porteuse de l'information qui a une structure en harmonie avec l'acquis auditif du malentendant.

3- Enfin il faut veiller à conserver la temporalité du signal de parole car le rythme est porteur d'information accessible au malentendant.

From the article by Mr. Jean Claude LAFON, three directions emerge which can be retained in the achievement of good prosthetic treatment.

1- It seems important to be able to transpose the whole of the acoustic structure, that is to say to bring the structural elements of speech carrying intelligibility into the perception area of the hearing impaired.

2- It also seems important to produce “natural” sounds, that is to say to reproduce synthetic speech carrying information which has a structure in harmony with the hearing impaired person's hearing.

3- Finally, care must be taken to preserve the temporality of the speech signal because the rhythm carries information accessible to the hearing impaired.

The idea behind the invention is to overcome the drawbacks above using a parametric model of the speech signal capable carry out relevant transformations for hearing correction for the hearing impaired by implementing a method capable of satisfy the three constraints mentioned above.

To this end, the subject of the invention is a method for correcting hearing impaired characterized in that it consists in extracting the parameters characterizing the pitch, the energy voicing and the spectrum of the speech signal, to modify the parameters to make the speech intelligible to the hearing impaired, and to reconstruct a speech signal noticeable by the hearing impaired from the modified parameters.

The invention also relates to a device for setting work of the aforementioned process.

The method and the device according to the invention have the advantages of implement the parametric models which are commonly used in vocoders for adapters to hearing impaired hearing. This allows you to no longer work on the sound signal, as do previous techniques but at the level of the symbolic structure of the signal to preserve its intelligibility The vocoders present in indeed the advantage of using an alphabet that incorporates the notions of "pitch", " spectrum "," voicing "and" energy "which are very close to the model physiological of the mouth and ear. Under the theory of SHANNON, the information transmitted then carries intelligibility of speech. The materialization of speech intelligibility in a form IT thus opens up a new perspective. The intelligibility can thus be acquired during the analysis operation, and it is restored during the synthesis.

Thanks to the invention, the operation of synthesizing a vocoder parametric can therefore be adapted to the hearing characteristics of deaf. This technique, associated with more conventional, allows to consider a prosthetic process particularly general that can serve a very large population, including people with intermediate deafness.

As another advantage, the method and the device according to the invention offer great freedom in settings, each parameter can be modified independently of the others without reciprocal impact, with a specific setting for each ear.

Other characteristics and advantages of the invention will appear with the aid of the description which follows made with reference to the appended drawings which represent:

Figure 1, the parameters of the speech signal modeling used in the implementation of the invention.

Figure 2, a parametric model of signal generation word.

Figure 3, the different steps required to set up work of the method according to the invention in the form of a flowchart.

Figure 4, a transformation curve during the synthesis of speech signal the energy of the speech signal measured during the process speech signal analysis.

Figure 5, an embodiment of a device for setting work of the method according to the invention.

The method for processing the speech signal according to the invention is based on a parametric model of the speech signal of the type of that commonly used in the technique of making HSX digital vocoders and a description of which can be found in the article by MM P. Gournay, F. Charité, entitled "A 1200 bits / s HSX speech coder for very low bit rate communications ", and published in IEEE Proceedings Workshop on Signal Processing System (Sips'98), Boston, 8-10 October 1998.

• un paramètre de voisement qui décrit le caractère plus ou moins périodique des sons voisés ou aléatoire des sons non voisés du signal de parole,
• un paramètre définissant la fréquence fondamentale ou « PITCH » des sons voisés,
• un paramètre représentatif de l'évolution temporelle de l'énergie,
• et un paramètre représentatif de l'enveloppe spectrale du signal de parole.

This model is mainly defined by four parameters:

• a voicing parameter which describes the more or less periodic nature of the voiced or random sounds of the unvoiced sounds of the speech signal,
• a parameter defining the fundamental frequency or “PITCH” of the voiced sounds,
• a parameter representative of the time evolution of the energy,
• and a parameter representative of the spectral envelope of the speech signal.

The spectral envelope of the signal, or "spectrum", can be obtained by autoregressive modeling using a linear prediction filter or by a short-term Fourier analysis synchronous with the pitch. These four parameters are estimated periodically on the speech signal of one to several times per frame depending on the parameter, for a frame duration typically between 10 and 30 ms.

The speech signal is restored in the same way represented in figure 2, by exciting by the pitch or by a noise stochastic, a digital synthesis filter 1 which models by its transfer function the vocal tract depending on whether the sound is respectively voiced or unvoiced.

A switch 2 transmits the pitch or noise to the input of the synthesis filter 1.

A variable gain amplifier 3 depending on the energy of the speech signal is placed at the output of the synthesis filter 1.

In the case of a simple parametric model comprising a binary decision its voiced / its unvoiced, the synthesis procedure can be summarize to that shown in Figure 2. However, the process according to the invention which is represented in FIG. 3 in the form of a flowchart, is more complex and takes place in four stages, breaking down into one step 4 of preprocessing, a step 5 of analysis of the signal obtained in step 4 for the extraction of the parameters characterizing the pitch, the voicing, the energy, and the spectrum of the speech signal, a step 6 during which the parameters obtained in step 5 are modified and a step 7 of synthesis a speech signal composed from the modified parameters of step 6.

Stage 4 is that which is conventionally implemented in vocoders. It consists notably, after having converted the speech signal in a digital signal, to reduce the background noise by using for example the method described by M.D. Malah, published in IEEE trans.Acoust., Speech Processing, Volume-12 N.6, pp.1109-1121 1984, entitled "Speech enhancement using a minimum square error short time spectral amplitude estimator ", to cancel the acoustic echoes using for example the method described in the article by MM.Murano, S. Unjani and F. Amano having for title: "Echo cancellation and applications" published in the IEEE journal Com. May, 28 (1), pp 49-55, January 1990, to place an order automatic gain, or even to boost the signal.

The parametric processing of the speech signal obtained at the end of step 4 is carried out in step 5. It consists of cutting the speech signal into samples of constant analysis duration (typically 5 to 30 milliseconds) to perform on each of them the estimation of the parameters of the speech signal model. Using the HSX analysis model described in the article by Messrs. Gournay and F. Chartier cited above, the pitch and the voicing are estimated every 22.5 milliseconds. The voicing information is given in the form of a transition frequency between a voiced low band and an unvoiced high band. The signal energy is estimated every 5.625 milliseconds. During unvoiced periods of the signal, it is estimated over a duration of 45 samples (5.625 ms) and expressed in dB per sample. During voiced signal periods, it is estimated over a whole number of fundamental periods at least equal to 45 and expressed in dB per sample. The spectral envelope S (ω) is estimated every 11.25 milliseconds. It is obtained by linear prediction (LPC) by an auto-regressive modeling of a filter of order OLPC = 16 of transfer function: S (ω) = 1 / A (z) 2 with z = exp (jω) and ω = 2 π f
where A (z) is defined by:

PitchAnalyse ;

VoisementAnalyse ;

EnergieAnalyse[i], i=O à 3 ;

LpcAnalyse[k], k=I à 16.

In the following, the parameters resulting from the analysis are noted:

PitchAnalyse;

VoisementAnalyse;

EnergieAnalyse [i], i = O to 3;

LpcAnalyse [k], k = I to 16.

The synthesis procedure consists, for each interval of duration of the analysis, to stimulate the synthesis filter giving S (ω) by the sum frequency weighted (low band / high band defined by frequency voicing), a pseudo-random white noise for the high band and a Dirac comb periodic signal of fundamental frequency equal to pitch for the low band.

According to the invention, numerous transformations can be applied to the parameters from the analysis in step 5. Each parameter can indeed be modified independently of the others, without reciprocal interaction. In addition, these transformations can be constant or be activated only under special conditions (for example, triggering of the modification of the spectral envelope for certain energy distribution configurations as a function of frequency, ...).

These modifications are carried out in steps 6 ₁ to 6 ₄ and they relate essentially to the value of the pitch which characterizes the fundamental frequency, the voicing, the energy and the spectral envelope.

For the course of step 6 _1, any transformation defining a new “pitch” value from the value of the analysis pitch obtained in step 5 is applicable.

0,25 < FacteurPitch < 4.0

50 Hz < PitchSynthèse < 400 Hz.

Elementary transformation is homothety, defined by the relation: PitchSynthèse = PitchAnalyse * FacteurPitch, with the following limitations:

0.25 <Pitch Factor <4.0

50 Hz <Pitch Synthesis <400 Hz.

The Pitch Factor is adjustable for the type of deafness considered.

As for the pitch, the voicing frequency can be modified by any transformation defining a "voicing frequency For each value of the voicing frequency analyzed in step 5.

0,25 < FacteurVoisement < 4.

0 Hz < VoisementSynthèse < 4000 Hz.

In the example of implementation of the invention the transformation chosen is a homothety, defined by the relation: VoisementSynthèse = VoisementAnalyse * FactorVoisement, with the following limitations:

0.25 <Voice Factor <4.

0 Hz <Voice Synthesis <4000 Hz.

When the voicing transition frequency from AnalysisVoisementAnalyse is maximum (fully voiced signal, VoisementAnalyse = VoisementMaximum) the voicing frequency used in synthesis is unchanged (VoisementSynthèse = VoisementMaximum). Applying a multiplicative factor to it would indeed totally arbitrary (VoisementAnalyse = VoisementMaximum does not mean not an absence of voicing above VoisementMaxmum). As example VoisementMaximum can be set to 3625 Hz.

The factor Factor Voisement is adjustable for the type of deafness considered.

The energy is processed in step 6 ₃ As before, any transformation defining an energy from the energy of the speech signal analyzed in step 6 ₃ is applicable. In the example described below, the method according to the invention applies to energy a compression function with four linear segments as shown in the graph of FIG. 4.

The energy used in synthesis is given by the relation: EnergieSynthèse [i] = Slope * EnergieAnalyse [i] + EnergieSynthèseSeuil - Slope * EnergieAnalyseSeuil, for i = O to 3, with
Slope = Low Slope for EnergieAnalyse <EnergieAnalyseSeuil;
Slope = High Slope for EnergieAnalyse> = EnergieAnalyseSeuil;
and with the following limitations:
EnergieSynthèse <= EnergieSynthèseMax;
EnergieSynthèse = - Infinite for EnergieAnalyse <EnergieAnalyseMin.
The EnergieAnalyseMin, EnergieSynthèseMax, SlopeLow, SlopeHigh and EnergieSynthèseSeuil treatment parameters are adjustable for the type of deafness considered.

The processing of the spectral envelope takes place in step 6 ₄ In this step any transformation defining a spectrum S '(ω) from the spectrum
S (ω) analyzed in step 5 is applicable.

In the embodiment of the invention described below the elementary transformation on the spectrum which is implemented is a homothetic compression of the frequency scale.

The frequency scale is compressed by a factor Factor Spectrum so that useful bands before and after treatment are respectively equal to [O..FECH / 2] and to [O..FECH / (2 * FacteurSpectre)], where FECH is the sampling frequency of the system.

The implementation of this homothetic compression is very simple when the compression factor is an integer value. It then suffices to replace z by z ^{FacteurSpectre} in the expression of the filter any pole of synthesis, then to apply to the synthesized signal a low-pass filtering of cut-off frequency FECH / (2 * FacteurSpectre).

A first theoretical justification of the validity of the process described above is to say that this operation is equivalent to operating a factor oversampling Factor Impulse response spectrum of the vocal tract, by insertion of FacteurSpectre-1 null samples between each sample of the original vocal tract impulse response then by low-pass filtering of the synthesized signal with a frequency of cutoff equal to FECHI (2 * FactorSpectrum).

A second theoretical justification consists in considering that this operation is equivalent to duplicating and moving the poles of the function of transfer.

Indeed, considering that the single pole OLPCs denoted zi = pi.exp (2iπFi) of the transfer function 1 / A (z), the "Spectrum Factor * pole OLPC" of l / A (Z ^{FactorSPectre} ) are then the " FactorSpectrum "complex roots of each of the zi. The poles retained by the low-pass filtering operation are of the type z'i = p _i ^{I / FacteurSpectre} exp (2. I.π.Fi / FacteurSpectre) which shows that their resonance frequency has actually undergone homothetic compression a "FactorSpectrum" factor.

avec :

OLPC2 = FacteurSpectre * OLPC ;

LpcSynthèse[k] = 0 pour k=I à OLPC2, k non multiple de FacteurSpectre.

LpcSynthèse [FacteurSpectre * k] = LpcAnalyse[k] pour k=I à OLPC;

The LPC filter used in synthesis can therefore be expressed in the form:

with:

OLPC2 = FactorSpectrum * OLPC;

LpcSynthesis [k] = 0 for k = I at OLPC2, k not multiple of FactorSpectrum.

LpcSynthèse [FacteurSpectre * k] = LpcAnalyse [k] for k = I at OLPC;

It is possible to restrict the compression factor of the spectral envelope to be an integer between 1 and 4 such that: 1 <SpectrumFactor <4.

Speech restored in step 7 can be further accelerated or slowed down by simple modification of the duration of the time interval taken into account for the summary phase.

In practice, this operation can take place by implementing a homothetic transformation procedure defined by the relation: Tsynthesis = Tanalyse * TimeFactor

If TimeFactor> 1, then it is a slowdown in the word. If TimeFactor <1, then it is an acceleration of speech.

In addition to the previous treatments, a number of post-treatments can be envisaged consisting for example of carrying out a bandpass filtering and linear equalization of the synthesized signal, or still a multiplexing of sound on both ears.

The objective of the linear equalization operation is to compensate the patient's audiogram by amplifying or attenuating certain bands of frequencies. As part of the prototype, the gain at 7 frequencies (0, 125, 250, 500, 1000, 2000 and 4000 Hz) can be adjusted over time between -80 and +10 dB according to the patient's needs or the specifics of his audiogram. This operation can be carried out for example by filtering by a fast Fourier transform (FFT) as described for example in the book of M.D Elliott entitled "Handbook of Digital signal processing" published in 1987 by Academic Press.

Multiplexing operation allows monophonic playback (for example a signal processed alone) or stereophonic (for example a signal processed on one channel and an unprocessed signal on another). Restitution allows the hearing impaired to adapt the treatment to each of his ears (two linear equalizers to compensate for two different audiograms for example), and possibly keep intact on one ear a form of signal to which it is accustomed and on which it can be supported to, for example, synchronize.

The device for implementing the method according to the invention which is represented in FIG. 5 comprises a first channel composed of an analysis device 8, a synthesis device 9 and a first equalizer 10 and d a second channel comprising a second equalizer 11, the assembly of the two channels being coupled between a sound pickup device 13 and a pair of headphones 12 _a , 12 _b . The analysis 8 and synthesis 9 devices can be implemented by borrowing the known techniques for producing vocoders and in particular that of the aforementioned HSX vocoders. The outputs of the equalizers of the two channels are multiplexed by a multiplexer 14 to allow reproduction of the monophonic or stereophonic sound. A processing device 15 formed by a microprocessor or any equivalent device is coupled to the synthesis device 9 to ensure the modification of the parameters supplied by the analysis device 8.

A pretreatment device 16 interposed between the sound recording 13 and each of the two channels ensures denoising and conversion of the speech signal into digital samples The samples denoised digital are applied respectively to the input of the equalizer 11 and at the input of the analysis device 8.

According to other embodiments of the device according to the invention, the processing device 15 can be integrated into the processing device. synthesis 9, as it is also possible to integrate all the treatments analysis and synthesis in the same software executable on a personal computer or answering machine for example.

Claims (3)

Method for hearing correction of the hearing impaired, of the type consisting in extracting (5) by an LPC analysis the parameters characterizing the pitch, the voicing of the energy and the spectrum of the speech signal, in modifying (6) the parameters for rendering the speech intelligible to the hearing impaired, and to reconstruct (7) a speech signal perceptible to the hearing impaired from the modified parameters, characterized in that: the parameter characterizing the pitch is modified (6 ₁ ) by applying a multiplier factor to the value of the extracted pitch, the parameter characterizing the voicing is modified (62) by a multiplying factor, the parameter characterizing the energy is modified (63) by a compression function, the parameter characterizing the spectral envelope is modified (64) by a homothetic compression of the frequency scale, and the duration of the time interval taken into account for the synthesis phase is modified (7) by multiplying the time interval by a time factor. Device for implementing the method according to claim 1, characterized in that it comprises an analysis device (8) for extracting the parameters of the speech signal coupled to a device for synthesis (9) and a processing device (14) coupled to the synthesis (9) to modify the parameters provided by the analysis device (8) and apply the modified parameters to the synthesis device (9) in order to reconstitute a speech signal with the modified parameters. Device according to claim 2, characterized in that the analysis device (8) and the synthesis device (9) comprise a linear prediction vocoder analysis and synthesis device.

Images