JP2008102551A - Apparatus for processing voice signal and processing method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve aural comprehension of words to masking by a voiced sound or successive time masking. <P>SOLUTION: An apparatus for processing a speech signal is provided with an amplitude changing means 15 which performs a change of the amplitude of a bandwidth of a consonant of an input speech signal S17, and an extraction means 21 which extracts a pitch component and formant component of the signal S17. The apparatus is provided with a level calculation means 22 which calculates a signal to indicate the level of the voiced sound from the extraction output of the extraction means 21, and a voiced sound start point detection means 23 which detects the start point and end point of the voiced sound in the input speech signal S17 from the output of the level calculation means 22. The apparatus is provided with a control means 24 which performs control to the amplitude changing means 15 so as to increase the gain of the amplitude changing means 15 with respect to a reference value only in the section from the start point of the voiced sound detected by the detection means 23 down to the fall from the end determination point and to lower the gain of the amplitude changing means 15 down to the reference value in the fall section from the end point of the voiced sound detected by a voiced sound end point detection means 23. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an audio signal processing apparatus and a processing method therefor.

  When transmitting or reproducing sound, if there is a lot of reverberation or echo in the transmission system or reproduction system, the clarity of the sound will be reduced. Therefore, in such a case, processing such as slowing down the utterance speed, finely disassembling continuously uttered speech sounds, and reproducing them at intervals.

  In addition, when high frequency such as consonant is difficult to hear, high frequency may be emphasized by frequency equalizer processing. Furthermore, a process of applying a weighting function in consideration of so-called successive masking (a phenomenon in which a consonant is masked by a vowel when a high energy vowel and a consonant continue) has been attempted.

  Furthermore, the above processing may be performed for a hearing impaired person or an elderly person.

For example, there are the following prior art documents.
JP-A-8-179792 Japanese Patent Laid-Open No. 9-16193

  However, as described above, if the utterance speed is slowed down or the speech that is continuously spoken is decomposed, the following problems occur.

1. There will be a time lag between the original voice and the immediacy will be lost. Therefore, it cannot be used for conversations. Also, even when listening to broadcasts, it takes a long time to finish listening.
2. The rate of change of the speech component is also an important clue for the perception of speech, so if the utterance speed is slowed, this clue may change and be perceived by another speech.
3. If the speech is decomposed and reproduced slowly, information as a unit of speech and information on transitional changes may be lost, resulting in poor clarity.
4). A sound whose frequency band is always amplified by the frequency equalizer process may be uncomfortable or difficult to hear because the tone color balance is lost.
5. The process of applying the weighting function considering the continuous masking causes a delay of at least the time length of the weighting function and loses immediacy. As a result, a time lag may occur between the mouth movement and the processed sound, which may adversely affect the intelligibility. In addition, when there is acoustic feedback from the earphone to the microphone, a phenomenon such as reverberation is caused by the time delay.

  The present invention is intended to solve the above problems.

In this invention,
Amplitude changing means for changing the amplitude of the consonant band of the input voice signal;
Extraction means for extracting the pitch component and formant component of the input audio signal;
Level calculation means for calculating a signal indicating the level of voiced sound from the signal extracted by the extraction means;
Voiced sound start point detecting means for detecting the start point of the voiced sound in the input voice signal from the output of the level calculating means;
Voiced sound end point detecting means for detecting the end point of the voiced sound in the input voice signal from the output of the level calculating means;
The gain of the amplitude changing means is increased with respect to the reference value only in the section from the start point of the voiced sound detected by the voiced sound start point detecting means and the voiced sound end point detecting means to the fall from the end determination point. And control for controlling the amplitude changing means so as to lower the gain of the amplitude changing means to the reference value in the fall period from the end point of the voiced sound detected by the voiced sound end point detecting means. And an audio signal processing device having means.

  According to the present invention, the sound is clear and the intelligibility can be improved. Moreover, there is no discomfort as in the case where the high frequency range of the voice is always emphasized. Furthermore, there is no time difference between the movement of the speaker's mouth and the processed sound.

  By the way, the voice of normal conversation is composed of a combination of a low frequency component and a high frequency component. In addition, sound existing in a general living environment is often a combination of a low frequency component and a high frequency component.

  In auditory sense, it is known that a low frequency component masks a high frequency component, and this masking works when perceiving speech. In normal hearing, this interference by masking is small, and even if there is masking, speech can be perceived correctly. It has become one of the.

  In addition, even a normal hearing person may have difficulty in listening to words when there is a large amount of low-frequency noise. In addition, so-called successive masking also reduces word clarity.

  Therefore, the present invention is intended to suppress a reduction in clarity resulting from such masking or successive masking.

  For this reason, in one embodiment of the present invention, a period of several milliseconds to tens of milliseconds from the start point of the voiced sound is set as a rising period, and a period of tens of milliseconds to several tens of milliseconds from the end point of the voiced sound. When the period is the falling period, the high frequency component is enhanced during the rising period and the falling period.

  FIG. 1 shows an embodiment of the present invention. An unprocessed audio signal S11 is supplied to a filter 12 having a passband of a voiced sound band through an input terminal 11, and a signal component S12 of the voiced sound is extracted. This signal component S12 is supplied to the adder circuit 13. Further, the signal S11 from the terminal 11 is supplied to the filter 14 having the passband of the consonant band to extract the consonant signal component S14, and this signal component S14 is supplied to the adder circuit 13 through the variable gain amplifier 15. .

  Therefore, since the signal component S12 and the signal component S14 are added in the adder circuit 13, if the gain G15 of the variable gain amplifier 15 is a reference gain (for example, 1 time), the adder circuit 13 The voice signal S13 having the voiced signal component S12 and the consonant signal component S14 included in the signal S11 in an equal proportion is obtained. Then, this signal S13 is taken out to the output terminal 16.

  Further, the signal S11 at the terminal 11 is sequentially supplied to the band pass filter 21 and the level calculation circuit 22 for preprocessing. In this case, the band pass filter 21 extracts the pitch component and the formant component as the signal S21 from the signal S11 so that the start point and the end point of the voiced sound can be easily detected and the influence of noise is reduced. Is. Therefore, the pass band of the band pass filter 21 is, for example, 150 Hz to 1000 Hz.

  Further, the level calculation circuit 22 forms a signal S22 indicating the level of the signal S21 by, for example, performing both-wave rectification on the signal S21 and extracting a low-frequency component (for example, a component of 60 Hz or less).

  Then, the calculation signal S22 of the level calculation circuit 22 is supplied to the detection circuit 23 to detect the start point and the end point of the voiced sound, and the detection signal S23 is supplied to the control circuit 24 to form the control signal S24. This signal S24 is supplied to the variable gain amplifier 15 as a control signal with a gain G15.

  In this case, the detection of the start point and the end point of the voiced sound and the magnitude of the gain G15 of the amplifier 15 have a relationship as shown in FIG. That is, when the level of the voiced sound indicated by the calculated signal S22 is smaller than the threshold value for the start determination, the gain G15 of the amplifier 15 is set to the reference value, but the level of the voiced sound is higher than the threshold value for the start determination. When the gain is increased, the gain G15 is gradually increased to the maximum value with a rising period of several milliseconds to several tens of milliseconds.

  Further, when the level of the voiced sound indicated by the calculation signal S22 is larger than the threshold value for the end determination, the gain G15 is kept high, but when the level of the voiced sound is lower than the threshold value for the end determination, The gain G15 is gradually reduced to the reference value with a falling period of about several tens of milliseconds to 200 milliseconds.

  According to such a configuration, when the signal component of voiced sound is included in the audio signal S11 before processing, the gain G15 of the amplifier 15 is increased by the signal S24 during the period from the start point to the end point. During the period from the start point to the end point, the consonant signal component S14 passing through the amplifier 15 increases.

  Accordingly, since the level of the consonant signal component S14 of the audio signal S13 output to the terminal 16 is increased during the period from the start point to the end point of the voiced sound, even if masking occurs in the reproduced sound of the signal S13, the masking is performed. The consonant will be louder than the size that fits, thus improving the listening of words.

  Also, during the period when the time interval from the end point of the voiced sound is short, the successive masking is large, but since the signal component S14 of the consonant is greatly amplified, the clarity is effectively enhanced even for the successive masking. Can do. Further, although the successive masking is small in the period where the time interval to the start point of the next consonant is long, the signal component S14 of the consonant is not much amplified during this period, so that the tone color balance is not lost.

  FIG. 3 shows one form of how the detection circuit 23 and the control circuit 24 form the control signal S24 from the detection signal S22. In other words, in this case, the entire circuit shown in FIG. 1 is digitized and configured by a DSP, for example. The audio signal S11 is a digital audio signal obtained by A / D converting the original analog audio signal before processing.

  In the detection circuit 23 and the control circuit 24, the processing routine 100 of FIG. 3 is executed for each sample of the digital audio signal S11, and the gain G15 of the amplifier 15 is controlled as shown in FIG. In the routine 100 and the following description, the meaning of each variable is as follows.

e (i): level indicated by the i-th sample of the audio signal S11.
threshold1: Threshold value for determining the end of voiced sound. Signal S11 is smaller than this value
When it becomes, the voiced sound is determined to be finished.
threshold2: Threshold for determining the start of voiced sound. Signal S11 is greater than this value
When it becomes, the voiced sound is determined to be started.
It is set to threshold1 ≦ threshold2.
w: Weighting factor for controlling the gain G15. 0 ≦ w ≦ 1
G15 = reference gain when w = 0, G15 = maximum gain when w = 1.
d1: Step width when the coefficient w is decreased.
d2: Step width when the coefficient w is increased.

  That is, in the routine 100, first, in step 101, it is determined whether or not the signal level e (i) of the i-th sample is smaller than the threshold value threshold 2 for start determination. Proceed to step 102.

  Then, in this step 102, it is determined whether or not the signal level e (i) of the i-th sample is smaller than the threshold value threshold 1 for the end determination. When it is smaller, the process proceeds from step 102 to step 103. In step 103, the coefficient w is decreased by the step width d1, and the routine 100 is terminated. Therefore, as shown in FIG. 2, when the end point of the voiced sound is detected, the gain G15 gradually decreases thereafter.

  In step 102, when the signal level e (i) of the i-th sample is equal to or higher than the threshold value threshold 1 for the end determination, the process ends the routine 100 from step 102. Therefore, as shown in FIG. 2, the gain G15 is maintained for a period until the end of the voiced sound is detected (a period in which the gain G15 is large).

  Further, in step 101, when the signal level e (i) of the i-th sample is equal to or higher than the threshold value threshold2 for the start determination, the process proceeds from step 101 to step 104. In step 104, the coefficient w is increased by the step width. The routine is finished by increasing d2. Therefore, as shown in FIG. 2, when the start point of the voiced sound is detected, the gain G15 gradually increases thereafter.

  In this way, according to the routine 100, the gain G15 of the amplifier 15 is controlled according to the level of the voiced sound to correct the consonant level, so that the auditory attenuation of the consonant component due to masking or successive masking is compensated. And intelligibility of speech such as consonant parts of conversation can be improved.

  FIG. 4 shows the observation result of the speech waveform. FIG. 4A shows an example of the waveform of the speech signal S11 not processed by the routine 100, and FIG. 4B shows an example of the waveform of the speech signal S13 processed by the routine 100. is there. The content of the utterance at this time is “Please write on the first line”.

  And, in the section B from the start point to the end point of the voiced sound, the consonant part is greatly amplified, and in the short period from the end point of the voiced sound (arrow A and F part), the successive masking is large. The consonant is greatly amplified, and during the long period until the start of the next consonant (arrows C, D, and E), since the successive masking is small, the consonant is not so amplified.

Therefore, according to the above-described processing circuit, the following effects can be obtained when sound is transmitted or reproduced in a system such as reverberation or echo, or when a hearing-impaired person or an elderly person listens to the sound.
1. Since the consonant is emphasized so that only the time masking to the next uttered sound is reduced, the voice is clear and the clarity can be improved.
2. Since the consonant is emphasized only when masking is occurring, there is no unpleasant feeling that the timbre is out of balance as in the case where the high range is always emphasized.
3. In principle, since immediate processing can be performed, there is no time difference between the movement of the speaker's mouth and the processed sound. Even if there is acoustic feedback from the earphone to the microphone, it does not sound like reverberation, so it is easy to hear.
4). The speed of change of speech components important for speech perception judgment, information as a unit of speech, and information of transitional changes are not lost.
5. According to the processing routine 100 of FIG. 4, since the number of steps of the processing is small, even a DSP that is somewhat slow in processing can sufficiently cope with it.

The routine 200 shown in FIG.
threshold = threshold1 = threshold2
Thus, the routine 100 is simplified. That is, in the routine 200,
threshold: Threshold value for voiced sound start / end judgment. Signal S11 is
If it is smaller than this value, it is determined that the process is finished.
It is determined that the beginning.
The others are the same as those in the routine 100.

  In step 201, the signal level e (i) of the i-th sample is compared with the threshold threshold. If the level e (i) is smaller than the threshold threshold, in step 202, the coefficient w is Otherwise, it is decreased by the width d1, otherwise, in step 203, the coefficient w is increased by the step width d2.

  Therefore, according to this routine 200, the processing is further simplified, and the burden on the DSP is further reduced.

It is a systematic diagram showing one embodiment of the present invention. It is a figure for demonstrating this invention. It is a flowchart which shows a part of one form of this invention. It is a wave form diagram for demonstrating this invention. It is a flowchart which shows a part of one form of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 12 ... Voiced sound band filter, 13 ... Adder circuit, 14 ... Consonant band filter, 15 ... Variable gain amplifier, 21 ... Band pass filter, 22 ... Level calculation circuit, 23 ... Detection circuit, 24 ... Control circuit, 100 ... Processing routine

Claims (8)

Amplitude changing means for changing the amplitude of the consonant band of the input voice signal;
Extraction means for extracting the pitch component and formant component of the input audio signal;
Level calculation means for calculating a signal indicating the level of voiced sound from the signal extracted by the extraction means;
Voiced sound start point detecting means for detecting the start point of the voiced sound in the input voice signal from the output of the level calculating means;
Voiced sound end point detecting means for detecting the end point of the voiced sound in the input voice signal from the output of the level calculating means;
The gain of the amplitude changing means is increased with respect to the reference value only in the section from the start point of the voiced sound detected by the voiced sound start point detecting means and the voiced sound end point detecting means to the fall from the end determination point. And control for controlling the amplitude changing means so as to lower the gain of the amplitude changing means to the reference value in the fall period from the end point of the voiced sound detected by the voiced sound end point detecting means. An audio signal processing apparatus comprising: means. The audio signal processing device according to claim 1,
The voiced sound processing device, wherein the voiced sound start point detecting means detects the voiced sound signal level when the signal level of the voiced sound exceeds a predetermined threshold value and detects it as a detection signal of the start point. The audio signal processing device according to claim 1,
The voiced sound end point detection means detects the end point detection signal when the signal level of the voiced sound falls below a predetermined threshold value. The audio signal processing device according to claim 1,
An audio signal processing apparatus in which the amplitude changing means is a variable gain amplifier. In the audio signal processing apparatus according to claim 1,
When the rising period is a period of several milliseconds to tens of milliseconds from the start point of the voiced sound, and the rising period is a period of tens of milliseconds to several tens of milliseconds from the end point of the voiced sound An audio signal processing apparatus configured to change the gain of the amplitude changing means during the period and the falling period. Amplitude changing means for changing the amplitude of the consonant band of the input voice signal;
Extraction means for extracting the pitch component and formant component of the input audio signal;
Level calculation means for calculating a signal indicating the level of voiced sound from the signal extracted by the extraction means;
Voiced sound start point detecting means for detecting the start point of the voiced sound in the input voice signal from the output of the level calculating means;
Voiced sound end point detecting means for detecting the end point of the voiced sound in the input voice signal from the output of the level calculating means;
The gain of the amplitude changing means is increased with respect to the reference value only in the section from the start point of the voiced sound detected by the voiced sound start point detecting means and the voiced sound end point detecting means to the fall from the end determination point. And a control means for controlling the amplitude changing means so as to change the gain of the amplitude changing means in accordance with the level of the voiced sound. An amplitude changing step for changing the amplitude of the consonant band of the input audio signal;
An extraction step for extracting a pitch component and a formant component of the input audio signal;
A level calculation step for calculating a signal indicating the level of voiced sound from the signal extracted in the extraction step;
A voiced sound start point detecting step for detecting a start point of the voiced sound in the input voice signal from an output of the level calculating step;
A voiced sound end point detecting step of detecting an end point of the voiced sound in the input voice signal from the output of the level calculating step;
A control step for controlling the gain of the amplitude changing step based on outputs of the voiced sound start point detection step and the voiced sound end point detection step;
In this control step, when the voiced sound start point detection step detects the start point, the control step supplies a control signal so that the gain is larger than a reference value with respect to the amplitude change step.
When the voiced sound end point detecting step detects the end point, the control signal is supplied so as to return the gain to the reference value with respect to the amplitude changing step. An amplitude changing step for changing the amplitude of the consonant band of the input audio signal;
An extraction step for extracting a pitch component and a formant component of the input audio signal;
A level calculation step for calculating a signal indicating a level from the signal extracted in the extraction step;
A voiced sound start point detecting step for detecting a start point of the voiced sound in the input voice signal from an output of the level calculating step;
A voiced sound end point detecting step of detecting an end point of the voiced sound in the input voice signal from the output of the level calculating step;
A control step for controlling the gain of the amplitude changing step based on outputs of the voiced sound start point detection step and the voiced sound end point detection step;
In this control step, when the voiced sound start point detection step detects the start point, the control step supplies a control signal so that the gain is larger than a reference value with respect to the amplitude change step.
A method for processing an audio signal, comprising: supplying the control signal so as to change the gain with respect to the amplitude changing step according to a level of voiced sound.

Images