JP3398401B2 - Voice recognition method and voice interaction device - Google Patents

Abstract

PURPOSE:To fetch and recognize audio input from a reader even when an audio response is issued from a system by cancelling the audio response when the audio response outputted from a speaker is inputted from a microphone. CONSTITUTION:An audio signal inputted from the microphone 1 is supplied to a speech recognition part 5 as it is. After that, the audio response for speech recognized by an interactive control part 6 is selected, and the audio response is outputted from an audio response part 7, and is supplied to an adaptive filter 3, and also, it is outputted from the speaker 8. Meanwhile, input speech on which the audio response is superimposed is fetched in the microphone 1 from the speaker 8, and it is supplied to a subtractor 4. The output of the speaker 8 fetched from the microphone 1 is cancelled by subtracting a signal in which the audio response outputted from the speaker 8 is corrected by using LSM/Newton algorythm from a signal inputted from the microphone 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice dialogue system in which a human and a computer talk by voice.

[0002]

2. Description of the Related Art In recent years, a voice dialogue system using voice information has been actively developed as an interface between a human and a computer.

The voice dialogue system is effective as a multimedia dialogue system for displaying visual information such as graphic information, images and animations together with voice output. When a speaker speaks into a microphone, the voice is recognized, A voice response to this is output from a speaker to have a dialogue with a human. An example of using such a voice dialogue system at a hamburger shop will be described. First, when the customer utters "2 hamburgers and 3 juices" into the microphone, the system recognizes this and outputs a utterance that confirms "2 hamburgers and 3 juices". afterwards,
If the customer answers yes, the order is confirmed to be 2 burgers and 3 juices and the employee is notified.

However, the customer mistakenly says, "Hamburger 3
If you say "individual ...", you can not cancel immediately, but when the system responds with a confirmation "3 hamburgers ...", you cancel and again "2 hamburgers". ... ". Also, for example, when a customer says "Please give me 2 burgers, cola and ice cream", the system misrecognizes and gives a response "4 potatoes, cola and ice cream". The customer wants to immediately interrupt and correct the response “4 potatoes ...”, but cannot correct it until all the system responses are completed. Therefore, the dialogue takes a long time, which is very annoying.

[0005]

As described above, in the conventional voice dialogue system, the voice input from the speaker and the voice response output cannot be performed at the same time, and after all the response voices from the system are finished, You have to input the voice. Therefore, when the system erroneously recognizes, it takes a long time to input again, and there is a disadvantage that an efficient dialogue cannot be performed.

The present invention has been made to solve such a conventional problem, and a first object thereof is to capture a voice input from a speaker even when the system emits a voice response. Voice recognition method that can be recognized by
And a voice interaction device .

The second purpose is a voice dialogue in which the output of the voice response can be changed according to the importance of the recognition content and the response content.
It is to provide a device .

[0008]

In order to achieve the above object, the first invention of the present application recognizes a voice input from an input means such as a microphone, and outputs a predetermined voice response based on the recognition result to a speaker or the like. In the voice recognition method for outputting the dialogue from the output means, the output means
The frequency spectrum of the impulse response of the voice response output from
To synthesize a transfer function that is a koutor to the voice response
Estimated using the response generation parameter of
The voice response is corrected by an impulse response, only the corrected voice response is canceled from the input voice, and the voice after canceling the voice response is recognized.

The second invention of the present application further comprises means for obtaining the background noise power in the absence of voice input, means for obtaining the synthesized sound power in the microphone signal based on the impulse response at the time of outputting the synthesized voice, A means for determining whether or not there is a voice input based on the duration of the voice input power exceeding the threshold, with the sum of the background noise power and the synthesized voice power being the detection threshold of the voice input power. And means for recognizing voice only when it is judged by the judging means that there is voice input.

Further, the third invention of the present application is a pattern recognition means for recognizing an input from a user by at least one of a voice, a keyboard and a pointing device, and a voice response based on an understanding result by the pattern recognition means. An interrupt management unit that determines the response content of the image response, and an interrupt that determines whether or not to accept an interrupt from the user, based on the understanding result by the pattern recognition unit and the response content output from the interaction management unit. Based on the control means and the interrupt control information from this interrupt control means and the response content from the dialogue management means, the image response or the voice response being output is terminated, or the interrupt control information and the response content are
It characterized by having a a response generating an output means for outputting the image response and voice response by changing the response generation parameters such as the speech rate, prosody power of the image response and voice response in the output based.

[0011]

In the first aspect of the present invention configured as described above, the voice response is corrected by the voice characteristics such as power and pitch in the voice response, and the corrected signal is subtracted from the microphone input. Therefore, after the voice response is removed from the speech signal of the user on which the voice response is superimposed, the voice is recognized. Therefore, the user can speak even during the voice response output.

If a smoothing filter for smoothing the voice response signal is provided and the output is controlled so as to stop the adaptation when the voice response is not output, the voice response is not output. Sometimes the transfer function estimation accuracy does not decrease, and high estimation accuracy can be maintained.

Further, in the second invention of the present application, the power of the background noise is obtained in advance, and the input voice is recognized when the input is larger than this. Then, even if the voice response is not completely removed and the voice response from the speaker is captured from the microphone, an error may occur by raising or lowering the threshold for recognizing the voice input according to the power of the voice response. Input is prevented. Therefore, highly accurate voice input is possible.

Further, in the third invention of the present application, when an interrupt input is made by the user during a voice response, whether the interrupt should be permitted based on the importance of the input contents and the importance of the voice response. It is determined whether or not to output the voice response. As a result, it is possible to perform a high level dialogue depending on the contents of the input voice and the voice response.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of a voice dialogue system to which the present invention is applied.

As shown, this spoken dialogue system
A microphone 1 for capturing an input voice from a speaker, a speaker 8 for outputting a voice response of the system, and a voice response removing unit 2 for removing a voice response superimposed on the input voice from the speaker.
A voice recognition unit 5 that captures the output of the voice response removal unit 2 to recognize the utterance content of the speaker; a dialogue control unit 6 that selectively controls the voice response corresponding to the recognized voice; The voice response unit 7 outputs a response to the speaker 8 and the voice response removal unit 2, and the display 16 displays visual data such as graphic information, images, and animations.

The voice response removing unit 2 stores in advance a look-up table 3a in which power information, pitch information, amplitude information of various voice responses, voiced / unvoiced, and silent information are stored in advance.
And an adaptive filter 3 for obtaining an impulse response by an LMS / Newton algorithm described later, and correcting and outputting a voice response by the impulse response, and a microphone 1.
Has a subtractor 4 for subtracting the output of the adaptive filter 3 from the input.

In such a structure, the operation of this embodiment will be described below with reference to the flow chart shown in FIG.

First, when a speaker inputs a voice from the microphone 1, this voice signal is supplied to the voice recognition unit 5 via the voice response removal unit 2. At this time, since there is no output from the voice response unit 7, the voice response removal unit 2 does not perform the process, and the voice signal input from the microphone 1 is directly supplied to the voice recognition unit 5. After that, the dialogue control unit 6 selects a voice response to the recognized voice (step ST1), and this voice response is output from the voice response unit 7. Therefore, the voice response is supplied to the adaptive filter 3 and the speaker 8 is also supplied. Is output from (STEP ST
2, ST3).

Then, the adaptive filter 3 obtains an impulse response by the following equation (1).

[0021]

## EQU1 ## W _{(k + 1)} = W _(k) + 2μR ' _(k) e _(k) X _(k ) (1) Equation (1) is an arithmetic expression called LMS / Newton algorithm. Here, k is a factor indicating the time phase, k is the current output, and k + 1 is the next output. R'is an inverse matrix of the correlation matrix of the voice response, which is given from the look-up table 3a.

Μ is a focusing coefficient, and the voice response output from the speaker 8 is not directly input to the microphone 1 but is reflected or attenuated depending on the surrounding environment. μ is a factor for determining the transfer function W in consideration of these changes. Further, e is an error and X is an input signal vector.

The impulse response thus obtained is multiplied by the voice response X to generate an output signal y, which is output to the subtractor 4 (step ST4).

That is, y = W ^T X (T is a transpose) (2).

On the other hand, the microphone 1 takes in the input voice on which the voice response from the speaker 8 is superimposed. Then, the taken-in audio signal d is supplied to the subtractor 4 (step ST5), and the subtractor 4 obtains the subtraction signal s by the following equation (3) (step ST6).

S = d−y (3) Then, the subtraction signal s is supplied to the voice recognition unit 5 (step ST7), and the input voice from the speaker is recognized,
A voice response corresponding to this is selected by the dialogue control unit 6 and output from the voice response unit 7. Then, the adaptive filter 3 takes in the voice response and obtains the next impulse response (step ST8), and the above-described operation is repeated until the voice input is completed (step ST9).

In this way, in this embodiment, the voice response output from the speaker 8 is corrected by using the LSM / Newton algorithm, and the corrected signal is input to the microphone 1.
Microphone 1 by subtracting from the signal input from
The output of the speaker 8 taken from is canceled. Therefore, even when the voice response is output from the speaker 8, the speaker can input the voice from the microphone 1.

Further, in the above embodiment, the algorithm is carried out by using the reciprocal R'of the autocorrelation macrix of the voice response, but when the voice response is regularly synthesized, the power of voice, voiced / unvoiced, vowel / Consonant, silence, duration information, etc. may be used. In particular, when the LMS / Newton algorithm is implemented using the power p of the voice, the arithmetic expression shown in the following expression (4) is used.

[0029]

## EQU2 ## W _{(k + 1)} = W _(k) +2 (μ / p _(k) L) e _(k) X _(k) (4) where L is the dimension of the input speech vector. Further, in the voice dialogue system of the present embodiment, since characteristics such as power information and pitch information of the voice response are stored in advance in the lookup table 3a, a suitable impulse response according to the characteristic of the voice response is obtained. be able to.

FIG. 2 is a characteristic diagram showing the power information of the voice response and the removal result by the voice response removing unit 2, which is the curve S ₃
Is the power information of the voice response, the curve S ₁ is the removal result of the voice response when the algorithm is performed with this power information as a constant value, and the curve S ₂ is the data when the power information changes as the curve S _3. It is the removal result of the voice response when the algorithm is implemented based on. As is apparent from the figure, it is understood that the result of removing the voice response is better and the voice response can be removed with higher accuracy when the algorithm is executed using the power information stored in the lookup table 3a. .

In this embodiment, the case where the response uttered from the speaker 8 is only voice is described. However, when it is desired to output music simultaneously with voice, the voice response unit 7 shown in FIG. Configure as follows. That is, the voice response unit 7 includes a voice synthesis unit 10 that outputs a voice signal, a music synthesis unit 11 that outputs a music signal, and a mixer 9 that synthesizes these. Then, the characteristic information of music can be easily obtained from the musical note, and if this is stored in the look-up table 3a shown in FIG. 1, the voice response is removed as in the case of only the voice signal described above. can do.

Further, the present invention can be applied not only to voice and music, but also to sound signals such as natural sounds (bird's bark, etc.) and buzzer sounds. Since it is known in advance that the buzzer sound is a periodic signal and the random noise is an irregular but stationary noise, it is possible to perform highly accurate noise cancellation by using these information.

The signal output from the voice response unit is
It is known that in the case of wide band noise (white noise), the transfer function W from the speaker 8 to the microphone 1 can be easily estimated. That is, voiced sounds (vowels, etc.) of voice signals
Is a periodic signal and has non-stationarity,
The short-time frequency spectrum becomes a line spectrum. For this reason, the spectrum component is not in a wide band, and the estimation accuracy of the impulse response is deteriorated. Therefore, FIG.
With the configuration shown in (1), a wideband signal such as noise or music can be added to a place where there is no frequency component of the voice response other than the voice message, and the accuracy of the LMS and FLMS algorithms can be improved.

Next, a second embodiment of the present invention will be described. In the above-described first embodiment, it is known that the accuracy of impulse response estimation is significantly reduced when the user inputs a voice into the voice interaction system. Therefore, in the second embodiment, the transfer function update control unit 15 is provided as shown in FIG. 8 to improve the estimation accuracy. Hereinafter, this operation will be described.

First, when the impulse response is estimated by using the LMS / Newton algorithm, the past impulse response is held for 5 seconds, for example, every 100 [ms].

That is, each transfer function before W ₀ ... present W _-1 ... 100 [ms] before W _-2 ... 200 [ms] ... ...... ............ W _-50 ... 5 [seconds] before Remembered. Then, in the voice recognition unit 5 shown in FIG. 1, when the voice of the user is detected, the setting of the impulse response is changed to that before the voice utterance. That is, for example, when a voice is input from the user 750 [ms] ago, the impulse response W _-8 800 [ms] ago is changed to W ₀ and used for sequential processing. Further, this operation will be described based on the time chart shown in FIG.

The curve S ₄ shown in the figure is the voice response signal, and the curve S ₅ is the speech signal of the user. Then, the voice response removal unit 2 removes the voice response while updating the impulse response every 100 [ms], and the voice recognition unit 5 detects the user's utterance and detects the start point t _S and the end point t _E of the utterance. .
When the user's utterance is detected, the impulse response update control unit 15 shown in FIG. 8 updates the estimated value W ₀ of the impulse response, or the estimated value W _i (i = −1 to past)
−50) is used for every 100 [ms]. As a result, the adaptive filter 3 can obtain a more accurate impulse response, which improves the removal efficiency of the voice response.

Further, in each of the above-described embodiments, the voice rule synthesis is performed to generate the voice response, and hereinafter, the accuracy is calculated from a series of internal information (eg, time series of pitch and power) necessary for this voice synthesis. A method for estimating a good impulse response will be described with reference to FIGS. Figure 5 shows "Cancel (Torikeshimas)
FIG. 7 is a diagram showing temporal changes in power and pitch when a voice response “u)” is synthesized. Further, FIG. 4 is a flowchart for obtaining the FLMS focusing coefficient. However FL
The MS estimates the transfer function, which is the frequency spectrum of the impulse response.

First, at time n = 0, it is determined from the power information shown in FIG. 5A whether or not there is a silent section (step ST11). When it is determined that the sound is silent (YES in step ST11), the FLMS focusing coefficient μ (f) is set to “0” at all frequencies (step ST14). As a result, the estimated value of the transfer function does not change even by the adaptive estimation, so that the estimated value of the transfer function is not affected even if noise is input from the microphone 1 in the silent section.

On the other hand, when it is determined that the phoneme is not silent (NO in step ST11), it is determined whether the phoneme is a consonant or a vowel (step ST12). This determination can be easily performed because the current phoneme is known.

If it is determined that the sound is a consonant ("consonant" side in step ST12), the power is further set to a threshold value (for example, ambient environmental noise level +20 dB).
It is determined whether or not the above is true (step ST15).
And, when it is less than the threshold value (N in step ST15)
In (O), μ (f) = 0 is set for all frequencies (step ST16). If it is equal to or more than the threshold value, μ (f) = a (a is a predetermined focusing coefficient) is set at all frequencies (step ST17).

On the other hand, if the phoneme is a vowel (step S
On the "vowel" side in T12), it is determined whether or not the power is equal to or higher than a threshold value (step ST13). Then, if it is equal to or less than the threshold value (NO in step ST13), μ (f) = 0 is set for all frequencies (step ST18).

If the threshold value is exceeded (step 13)
YES), for example, in the range of ± (1/3) f _p around a frequency that is an integer multiple of the pitch frequency f _p , μ (f) =
a. Further, outside this range, μ (f) = 0 is set (step ST19). That is, it is the following expression (5).

[0044]

[Mathematical formula-see original document] μ (f) = a (f _p · n−1 / 3f _p <f <f _p · n + 1 / 3f _p ) μ (f) = 0 (other than the above) (5) Then, the above-mentioned operation Is repeated every 10 [ms], for example (step ST20).

In this way, since the transfer function estimation value is updated by emphasizing the frequency component having a large power in the voice response signal, highly accurate estimation is possible.

Next, a third embodiment of the present invention will be described. In the transfer function estimation by the LMS / Newton algorithm described above, it is known that the estimation accuracy changes and the estimation operation becomes unstable when a non-stationary signal such as voice is input. However, the interactive system needs stable impulse response estimation even when synthetic speech is input. Therefore, a method of stably obtaining a highly accurate impulse response even if the input signal has a large power fluctuation will be described below.

FIG. 9 is a block diagram showing the structure of the third embodiment, showing the internal structure of the voice response removing unit 2 shown in FIG. As shown in the figure, the voice response removing unit 2 includes A / D converters 31 and 32 respectively provided on a synthetic input side (voice response) and a microphone input side, and a first smoothing voice response signal power. Smoothing filter 33, second smoothing filter 34, and adaptive / stop switching unit 35 for determining whether or not to perform the adaptation based on the output signals of the respective smoothing filters.
, Adaptive filter 3, and convolution operation unit 36
And a subtracting section 4.

The time constant of the first smoothing filter 33 is set small, for example, the time constant t ₁ is 10 [ms].
Is.

The time constant of the second smoothing filter 34 is set large, and for example, the time constant t ₂ is 100 [m.
s].

The adaptive / stop switching unit 35 stops the adaptation by the adaptive filter 3 when the output of the first smoothing filter 33 becomes equal to or lower than _a predetermined threshold value V _a , and the second smoothing is performed. It operates so as to start the adaptation when the output of the filter 34 exceeds a predetermined threshold value V _b .

FIG. 13 shows the power information of the voice "Please". The figure (a) shows the output of the first smoothing filter 33, and the figure (b) shows the second smoothing filter. The output of 34 is shown. Needless to say, the output signal of the second smoothing filter 34 is smoother due to the difference in time constant.

FIG. 14 is a diagram in which the outputs of the filters 33 and 34 near the point where the sound is interrupted in the voice output "Please" are overlapped. Usually, when the power of the voice changes greatly like the part between the silent part and the voice part, the estimation accuracy of the transfer function is within a short time, for example, 1 [msec].
Falls sharply during. Therefore, a high estimation accuracy can be maintained by stopping the adaptation as soon as the power of the voice changes significantly. Therefore, as shown in FIG. 14, the output P _{a of} the first smoothing filter 33 is
The adaptation is stopped when (t) becomes equal to or _smaller than the threshold value V _a, and the adaptation is performed when the output P _b (t) of the second smoothing filter 34 becomes equal to or larger than the threshold value V _b. Once started, no adaptation is done when the power of the voice changes significantly. This makes it possible to maintain high estimation accuracy.

FIG. 10 shows the estimation result of the impulse response when the synthetic speech "Welcome" is input. The curve S11 shows the case where the adaptive estimation stop is performed and the curve S12 does not. It is an estimation result. As is clear from the figure, it is understood that the impulse response can be estimated with higher accuracy by performing the stop.

FIG. 11 shows the recognition result of the voice after the response is removed. As is clear from the figure, the higher the impulse response accuracy, that is, the larger the amount of synthesized speech removed, the higher the speech recognition rate, and the effect of the synthesized speech removal can be understood. Further, the recognition method is not limited to the combination of the keyword spotting and the noise immunity learning described above, and may be word speech recognition or continuous speech recognition method by HMM.

FIG. 15 is a flow chart showing the operation for obtaining the step gain which is the coefficient for updating the filter in the third embodiment.

First, at time k = 0 (step ST
31), the output power p of the first smoothing filter 33
It is determined whether or not _a (k) is less than or equal to a threshold value V _a (for example, V _a = −20 dB which is the average power of the synthesized sound) (step ST32). If it is determined that the threshold value is equal to or lower than the threshold value V _a (YES in step ST32).
S), μ of LMS is set to 0 (step ST36) so that the transfer function is not updated. This is easily understood from the above equation (4), and W _k is not updated when the focusing coefficient μ = 0.

On the other hand, when it is determined that the power p _a (k) is the threshold value V _a or more (NO in step ST32),
Next, the output power p _b (k) of the second smoothing filter 34
Is less than or equal to the threshold V _b (step ST33). If it is determined that the threshold value is equal to or lower than the threshold value V _b (YES in step ST33), the focusing function μ is set to 0 (step ST37) and the transfer function is not updated. That is, it shows the "stop" portion in FIG.

Then, the power p _b (k) is equal to the threshold V _b.
When the above is reached (NO in step ST33), the step gain is calculated by the following equation (5).

[0059]

## EQU00004 ## Step gain = 2 .mu..e (k) / { _p.sub.b (t) .L} (5) Thus, the transfer function is updated.

In this way, in the third embodiment, when the power of the audio signal is reduced, the adaptation is stopped, so that high estimation accuracy can be maintained.

In this embodiment, the smoothing filter is set to 2
Although the configuration is provided individually, the configuration is not particularly limited to this, and it is obvious that one, or three or more smoothing filters can be used.

Further, when estimating the transfer function, it is necessary that the synthetic speech which is the input signal of the adaptive filter and the microphone signal which is the desired output are always obtained with a constant time difference. That is, the synthetic sound component in the microphone signal has a time difference from the synthetic sound output from the speaker by the propagation delay of the acoustic transfer system, and this must be preserved when the transfer function is estimated. When the synthesized voice of the input signal is directly obtained from the inside of the computer, the synthesized voice held inside the computer may not be output from the speaker at an expected timing due to the load of the computer or an unexpected malfunction. Even in such a case, since the transfer function is estimated stably, as shown in FIG.
By obtaining the microphone signal and the synthesized voice signal by the converters 31 and 32, it is possible to obtain two signals at a fixed timing.

Since the transfer function estimation requires a large amount of calculation,
The voice response removal unit can be configured using a DSP board to finish the calculation in real time.

FIG. 12 is an external view of a voice dialogue system equipped with a synthetic voice removing device. The user inputs a voice into the microphone 23, and the synthesized voice response of the system is output from the speaker 21. The AD converter uses a sampling frequency of 12 [kHz] in consideration of the band of the audio signal. The user proceeds with the dialogue while watching the auxiliary information on the monitor 22, but since the synthesized voice is canceled by the synthesized voice removing device and only the voice of the user is input to the voice recognition device, the user is You can interrupt and input voice even when the system is responding. At this time, the microphone may be a directional one so as not to pick up the synthesized voice from the speaker as much as possible, but the reflected sound from the surrounding wall remains, so the synthetic voice can be obtained only by using the directional microphone. It cannot be erased. Also, in order to improve the SN ratio of the input voice, as close to the microphone as possible,
For example, it is desirable to speak within a distance of about 30 cm from the microphone, but the synthesized sound reflected by the user's body will enter the microphone. This level has the highest level in the reflected sound because the user and the microphone are close to each other, and the amplitude and time delay change depending on the movement of the body. Even in the above cases, since the transfer function is updated by the adaptive filter, changes in the transfer function due to reflections from surrounding walls, user movements, or other people's movements can be tracked effectively, and a synthetic sound can be effectively generated. Can be removed.

Next, a fourth embodiment of the present invention will be described. This is an example of preventing the system from erroneously detecting a synthetic sound.

FIG. 16 is a block diagram showing the structure of the fourth embodiment. As shown in the figure, the voice interaction system is provided with a voice detection unit 31 on the output side of the subtractor 4.

The voice detection unit 31 determines whether or not there is voice input based on the output signal of the subtracter 4, the background noise, and the signal in which the synthesized voice to be removed remains by mistake. As shown in FIG.
And a voice determination unit 33 and an impulse response estimation unit 34.

The impulse response estimation unit 34 is arranged in the speaker 8
The impulse response between the microphone 1 and the microphone 1 is estimated, and this is supplied to the detection threshold value determination unit 32.

The detection threshold value determining unit 32 sets a threshold value for determining whether or not the output of the subtractor 4 is a voice input based on the impulse response and the synthesized voice output from the speaker 8. decide.

The voice determination unit 33 determines whether or not the input signal is a voice input, based on the duration of the signal exceeding the threshold value and the like, as will be described later.

A detailed description will be given below with reference to FIG.
This figure shows an example of the detection parameters used for voice detection, where the beginning of the voice is A and the end is B. The background noise power Po is measured in advance, and a value obtained by adding a margin Ms for determining the starting point, for example, 5 dB to the starting point detection threshold P
A value obtained by adding s and the margin Me for determining the end, for example, 3 dB is defined as the end detection threshold Pe. Also, the voice duration Ts for determining the start edge is, for example, 20 ms, the silent duration Te for determining the end is, for example, 200 ms, and the minimum voice duration T.
For example, v is set to 200 ms.

Then, the input signal power is calculated at a certain time interval, for example, every 10 ms, and while comparing with the detection threshold value each time a new value is obtained, for example, the state of the detected state is changed according to the state transition diagram of FIG. Transitions can be made and voice detection can be performed. The time is represented by a multiple of the power calculation time interval, and the time measured from the starting point A in FIG. 19 is ns,
The time measured from the end is ne. Further, time is represented by i, and power at time i is represented by Pi. Also,
The arrow indicates the transition destination of the state, and the expression beside the arrow indicates the transition condition. The number of states is 6, and there is a silent state (S0) that represents a state in which no voice is input, a start-end assumed state (S1) that represents a state in which a provisional start end is defined, and a start-end defined state (in which a start end is defined) S2), a voice confirmation state (S3) indicating that the voice is confirmed, an end assumption state (S4) indicating a state in which the temporary end is determined, and a voice continuation indicating that the voice is still continuing. There is a state (S5), a termination confirmed state (S6) that represents a state in which the termination is determined and voice detection is completed.

First, when there is no voice input, there is no sound (S0).
Is in the state, the power Pi at a certain time i _s are defined time i _s the start of the temporary exceeds leading end detection threshold Ps, transitions to start assumed state (S1). When it does not exceed Ps, it remains silent (S0).

From the time when the starting end assumption state (S1) is reached, n
Introduction Measure s, defines the time i _s to be starting beginning commit state if the power ns while beyond leading end detection threshold Ps is equal to or higher than the sound duration Ts for starting determination (S2) Transition to. Until the time Ts elapses, the starting end assumption state (S1) is maintained. If the power falls below the start edge detection threshold Ps before the time reaches Ts, the state transitions to the silent state (S0). Next, the start end is determined (S2)
Power regarded as input signals from the time i _s to date when leave time ns than Ps is equal to or greater than the minimum sound duration Tv is audio in, audio definite state (S
Transition to 3). When the power falls below Ps before reaching Tv, the state transits to the silent state (S0).

Then, when the power is lower than Pe in the voice fixed state (S3), it is assumed that the time i _{e at} this time is the end, and the state is changed to the end presumed state (S4). The time length parameter ne for determining the end is started from time i _e . If the power is Pe or higher, the voice remains in the determined state (S3). After that, assume end state (S4)
If the ne becomes equal to or longer than the silent duration Te for determining the end while the power is lower than Pe, it is determined that the end has been determined, the state is changed to the end determination state (S6), and the detection process is ended. If the power P becomes Pe or more before reaching Te, the state transitions to the voice continuation state (S5). Next, when the power Pi is lower than Pe in the voice continuation state (S5), it is assumed that the time i _e ′ at this time is the termination, and the transition to the termination assumed state (S4) is made. Power is P
In the case of e or more, the voice continuation state (S5) remains.
In this way, the voice input is recognized.

Next, a method of detecting a voice when there is a voice response, that is, when the voice response from the speaker 8 is not completely removed will be described. When a voice response is output, the input signal level rises by the amount of the synthesized voice, and thus the false detection of the voice can be eliminated by raising the detection threshold value accordingly. For safety reasons, if the threshold value is increased with a large constant value so as not to be detected even when a high-level synthetic speech is input, the detection performance in the case of no voice response is deteriorated. Therefore, in order to keep the detection performance high at all times, it is desirable to set the threshold value every hour in accordance with the power of the response voice with a minimum increment. The threshold value setting method according to the power of the voice response will be described below with reference to the time chart of FIG.

First, in the absence of voice input, the background noise power Po is measured (step ST41), and a synthesized sound is output for a fixed time, for example, 3 seconds to estimate the impulse response between the speaker and the microphone (step ST4).
2). Since the impulse response estimation is performed by the response voice removing unit 2, the result can be used and it is not necessary to newly provide an estimating unit (step ST43). Next, the voice response signal is convoluted with the estimated impulse response to obtain the synthetic sound component in the microphone signal and its power Ps (step ST44). Synthetic sound power Ps and background noise power Po
By setting the sum P of and P as the base level Pb for voice detection, it is possible to set the threshold value according to the synthesized voice power (step ST45). After time i = 0, the power calculation can be reduced by performing the power calculation at regular time intervals, for example, every 10 ms. At this time, the new impulse response estimated by the response voice removing unit 2 is used to change the acoustic system. Can also be used. The synthesized voice is the voice response removal unit 2
The estimated value Ps of the voice response power can be set to a smaller value because it has been deleted by, but when the acoustic system is changing, the impulse response estimation cannot follow the change of the acoustic system. Since the erasing rate may decrease, it is safe to use Ps as it is.

Next, an example of highly accurate impulse response estimation will be described. It is known that the voice signal which is the input of the adaptive filter has a non-flat frequency spectrum, and therefore the convergence speed of the adaptive filter by the LMS algorithm becomes slow. Therefore, it is possible to increase the S / N of all frequencies by adding wideband noise to the synthesized speech, and perform highly accurate estimation of the transfer function. At that time, the noise power can be prevented from being offensive to the user by changing the noise power according to the response voice signal power. In particular, noise is likely to be noticed in the silent part, so the noise amplitude should be set to 0.

The noise to be added is environmental noise in the place where the system is used, for example, noise of crowded people at a station or noise in a computer room is recorded, or similar noise is continuously generated at a constant amplitude. Even if it outputs it, it can be made not to be offensive.

It is also known that the decrease in the convergence speed when the adaptive filter is driven by the audio signal is also improved by flattening the spectrum of the input signal. Although an inverse filter is usually used for flattening, the power biased to the low frequency component can also be corrected by taking an input difference signal. Since the difference processing is a very simple process, the amount of calculation is small, which is convenient for a real-time system. FIG. 21 shows the frequency spectrum of the "I" sound of "Welcome" of the synthetic sound. The curve a shows the original spectrum after the difference processing. It is understood that the difference processing makes the power of the middle and high frequency components equal to that of the low frequency region and flattens it.

Also, FIG. 22 shows "Are you sure?"
Is a transfer function estimation result when a synthetic voice is input. A curve c is an estimation result when white noise of a level lower than the voice response power by 20 dB is added, a curve b is an estimation result when difference processing is used, and a d is an estimation result when neither processing is performed. It can be understood that the estimation accuracy is improved by each processing. Furthermore, the curve a is an experimental result when noise addition and difference processing are used together, but it can be understood that the estimation accuracy is further improved by using both of these processing together.

Next, an example of a method of setting the volume of the synthesized sound and the position and orientation of the speaker and the microphone when canceling the voice response using the synthesized sound canceller will be described below.

FIG. 23 shows the relationship between the power of the synthesized sound and the canceling performance. In the figure, a is the erased power, b
Represents the residual power. The larger the synthesized sound, the greater the erasing power, but the greater the residual power, so
It is understood that it is more effective to set the synthetic sound to be small for the voice recognition. In addition, the microphone for voice input and the speaker for output have directivity, and the power of the voice response input to the microphone differs depending on the setting, so that there is a difference in the effect of cancellation. FIG. 24 is a diagram showing the relationship between the direction of the microphone and the cancellation performance.
This is a result of changing the angle φ formed by the microphone and the speaker with the setting as shown in FIG. In the figure, b represents erased power and c represents residual power. The microphone is a widely used unidirectional type, and the blind spot with the minimum sensitivity is in the direction of the grip of the microphone. The erasing power is highest when the head of the microphone is directed toward the speaker, but the residual power is also high. On the contrary, when the blind spot is directed toward the speaker, the residual power is the smallest, and it is understood that it is effective for speech recognition.

FIG. 26 shows the relationship between the distance between the microphone and the speaker and the cancel performance. In the figure, a represents erased power and b represents residual power. It is understood that the larger the distance, the smaller the residual power.

From the above, it is understood that reducing the synthesized sound input to the microphone as much as possible is an effective acoustic system setting for voice recognition. Therefore, (1) the output synthesized sound should be as low as possible within the range that does not interfere with the dialogue, (2) put the speaker so as to be in the blind spot of the microphone, (3) distance the speaker and microphone as far as possible, That is an effective acoustic system setting.

Next, a fifth embodiment of the present invention will be described. The fifth embodiment is a voice dialogue system that takes into account the user's interruption and input during response output from the system, and as shown in FIG. 27, an input recognition understanding unit 41 and dialogue management. A unit 42, a response generation / output unit 43,
The interrupt control unit 44 is included. Then, for example, from the dialog shown in FIG. 28 (b), the dialog in which the interrupt input from the user cannot be received during the response as shown in FIG. 28 (a).
As shown in (c) and (d), the keyword necessary for understanding the meaning of the interrupt input is recognized, or the power of the input voice continues for the minimum voice duration T _V or more and the start edge detection threshold P _S. If it exceeds, it will be detected as if there was an interrupt input. The time required for this detection is T _det . When the response is interrupted when an interrupt is received (b), when the response is faded out when an interrupt is received (c),
Then, when an interrupt is received, a flexible dialogue is made possible such as when the response is output up to a good point (d).

The pattern recognition / understanding unit 41 is for detecting and recognizing an input from the user to understand its contents, and uses a pointing device such as a voice, a keyboard, a mouse or a touch panel as an input medium. . In voice input, the contents of speech are recognized and the meaning is understood by a method such as HMM or keyword spotting. A character string is analyzed by keyboard input, and a pointing device understands its meaning from, for example, point position, moving direction, and moving speed information.

The dialogue management unit 42 uses the pattern recognition understanding unit 4
From the understanding result of the input obtained from 1, the content of the response to be output next is determined. For example, the flow of dialogue is expressed by state transition so that the internal state of the computer is determined from the understanding result of the input, the history of the input, and the response contents of the system immediately before the input, and the response to be output in each predetermined state The content of the response is determined by referring to the content table. Tables 1 to 5 show examples of response contents.

[0089]

[Table 1]

[Table 2]

[Table 3]

[Table 4]

[Table 5] First, Tables 1 to 3 are the response contents of "Display of the list of mails that came yesterday." The example in Table 1 is a normal case where there is no particular emphasis in the response content. Table 2 is an example of the response contents when confirming whether or not it is “Kino”, and the importance of the “Kino” part is increased. Table 3 is an example of the response contents when confirming whether or not to "display", and the importance of the "display" part is increased. Tables 4 and 5 show the response contents for the warning "There is no response from the host panda". Some examples of the response contents are highly important, and examples of the entire response are highly important.

The response generation / output unit 43 processes the response media including voices, for example, phonological processing according to the response contents, generation of acoustic parameters, and generation of a voice waveform according to the response contents determined by the dialogue management unit 42. Response generation using auditory media such as synthetic speech,
Generates and outputs a response using visual media such as graphics showing the internal state of the system according to the same response sentence as the voice response or its summarized contents, or the text and response contents of the words that are the points. . When the response content is passed from the dialogue management unit 42, the response output and the response output position information indicating the output timing are determined, and the response output is started accordingly. An example of the response output position information is shown in Table 6 and FIG.

[0091]

[Table 6] In this example, only the voice response is described, but the application is not limited to this, and response output position information indicating the same output timing can be determined for other auditory media or visual media.

In the case of a voice response, the response output position information is, for example, a sentence, a clause, a phrase, a clause, a word, a syllable, or a sequence forming a semantic unit of a plurality of these as a synthesis unit. It is a list of data indicating the composition unit and its output time. Such a list of output times for each synthesis unit can be easily created from the speech rate, the duration of the synthesis unit, and the response output start time.
With this response output position information, as shown in FIG. 29, when there is a user interrupt in the middle of response output, the time at which the interrupt occurred can be known in association with the response output. Outputs interrupt control information, for example, aborts response output or fades out,
You can change the response generation parameters.

The response generation / output unit 43 can generate a voice response by a known method, for example, Tsune Kawai: “Speech Synthesis System from Japanese Text”, The University of Tokyo Dissertation (Showa 63).
30), the values of the response generation parameters such as the speech rate, the prosody, and the power of the voice response are determined by the speech rate determining unit 45 and the prosody determination, respectively, as shown in the configuration example in FIG. In the unit 46 and the power determination unit 47,
Determined according to the response content. The response generation parameter value is
Determined when generating acoustic parameters. The power value can be changed after the waveform is generated, as described later. For example, as will be described later, if the response content is highly important, the utterance speed is slowed down, the intonation variation range is increased, and the power is increased. The width of change of intonation can be determined by a known method, for example, Fujisaki and Sudo: "Fundamental frequency pattern of Japanese word accent and model of its generation mechanism", The Acoustical Society of Japan, 27, 9, pp4.
It can be easily controlled by the method of 45-453 (Showa 46).

Further, when the response generation / output section 43 receives the response interrupt control information from the interrupt control section 44 as shown in the configuration example of FIG. 30, whether the response generation output section 43 terminates the response including the voice being output. , The response generation parameters including the speech rate, prosody, and power of the voice response being output are changed. When the response is aborted, the synthesis unit being output is output and the output is aborted there. When the synthesis unit is a syllable, for example, the response is output up to the boundary immediately after the syllable or word or phrase being output. As described above, there are various cases in which the unit of composition is possible, and the method of selecting the place to cut off the output is not limited to this. In such a response interruption method, the response output can be naturally cut off by setting the synthesis unit to a syllable, a word, a phrase, a phrase, or the like. In the case of rule synthesis etc., it synthesizes in units such as phonemes, words, clauses, phrases, etc.When aborting midway, it is interrupted so that the response ends up to the synthesis unit being output, and the recorded voice is recorded. In the case of reproduction, the response may be terminated as it is when the output of the voice unit being output is finished. When changing the response generation parameter, the speech rate determining unit 45 changes the speech rate by, for example, ± 30%, or the prosody determining unit 46 changes the rate of change of intonation corresponding to the accent phrase by ± 50%. The power determination unit 47 prepares an attenuation curve that fades out so that it becomes 0 after 1 second, for example, and convolution of the response output waveform, or when generating the acoustic parameter, this attenuation curve is used for the time change of the power. It is controlled by a method such as folding. It is possible to prepare a plurality of attenuation curves for censoring and fade-out. When the output becomes 0 as a result of the convolution, the response output is considered to be completed and the next process is performed. It should be noted that the examples of the values of these change rates may change depending on the application, and are not necessarily limited thereto.

Table 7 shows the interrupt control information, which is shown in FIG.
Shows the response output when the response is terminated, and FIG. 7B shows the response output when the response is terminated in the fourth output unit. Further, FIG. 32A is a flowchart showing the response abort control, and FIG. 32B is a flowchart showing specifically the generation and output of the n-th response of the response content. In this example, generation of a voice synthesis response using a CV syllable parameter as a synthesis unit is shown. Depending on the application, it is also possible to use the CVC syllable parameter as a synthetic segment or reproduce recorded voice, and the method of response generation and output is not limited to this.

[0096]

[Table 7] In such a control flow, the timing to cut off or fade out the response, or the timing to start changing the response generation parameter value is specified by the interrupt control information. For example, when changing the speech rate, the speech rate is changed from the timing designated by the interrupt control information as shown in FIG. In this example, the speed increases from the fourth response in the response content. The value may be changed for each composition unit, but may be changed to a smooth target value from the designated timing. The case of prosody control is shown in FIGS. 34 and 35. FIG. 34 shows an example in which the prosody change is normal, and FIG. 35 shows a large change from the fourth response of the response content. When playing back the recorded voice, several kinds of synthetic pieces with different prosodic variation widths are prepared, and the interruption control information is received, and a piece according to the variation width is selected and reproduced.

Further, FIG. 36 shows an example of power control, and this power control curve is folded into the power parameter value or used as the offset value of the power parameter. The figure (a) shows an example in which the power increases from the fourth response, the figure (d) shows the example in which the power decreases from the fourth response, and the figure (c) shows an example in which the power fades out from the fourth response. Is. For parameters such as power that causes noise essentially when abruptly changed with time, a smooth curve, such as a step response curve of a critical braking system, a polynomial curve, or a curve with a trigonometric function, is convolved. .

On the other hand, the interrupt control section 44 outputs the response interrupt control information according to the flow of each flowchart shown in FIGS.

FIG. 37 shows an example in which the interrupt is not permitted when the length of the non-output response is small. During the response output (YES in step ST51), the length of the non-output response is not less than the reference value. It is determined whether or not there is (step ST5).
2). The reference value is determined in units of the number of synthesis units, the number of moras, the number of words, the number of clauses, and the like. For example, the value is set to 8 mora, 3 words, or one synthesis unit. If it is equal to or greater than the reference value (YES in step ST52), it is considered that necessary information has already been output, and control such as response termination is performed (step ST53).
On the other hand, when the length of the non-output response is less than the reference (NO in step ST52), the non-output response is output as it is (step ST54). After that, the content of the next response is determined and the response is generated and output (step ST55).

FIG. 38 shows an example in which if the contents of the response being output are important, control is performed so that the response is output without interruption.
During response output (YES in step ST61), the importance of the response content being output is determined (step ST62). If it is important (N in step ST62,
O), for example, control such as reducing power or slowing speech rate is performed (step ST62). If the response content being output is important (step ST62)
YES), a non-output response is output (step ST6).
4). After that, the contents of the next response are determined and a response is generated and output (step ST68). As described above, the importance of the response content can be determined for the entire response or for each synthesis unit that is a part of the response. Specific examples of each case will be described later.

FIG. 39 is an example of controlling by comparing the importance of the understanding content of the interrupt input with the importance of the response content being output.
That is, the input contents from the speaker and the response contents from the speaker are compared to give priority to the important one.

Now, during response output (step ST71
YES), the importance of the input understanding content and the output understanding content are compared (step ST72). As a result, if the input understanding content is more important (YES in step ST72), the response output is controlled by reducing the power of the response output or slowing the speech rate (step ST74). When the output understanding content is more important (NO in step ST72), the non-input response is output as it is (step ST73). After that, the content of the next response is determined and a response is generated and output (step ST7).
5).

FIG. 40 shows an example in which the interrupt is not controlled while important contents are included in the non-output response. Now, during response output (YE in step ST81)
S) It is determined whether or not there is important content in the non-output response (step ST82). Then, if there is important content (YES in step ST82), response generation and output of the unoutput portion is performed (step ST83), and the process is repeated until important content is output. When important contents are output (NO in step ST82), the response output is interrupted, for example, by aborting the response (step ST84).
After that, the content of the next response is determined and a response is generated and output (step ST85).

When the understanding result of the pattern recognition and understanding unit 41 is used, as shown in Table 8, the importance of the contents of the interruption utterance of the user is evaluated.

[0105]

[Table 8] For example, the input content importance is evaluated so that the utterances meaning correction are higher than the syntactics and the utterances requiring interruption of the response are higher than the normal interrupt utterance. When the evaluation result of the importance of the understanding result of the input is low, for example, when there is an interrupt that does not require interruption of the response during output such as mutual cooperation, the response being output is output as it is. When the evaluation result is normal or important, the response interrupt control information for interrupting the response being output or changing the response generation parameter is output. For example, if there is an interrupt requesting to interrupt the response, either interrupt the response or
Alternatively, the speech speed is increased to end the response earlier. Note that the understanding contents and examples of importance shown in Table 8 are merely examples, and are not limited to these depending on the application.

When using the response contents being output by the response generation / output unit 43, the priority of the response output is evaluated with reference to the importance of the response contents and the interrupt timing. As shown in Tables 1 to 5, examples of the priority of the response output are shown in Tables 9 to 12 by referring to the importance of the response composition unit or the entire response content. Evaluate to.

[0107]

[Table 9]

[Table 10]

[Table 11]

[Table 12] For example, if there is an interrupt when the content of the response is a message that warns the user or sends a message of high urgency to the user, that is, if the priority of the response output is high, the interrupt is generated as shown in the example of FIG. Do not accept input. Alternatively, if there is an interrupt during the output of a warning or an extremely urgent response, or if the priority of the response output is extremely high, response interrupt control information that slows down the speaking speed and raises the pitch power is displayed. Output. By doing this, it is possible to convey that the response from the system is of extremely important content that does not allow interruption. If an interrupt occurs when the priority of response output is high to some extent, response interrupt control information is output so that the speech speed is increased and pitch power is increased. By outputting such a response in the case of outputting a general warning or a message with a relatively high degree of urgency, although it is not possible to stop the response immediately in response to the interrupt, it informs that the interrupt is being dealt with as soon as possible. be able to. In addition, the response contents shown in Table 9,
The importance is just an example, and is not limited to this depending on the application.

Next, the processing of each unit when there is an interrupt input will be described step by step. The content of the response from the system is determined by the dialogue control unit in the form of the response content illustrated in Tables 1 to 5. According to this, the response generation and output unit first uses the speech rate determination unit, the prosody determination unit, and the power determination unit for the speech rate,
Seeking prosody and power. In the case of a normal response, the speech rate is set to, for example, about 7 mora per second, and the prosody is a known method, for example, Hirose, Fujisaki, Kawai, Yamaguchi "Sentence voice based on the fundamental frequency pattern generation process model. Synthesis of the Institute of Electronics, Information and Communication Engineers, A, vol. J72-
A, No. 1, pp32-40 (January 1989). According to this speech rate, the response output position information, an example of which is shown in Table 6, is generated from the time length of the synthesis element and the response output start time. At the same time, a response is generated and output is started. When there is an interrupt input from the user, the pattern recognition understanding unit detects this input, notifies the interrupt control unit, and understands its meaning. When notified of the input detection, the interrupt control unit checks the interrupt input timing by collating with the response output position information. If the interrupt input timing is after completion of the response output, the interrupt control unit does not output the response interrupt control information, and the dialogue control unit determines the next response content. When the interrupt input timing is before the completion of the response output, the response interruption is performed by using one or both of the understanding result of the pattern recognition understanding unit 41 of the input and the response content being output by the response generation output unit 43. Output control information. The response interrupt control information is sent to the speech rate determination unit, the prosody determination unit, the power determination unit, and the response cutoff control unit to control the speech rate, cut off the response, and fade out the power as described above. do. The response interrupt control information also includes information about when to change the response output,
For example, the response output is changed from the next synthesis unit being output among the response contents.

[0109]

As described above, according to the first aspect of the present invention, even when the voice response of the user is superimposed on the voice response and input from the microphone, the voice response is removed and only the voice signal is recognized. It Therefore, the utterance from the user can be recognized even when the voice response is output from the speaker. As a result, the effect that extremely smooth dialogue is possible is obtained. Further, it is also extremely effective especially in a multimedia system in which visual information such as graphic information, images, and animation is displayed to interact with the user. Moreover, if the adaptation is stopped when the power of the audio signal is reduced, the estimation accuracy of the transfer function does not decrease, and high estimation accuracy can be maintained at all times.

Further, in the second aspect of the present invention, the threshold value for recognizing the voice input is changed according to the power of the voice response taken from the microphone. Therefore, erroneous input can be prevented, and highly accurate voice recognition can be performed.

Further, in the third aspect of the present invention, when there is an interruption from the user during the voice response output, the voice response output is continued, aborted, or continued halfway depending on the input content. Controls whether or not. As a result, it is possible to quickly move to the next response, and it is possible to obtain an effect that an advanced dialogue according to the input content becomes possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of a voice dialogue system to which the present invention is applied.

FIG. 2 is a diagram showing a removal characteristic of a voice response.

FIG. 3 is a flowchart showing the operation of the first embodiment.

FIG. 4 is a flowchart showing an operation for determining a step gain μ (f).

FIG. 5 is a time chart showing changes over time in power and pitch of voice response.

FIG. 6 is a block diagram showing an internal configuration of a voice response unit.

FIG. 7 is a time chart showing a time response of a voice response and a speech signal of a user.

FIG. 8 is a block diagram showing a configuration of a second embodiment of a voice dialogue system to which the present invention is applied.

FIG. 9 is a block diagram showing a configuration of a third exemplary embodiment of the present invention.

FIG. 10 is a characteristic diagram showing the estimation accuracy of a transfer function.

FIG. 11 is a characteristic diagram showing a relationship between estimation accuracy and voice recognition rate.

FIG. 12 is a diagram showing an appearance of a voice dialogue system.

FIG. 13 is a diagram showing the output power of each smoothing filter.

FIG. 14 is an explanatory diagram showing a stop period of adaptation.

FIG. 15 is a flowchart showing the operation of the third embodiment.

FIG. 16 is a block diagram showing a configuration of a fourth exemplary embodiment of the present invention.

FIG. 17 is a block diagram showing details of a voice detection unit.

FIG. 18 is an explanatory diagram showing threshold values when recognizing a voice signal and voice.

FIG. 19 is a state transition diagram when recognizing a voice.

FIG. 20 is a flowchart showing an operation of changing a threshold value.

FIG. 21 is a characteristic diagram showing an original spectrum and a spectrum after difference processing.

FIG. 22 is a characteristic diagram showing a transfer function estimation result when a synthetic voice “Is this all right?” Is input.

FIG. 23 is a characteristic diagram showing the relationship between the power of synthetic speech and the canceling performance.

FIG. 24 is a characteristic diagram showing the relationship between the microphone orientation and the canceling performance.

FIG. 25 is an explanatory diagram showing a positional relationship between a microphone and a speaker.

FIG. 26 is a characteristic diagram showing a relationship between the cancel performance and the distance between the microphone and the speaker.

FIG. 27 is a block diagram showing a configuration of a fifth exemplary embodiment of the present invention.

FIG. 28 is a time chart showing output timings of voice response and voice input.

FIG. 29 is a time chart showing the timing of an interrupt utterance and a response output.

FIG. 30 is a block diagram showing a detailed configuration of a response generation / output unit.

FIG. 31 is a time chart showing response outputs with and without response termination.

FIG. 32 is a flowchart showing the flow of response abort control.

FIG. 33 is a time chart showing an example of increasing the speech rate.

FIG. 34 is a time chart showing each signal when the prosody changes are the same.

FIG. 35 is a time chart showing each signal when a prosody change becomes large.

FIG. 36 is a time chart when the power is changed.

FIG. 37 is a flowchart showing an operation of prohibiting interrupt control when the amount of non-output response is small.

FIG. 38 is a flowchart for controlling not to interrupt when the response content during output is important.

FIG. 39 is a flowchart for deciding whether or not to permit interrupts according to the importance of interrupt contents and output contents.

FIG. 40 is a flowchart for controlling to prohibit interrupts when important contents are included in a non-output response.

[Explanation of symbols] 1 microphone 2 Voice response removal unit 3 Adaptive filter 3a lookup table 4 subtractor 5 Speech recognition unit 7 Voice response section 8 speakers 10 Speech synthesizer 11 Music Synthesis Department 15 Transfer function update control unit 31 A / D converter 32 A / D converter 33 First smoothing filter 34 Second Smoothing Filter 35 Adaptation / stop switching unit 37 voice detector 38 Detection threshold value determination unit 39 Voice judgment unit 40 Impulse response estimation unit 41 Input recognition and understanding section 42 Dialog Management Department 43 Response generation / output section 44 Interrupt control unit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI G10L 15/28 21/02 (72) Inventor Shigenori Seto 1 Komukai Toshiba-cho, Kouki-ku, Kawasaki-shi, Kanagawa Toshiba Research Institute Co., Ltd. (72) Inventor Yasushi Yamashita 8-6-26 Motoyama-cho, Higashinada-ku, Kobe-shi, Hyogo Inside Toshiba Kansai System Center (56) References JP-A-25-250099 (JP, A) JP-A-59-195739 ( JP, A) JP 60-216392 (JP, A) JP 61-262798 (JP, A) JP 60-114900 (JP, A) Jitsukaihei 2-83600 (JP, U) (58) ) Fields surveyed (Int.Cl. ⁷ , DB name) G10L 13/00 G10L 21/02

Claims (4)

(57) [Claims] 1. A recognizes the voice input from the input means such as a microphone, a predetermined audio response based on the recognition result in the speech recognition method for performing the conversation outputted from the output unit such as a speaker, A transfer function which is the frequency spectrum of the impulse response of the voice response output from the output means is estimated using a response generation parameter for synthesizing the voice response, and the transfer function used when estimating the impulse response Estimation
When a value is stored and a voice is detected from the input means,
Estimate impulse response from stored past estimates
Or update the estimate to estimate the impulse response.
The voice response is corrected by the impulse response estimated as a result of the determination, only the corrected voice response is canceled from the input voice, and the voice after canceling the voice response is recognized. A speech recognition method characterized by the above. 2. Input from input means such as a microphone
The recognized voice is recognized, and the predetermined voice is recognized based on the recognition result.
The response is output from the output means such as a speaker and the dialogue is performed.
In the voice recognition method for listening, the impulse response of the voice response output from the output means.
The transfer function, which is the frequency spectrum of
Estimating using the response generation parameter for synthesis , smoothing the voice signal power of the voice response, and estimating the transfer function when it is determined that no voice response is output based on the smoothed voice signal. The value is not updated, and the voice response is corrected by the estimated impulse response.
Then, only the corrected voice response is captured from the input voice.
And recognize the voice after canceling the voice response.
Characteristic voice recognition method. 3. The voice signal power of the voice response is smoothed, and when it is determined that a voice response is not output based on the smoothed voice signal, the estimated value of the transfer function is not updated. The voice recognition method according to claim 1. 4. A means for obtaining a background noise power in the absence of voice input, a means for obtaining a synthesized sound power in a microphone signal based on an impulse response at the time of synthesized speech output, the background noise power and the synthesized speech. A sum of the power and a voice input power detection threshold is used, and means for determining whether or not there is voice input based on the duration of the voice input power exceeding the threshold; Means for performing voice recognition only when it is determined that there is a voice interaction device.