JP2015194766A - speech recognition device and speech recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech recognition device which selects a start trigger suitable for use environment.SOLUTION: The speech recognition device has multiple pieces of trigger detection means, trigger selection means and recognition means. Each trigger detection means detects a start trigger which instructs to start recognition of a command speech for operating a device. The trigger selection means selects a piece of trigger detection means suitable for the use environment of the device from among the multiple pieces of trigger detection means on the basis of signal(s) from one or more sensors installed in the device. The recognition means starts recognition processing of the command speech when the piece of trigger detection means whose operation is validated by the trigger selection means detects the start trigger.

Description

Embodiments described herein relate generally to a speech recognition apparatus and a speech recognition method.

In recent years, a speech recognition apparatus that recognizes a command (command utterance) uttered by a user and operates a device has been put into practical use. As an instruction to start recognition of command utterance (start trigger), utterance, gesture, applause and the like of a specific keyword by a user have been proposed. In the speech recognition apparatus using these start triggers, the command utterance recognition process is started after the start trigger is detected.

However, the start trigger described above has merits and demerits according to the use environment of the device to be operated, and there is a problem that the start trigger detection accuracy is lowered when a start trigger that is not suitable for the use environment is used. For example, when the surroundings of the device are dark, the accuracy of image recognition is reduced, so that the start trigger by the gesture cannot be detected correctly. In addition, in a speech recognition apparatus that can accept a plurality of start triggers, it is difficult for the user to appropriately select a start trigger suitable for the usage environment.

JP 2010-204266 A JP 2010-193355 A

  The problem to be solved by the invention is to realize a speech recognition device that selects a start trigger suitable for a use environment.

The speech recognition apparatus according to this embodiment includes a plurality of trigger detection means, trigger selection means, and recognition means. The trigger detection means detects a start trigger instructing start of recognition of command utterance for operating the device. The trigger selection unit selects a trigger detection unit suitable for the use environment of the device from the plurality of trigger detection units based on signals from one or more sensors installed in the device. The recognizing unit starts the command utterance recognizing process when the trigger detecting unit activated by the trigger selecting unit detects the start trigger.

1 is a block diagram illustrating a voice recognition device according to a first embodiment. The figure which shows the hardware constitutions of the speech recognition apparatus of embodiment. The flowchart of the applause trigger detection part of embodiment. The figure which shows an example of the applause detected by the applause trigger detection part of embodiment. The flowchart of the speech recognition apparatus of embodiment. The flowchart of the trigger selection part of embodiment. The flowchart of the trigger selection part of the modification 1. The figure which shows an example of the image displayed on the display of the television receiver. The figure which shows an example of the image displayed on the display of the television receiver.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
The speech recognition apparatus according to the first embodiment is an apparatus that recognizes a user's command utterance and operates a device. The voice recognition device is built in the television receiver, and the user can instruct the switching of the channel of the television receiver, the search of the program guide, etc. by command utterance.

The voice recognition device according to the present embodiment does not require an operation such as pressing an utterance button for an instruction (start trigger) for starting recognition of a command utterance, and can be selected from three types of start triggers: a gesture trigger, a voice trigger, and a clap trigger. Then, a start trigger suitable for the use environment of the television receiver is selected. Here, the gesture trigger is a user's specific gesture, the voice trigger is a user's specific keyword utterance, and the applause trigger is a user's applause as a start trigger.

FIG. 1 is a block diagram showing a speech recognition apparatus 100 according to the first embodiment. The voice recognition device 100 according to the present embodiment includes a voice trigger detection unit 101 that detects a start trigger by a user's specific keyword utterance from sound acquired by a microphone 208, and a user's specific gesture from an image captured by a camera 209. A gesture trigger detection unit 102 for detecting a start trigger by the user, a clap trigger detection unit 103 for detecting a start trigger by a user's applause from the sound acquired by the microphone 208, and a volume sensor 210 for measuring the volume around the television receiver, Based on the signals from the distance sensor 211 that measures the distance from the television receiver to the user and the light amount sensor 212 that measures the amount of light around the television receiver, among the above trigger detection units, a start trigger suitable for the operating environment is selected. A trigger selection unit 104 that validates the operation of the trigger detection unit to be detected, and a trigger selection unit 104 If any of the trigger detection unit that is enabled operation detects a start trigger, and a recognition unit 105 starts the recognition processing on the obtained command uttered speech waveform by the microphone 208.

  The speech recognition apparatus according to the present embodiment enables the operation of the trigger detection unit suitable for the use environment of the device based on a signal from a sensor installed in the device to be operated (television receiver). Thereby, the start trigger can be detected with high accuracy, and as a result, the recognition accuracy of the user's command utterance can be improved.

(Hardware configuration)
The speech recognition apparatus according to the present embodiment can be configured by hardware using a normal computer as shown in FIG. 2, and includes a control unit 201 such as a CPU (Central Processing Unit) that controls the entire apparatus, and various data. And storage unit 202 such as ROM (Read Only Memory) and RAM (Random Access Memory) for storing various programs and HDD (Hard Disk Drive) and CD (Compact Disk) drive devices for storing various data and various programs An external storage unit 203, an operation unit 204 that receives user instruction input, a communication unit 205 that controls communication with an external device, a microphone 208 that acquires user command utterances, and a camera 209 that captures user gestures A volume sensor 210 that measures the volume around the television receiver, and a distance sensor 21 that measures the distance from the television receiver to the user. When provided with a light amount sensor 212 that measures the amount of ambient light television receiver, and a bus 206 connecting these.

In such a hardware configuration, the following functions are realized when the control unit 201 executes various programs stored in the storage unit 202 such as the ROM or the external storage unit 203.

(Trigger selection part)
Based on the signals from the volume sensor 210, the distance sensor 211, and the light quantity sensor 212, the trigger selection unit 104 selects a trigger detection unit suitable for the use environment of the television receiver among the trigger detection units described later, and the operation thereof Enable

Here, the volume sensor 210 is a sensor that measures the volume around the television receiver, and measures the volume of ambient noise acquired by the microphone 208 and the volume of sound reproduced by the television receiver itself from the speaker. Note that the volume sensor 210 may acquire sound as a time-series digital signal, and the trigger selection unit 104 may calculate the volume (for example, the power of the signal in a predetermined section) from the signal. In this case, the volume sensor 210 can be replaced with the microphone 208.

The distance sensor 211 is a sensor that measures the distance from the television receiver to the user. A human sensor that determines whether or not a person is present within a predetermined distance may be used instead.

The light amount sensor 212 is a light amount sensor that measures the amount of light around the television receiver.

Details of the operation of the trigger selection unit 104 based on signals from these sensors will be described later.

(Voice trigger detector)
The voice trigger detection unit 101 detects a start trigger due to the user's keyword utterance from the sound acquired by the microphone 208.

In voice recognition using a voice trigger, a specific keyword utterance is detected as a start trigger, and then the user's command utterance is recognized (Japanese Patent Laid-Open No. 2001-67091). For example, in the case of using “Hello” as a keyword, when a keyword utterance of “Hello” is detected by the user, a “beep” sound is output to prompt the user to utter a command. Then, the user's command utterance such as “8 channels” is recognized.

The voice trigger detection unit 101 continuously performs recognition processing using a specific keyword as a recognition vocabulary, and determines that the specific keyword is uttered when the obtained reliability score exceeds a threshold value L. The threshold value L is a value that appropriately determines the distribution of the reliability score when a specific keyword is uttered and the distribution of the reliability score when the other keyword is uttered in advance and appropriately distinguishes these two distributions. Can be set to

The voice trigger detection unit 101 always captures the sound acquired by the microphone 208 and performs recognition processing. However, since the recognition vocabulary can be narrowed down to a specific keyword, the risk of erroneous recognition due to ambient noise can be reduced.

  However, when the ambient noise or the output sound of the television receiver is very large or when the user's voice is low, the SNR of the keyword utterance is lowered, so that the start trigger detection accuracy due to the keyword utterance is lowered.

(Gesture trigger detector)
The gesture trigger detection unit 102 detects a start trigger by a user's specific gesture from the video imaged by the camera 209.

In voice recognition using a gesture trigger, a user's specific utterance is recognized after detecting a user's specific gesture as a start trigger (Japanese Patent Laid-Open No. 2010-182014). For example, when an operation of “waving your hand to the left and right” is used as a gesture, if a “hand to shake your hand to the left and right” operation is detected by image recognition, a sound of “beep” is output to prompt the user to speak a command. Then, the user's command utterance such as “8 channels” is recognized.

Since the gesture trigger detection unit 102 detects a start trigger using image recognition, the gesture trigger detection unit 102 is not affected by ambient noise, but has a restriction that a user needs to make a gesture within a range that can be captured by a camera. In addition, the gesture recognition accuracy varies depending on the illumination conditions. Furthermore, since it is necessary to always operate an advanced image recognition process, the power consumption becomes larger compared to other trigger detection units.

(Applause trigger detector)
The applause trigger detection unit 103 detects a start trigger caused by a user's applause from the sound acquired by the microphone 208. Here, the applause in the present embodiment is an applause of two consecutive times called “bread and bread”.

  In voice recognition using a clap trigger, when two consecutive claps are detected, a beep sound is output to prompt the user to speak a command. Subsequently, the user's command utterance is recognized.

The process of the applause trigger detection unit 103 will be described using the flowchart of FIG. In this process, as shown in FIG. 4, a sound waveform whose power exceeds the threshold value S twice during a predetermined interval (threshold value T ₀ ) is detected as applause.

Here, the threshold T ₀ can be set in advance to a minimum value in which the distribution of the duration between claps in the case of performing two consecutive applauses is obtained in advance. In addition, the threshold value S is obtained by experimentally obtaining in advance a power distribution when no applause is performed and a power distribution when applause is performed, and an optimum value for appropriately distinguishing the two distributions is used. Can be set.

First, in step S1 of FIG. 3, the time when sound acquisition is started by the microphone 208 is set to t = 0. Here, t represents the frame number when the acquired sound waveform is divided at a frame length of 25 ms and an interval of 8 ms. In step S2, t is updated to t + 1. In step S3, the power of the waveform in the t-th frame is calculated, and the value is compared with a preset threshold value S. If the power exceeds the threshold value S, the process proceeds to step S4. In step S4, T = 0 is set. In step S5, T is updated to T + 1 and t is updated to t + T. In step S6, T is determined whether or not the threshold value T ₀ is less than or a preset. T is to step S7 if T ₀ is less than, otherwise the process proceeds to step S2. In step S7, the power of the waveform in the t-th frame is calculated, and the value is compared with the threshold value S. When the power exceeds the threshold value S, the process proceeds to step S8, and it is determined that two consecutive applause has been detected. Otherwise, the process proceeds to step S2 and the process is continued.

The applause trigger detection unit 103 uses the applause trigger even when the ambient noise is large to some extent, since the applause of “pan, pan” twice used in this embodiment has a special feature compared to other ambient noises. Can be detected.

(Recognition part)
The recognizing unit 105 starts the command utterance recognizing process when any of the trigger detecting units enabled by the trigger selecting unit 104 detects the start trigger. Specifically, the recognition unit 105 starts capturing the sound acquired by the microphone 208 after any trigger detection unit detects a start trigger, and executes recognition processing for the command utterance included in this sound. .

In addition to this, it is also possible that the sound acquired by the microphone 208 is continuously captured and the recognition process is continuously performed so that the recognition unit 105 outputs only the recognition result generated after the start trigger is detected.

(flowchart)
The processing of the speech recognition apparatus according to the present embodiment will be described using the flowchart of FIG.

In step S11, the trigger selection unit 104 receives a television image from the voice trigger detection unit 101, the gesture trigger detection unit 102, and the applause trigger detection unit 103 based on signals from the volume sensor 210, the distance sensor 211, and the light amount sensor 212. Enables the trigger detection unit that is suitable for the operating environment of the machine.

Details of step S11 will be described using the flowchart of FIG. First, in step S21, the trigger selection unit 104 invalidates the operations of all trigger detection units (voice trigger detection unit 101, gesture trigger detection unit 102, applause trigger detection unit 103) for initialization.

In step S <b> 22, the trigger selection unit 104 determines whether or not the distance from the television receiver measured by the distance sensor 211 to the user exceeds a preset threshold D. When the threshold value D is exceeded, there is a possibility that the distance to the user is far and the gesture recognition accuracy in the gesture trigger detection unit 102 is lowered. Therefore, in this case, the gesture trigger detection unit 102 determines that the gesture trigger detection unit 102 is not suitable for this use environment, and proceeds to step S25. When the distance to the user does not exceed the threshold value D, the process proceeds to step S23.

Note that the threshold value D can be set based on the relationship between the distance to the user obtained experimentally in advance and the gesture detection accuracy.

In step S23, it is determined whether or not the light quantity around the television receiver measured by the light quantity sensor exceeds a preset threshold value L. When the threshold value L is not exceeded, the recognition accuracy of the gesture in the gesture trigger detection unit 102 may decrease because the surroundings are dark. Therefore, in this case, the gesture trigger detection unit 102 determines that the gesture trigger detection unit 102 is not suitable for this use environment, and proceeds to step S25.

  On the other hand, if the amount of light exceeds the threshold value L, the process proceeds to step S24, where both the distance to the user and the amount of light are suitable for the gesture image recognition in the gesture trigger detection unit 102, and the operation of the gesture trigger detection unit 102 Enable

The threshold value L can be set based on the relationship between the light amount experimentally obtained in advance and the gesture detection accuracy.

  In step S25, the trigger selection unit 104 determines whether or not the volume around the television receiver measured by the volume sensor 210 exceeds a preset threshold value N. When the threshold value N is exceeded, there is a possibility that the detection accuracy of the keyword utterance in the voice trigger detection unit 101 is lowered because the ambient noise is too large. Therefore, in this case, the voice trigger detection unit 101 determines that the voice trigger detection unit 101 is not suitable for this use environment, and proceeds to step S27.

  On the other hand, if the volume does not exceed the threshold value N, the process proceeds to step S26, and the operation of the voice trigger detection unit 101 is validated on the assumption that the surrounding noise is small and suitable for the keyword utterance recognition in the voice trigger detection unit 101. .

The threshold value N can be set based on the relationship between the volume obtained experimentally in advance and the detection accuracy of the keyword utterance.

  Finally, in step 27, the operation of the applause trigger detection unit 103 is validated. In this embodiment, the operation of the applause trigger detection unit 103 is always enabled. This is because the applause trigger detection unit 103 can detect the start trigger with relatively high accuracy even if the surrounding noise is large or the distance to the user is long.

Returning to the flowchart of FIG. In step S12, the speech recognition apparatus starts the operation of the trigger detection unit validated in step S11.

  In step S13, it is determined whether any of the trigger detection units that have started operation in step S12 has detected a start trigger. If a start trigger is detected, the process proceeds to step S14. When not detected, it waits until any trigger detection part detects a start trigger.

  In step S14, after the start trigger is detected, the process for recognizing the user's command utterance is started.

(effect)
As described above, the speech recognition apparatus according to the present embodiment enables the operation of the trigger detection unit suitable for the use environment of the device based on the signal from the sensor installed in the device to be operated. Thereby, the start trigger can be detected with high accuracy, and as a result, the recognition accuracy of the user's command utterance can be improved.

(Modification 1)
The trigger selection unit 104 of the present embodiment has selected whether the operation of each trigger detection unit is valid or invalid based on signals from a total of three sensors, the volume sensor 210, the distance sensor 211, and the light amount sensor 212. It is also possible to select the operation of the trigger detection unit using one sensor. For example, if there is the volume sensor 210, the validity / invalidity of the operation of the voice trigger detection unit 101 can be selected using the same process as in step S25 of FIG.

  Further, based on the signal from the distance sensor 211, it is possible to select whether the operation of the voice trigger detection unit 101 is valid or invalid. In this case, when the distance measured by the distance sensor 211 becomes equal to or less than the threshold value D, the operation of the voice trigger detection unit 101 is validated. This is because when the distance is small, the user's voice received by the television receiver becomes louder, so that the start trigger detection accuracy in the voice trigger detection unit 101 becomes higher.

  In addition, the trigger selection unit 104 can select the validity / invalidity of the operation in each trigger detection unit using a control signal from other than the sensor. As a control signal from other than the sensor, there is a mode (power mode) regarding power consumption instructed by the user. For example, when the user selects a mode with low power consumption (power saving mode), the trigger selection unit 104 can invalidate the operation of the gesture trigger detection unit 102 that increases power consumption during normal operation.

  FIG. 7 is a flowchart showing the operation of the trigger selection unit 104 when the power mode is used. In step S31 of this flowchart, the trigger selection unit 104 determines the power mode instructed by the user. When the power mode is a normal mode (normal mode), the process proceeds to step S22, and valid / invalid of operation in each trigger detection unit including the gesture trigger detection unit 102 is selected. On the other hand, when the power mode is the power saving mode, the process proceeds to step S25 so that the operation of the gesture trigger detection unit 102 with high power consumption is not enabled.

  As described above, the speech recognition apparatus according to the present modification can select whether the operation of the trigger detection unit is valid or invalid based on the control signal that cannot be acquired by the sensor.

(Modification 2)
The speech recognition apparatus according to the present embodiment can present to the user the selection result of the validity / invalidity of the operation of each trigger detection unit in the trigger selection unit 104 via a presentation unit (not shown). As the presentation unit, a display of a television receiver or the like can be considered.

  8 and 9 show images displayed on the display 400 of the television receiver. For example, the mark 401 in FIG. 8 indicates that the operation of the voice trigger detection unit 101, the mark 402 indicates that the operation of the applause trigger detection unit 103, and the mark 403 indicates that the operation of the gesture trigger detection unit 102 is effective. That is, in the state of FIG. 8, the user can instruct the start of recognition of command utterance using all start triggers.

  On the other hand, in FIG. 9, only the mark 401 and the mark 402 are displayed, and the mark 403 is not displayed. That is, in the state of FIG. 9, it means that the user cannot select a gesture as a start trigger.

  Thus, by presenting the information of the trigger detection unit that is operating effectively to the user, the user can select the start trigger to use without hesitation.

The method of presentation to the user is not limited to the above-described method, and the same number of LEDs as the trigger detection unit are attached to the television receiver so that the LED corresponding to the trigger detection unit that is operating effectively is lit. May be.

(Modification 3)
The command utterance in the present embodiment includes not only an isolated word utterance such as “8 channels” but also a natural sentence utterance such as “search for a sports program”.

Further, even when the voice recognition apparatus is installed on the cloud server and the television receiver is connected to the voice recognition apparatus via the network, the voice recognition apparatus can execute the same processing as in the present embodiment.

Moreover, although the trigger detection part in this embodiment is three types, the voice trigger detection part 101, the gesture trigger detection part 102, and the applause trigger detection part 103, a trigger detection part is not restricted to this. A trigger detection unit that detects other types of start triggers may be used.

In this embodiment, the trigger detection unit suitable for the usage environment is selected and the operation of the trigger detection unit is enabled. However, a plurality of trigger detection units are always operated and selected by the trigger selection unit 104. The command utterance recognition process may be started when the trigger detection unit detects a start trigger.

  Some or all of the functions in the present embodiment described above can be realized by software processing.

Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 101 Voice trigger detection part 102 Gesture trigger detection part 103 Applause trigger detection part 104 Trigger selection part 105 Recognition part 201 Control part 202 Storage part 203 External storage part 204 Operation part 205 Communication part 206 Bus 208 Microphone 209 Camera 210 Volume sensor 211 Distance sensor 212 Light quantity sensor 400 Television receiver display 401 Mark indicating that the voice trigger is valid 402 Mark indicating that the applause trigger is valid 403 Mark indicating that the gesture trigger is valid

Claims (8)

A plurality of trigger detection means for detecting a start trigger for instructing start of recognition of a command for operating an operation target;
A recognition unit that starts recognition processing of the command when at least one of the plurality of trigger detection units detects the start trigger based on signals from a plurality of sensors;
A recognition device comprising: Voice trigger detection in which at least one of the plurality of sensors is a volume sensor for measuring a surrounding volume, and at least one of the plurality of trigger detection means detects a start trigger by a user's specific keyword utterance. In the case of means,
The recognition apparatus according to claim 1, wherein the voice trigger detection unit operates when the volume is equal to or lower than a predetermined threshold. At least one of the plurality of sensors is a light amount sensor that measures the amount of ambient light, and at least one of the plurality of trigger detection units detects a start trigger by a user's specific gesture. In the case
The recognition apparatus according to claim 1, wherein the gesture trigger detection unit operates when the light amount exceeds a predetermined threshold. At least one of the plurality of sensors is a distance sensor that measures a distance to a user, and at least one of the plurality of trigger detection means detects a start trigger by a user's specific gesture. In the case of means,
The recognition apparatus according to claim 1, wherein the gesture trigger detection unit operates when the distance is equal to or less than a predetermined threshold. At least one of the plurality of sensors is a distance sensor that measures a distance to the user, and at least one of the plurality of trigger detection means detects a start trigger due to a specific keyword utterance of the user. In the case of detecting means,
The recognition apparatus according to claim 1, wherein the voice trigger detection unit operates when the distance is equal to or less than a predetermined threshold. The recognition apparatus according to claim 1, further comprising: a trigger selection unit that selects at least one of the trigger detection units based on a control signal other than a signal from the sensor. A plurality of trigger detection steps for detecting a start trigger for instructing start of recognition of a command for operating an operation target;
A recognition step of starting recognition processing of the command when at least one of the plurality of trigger detection steps detects the start trigger based on signals from a plurality of sensors;
A recognition method comprising: In the recognition device,
A plurality of trigger detection steps for detecting a start trigger for instructing start of recognition of a command for operating an operation target;
A recognition step of starting recognition processing of the command when at least one of the plurality of trigger detection steps detects the start trigger based on signals from a plurality of sensors;
A recognition program for realizing

Images