KR102044526B1 - Method of increasing speech recognition based on artificial intelligence and device of implementing thereof - Google Patents

Abstract

The present invention relates to a method for improving speech recognition based on artificial intelligence and an apparatus for implementing the same. The apparatus for improving speech recognition according to an embodiment of the present invention comprises a voice input unit for receiving a user's voice and uttering a voice. A central control unit for analyzing a verification result of a user confirmation unit, an input voice, and a user confirmation unit confirming a user, and generating a start word determination result instructing the follow-up of a command for controlling the function of a voice device or a peer device adjacent to the device. And a device function controller for controlling the function of the device based on the control of the central controller.

Description

METHOD OF INCREASING SPEECH RECOGNITION BASED ON ARTIFICIAL INTELLIGENCE AND DEVICE OF IMPLEMENTING THEREOF}

The present invention relates to a method for improving speech recognition and an apparatus for implementing the same.

Recently, in controlling home appliances, home appliances, and the like, a variety of technologies for controlling voice based on them have been proposed. In particular, in the control of devices such as home appliances and home appliances, research has been continuously conducted to variously install microphones or newly develop a voice recognition processing module to increase the voice recognition rate in order to recognize human voices quickly.

Meanwhile, since speech recognition must be performed in a situation where various variables occur, it is necessary to operate the device correspondingly to increase the accuracy of speech recognition based on these variables. Thus, the present specification proposes a method for precisely performing speech recognition by devices disposed in a predetermined office space of a house or a building.

In this specification, to solve the above-described problem, it is intended to provide a method and apparatus for adjusting a noise or uncertain voice recognition state of the surroundings in changing the mode of the device to receive a command.

In the present specification, to improve the accuracy of starting word recognition for entering a command input mode, a device and a method of providing a device or an adjacent device to check the presence of a person are provided.

In the present specification, a method and apparatus for maintaining the recognition rate of a general command by temporarily controlling the sensitivity of speech recognition during the entry process of the command input mode and increasing the accuracy of the speech recognition, restoring the sensitivity of the speech recognition again after the speech recognition. To provide.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention, which are not mentioned above, can be understood by the following description, and more clearly by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an embodiment of the present invention, an apparatus for improving speech recognition includes analyzing a verification result of a voice input unit receiving a user's voice, a user confirmation unit confirming a user who uttered a voice, an input voice, and a user confirmation unit. A central control unit for generating a start word determination result instructing subsequent to the command for controlling a function of the device or a peer device adjacent to the device, and a device function control unit for controlling the function of the device based on the control of the central controller.

According to another aspect of the present invention, there is provided a method for improving speech recognition, comprising: receiving a voice of a user by a voice input unit, and checking a user who has spoken a voice; And generating, by analyzing the input voice and the confirmation result of the user verification unit, the central controller to generate a start word determination result instructing the voice to follow a command for controlling a function of the device or a peer device adjacent to the device.

In the case of applying the present invention, if the input start word does not match the original stored voice model, but is close to the sensitivity parameter, the device may be entered into the start word verification mode to increase the recognition rate of a subsequent start word.

In addition, when applying the present invention it is possible to increase the recognition rate of the starting word by controlling the microphone and the like to reduce the noise or increase the recognition rate of the starting word in the device verification mode.

In addition, according to the present invention, when a person is in a state where it is unclear whether the recognized voice is a starting word, the sensitivity parameter is changed high for a specific time and the device noise is reduced, and if the starting word is uttered again, the previous level Even if they have similarity of, the second one can be accepted as a maneuver.

The effects of the present invention are not limited to the above effects, and those skilled in the art can easily derive various effects of the present invention from the configuration of the present invention.

1 is a view showing the configuration of an apparatus according to an embodiment of the present invention.
2 is a diagram illustrating an operation process of a device when a voice of a user is input according to an embodiment of the present invention.
3 is a diagram illustrating a mode that an apparatus according to an embodiment of the present invention may have.
4 and 5 are views illustrating an operation process of a device including a PIR sensor by a user confirmation unit according to an embodiment of the present invention.
6 and 7 illustrate an operation process of a device including a camera sensor by a user confirmation unit according to another embodiment of the present invention.
8 is a diagram illustrating a process of recognizing a start word in cooperation with peer devices according to an embodiment of the present invention.
FIG. 9 is a diagram illustrating a process of reducing noise or performing a task for inputting a start word, even in a peripheral device of a device that has entered a start word verification mode according to an embodiment of the present invention.
10 is a diagram illustrating a process of performing voice recognition by a plurality of devices according to another embodiment of the present invention.
11 is a diagram illustrating a process of controlling other adjacent devices to enter a command input mode by performing only voice recognition according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention.

In order to clearly describe the present invention, parts irrelevant to the description are omitted, and like reference numerals designate like elements throughout the specification. In addition, some embodiments of the invention will be described in detail with reference to exemplary drawings. In adding reference numerals to components of each drawing, the same components may have the same reference numerals as much as possible even though they are shown in different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description may be omitted.

In describing the components of the present invention, terms such as first, second, A, B, (a), and (b) can be used. These terms are only to distinguish the components from other components, and the terms are not limited in nature, order, order, or number of the components. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to that other component, but between components It will be understood that the elements may be "interposed" or each component may be "connected", "coupled" or "connected" through other components.

In addition, in the implementation of the present invention may be described by subdividing the components for convenience of description, these components may be implemented in one device or module, or one component is a plurality of devices or modules It can also be implemented separately.

In the present specification, a home appliance, a home appliance, and the like, which are disposed in a specific space and perform a predetermined function, are collectively called an apparatus. Among the devices, devices that perform voice recognition are referred to as voice recognition devices. In addition, two or more devices disposed in a specific space may transmit and receive a control message using a communication function.

In the present specification, a message is required by the user when the user speaks the voice to classify the message into two types to increase the voice recognition rate. The language spoken by the user is divided into three types: maneuver, command, and dummy. A startup word performs a function that tells the device that a command follows. For example, the device's categorical name ("TV", "radio", "fridge"), or the device's brand ("Wissen", "Trom"), or words of admiration or dialogue ("Hey", "here") Can be

The command indicates the operation of the device and may be variously configured according to the classification of the device. In one embodiment, the instructions may direct to control the on / off of the device or to perform a particular function configured in the device. This may be configured in various ways for each device.

Hereinafter, a description will be given of the corresponding configuration and method by dividing a start word and a command which is arranged in a device and spoken by a user's voice.

1 is a view showing the configuration of an apparatus according to an embodiment of the present invention. The central controller 150 controlling the apparatus 100 controls various components of the apparatus 100. The components constituting the device 100 include the voice input unit 110, the user identification unit 120, the device function control unit 130, the start language database unit 160, the command database unit 170, and the communication unit 180. ) Is presented. The central controller 150 may recognize the starting word or accept the recognized starting word and control the operation of the device.

Recognition of the start word means that the central control unit 150 controls the apparatus 100 to enter the start word verification mode based on the start word determination result with respect to the input voice (first voice), and then the voice input unit 110. Receives a voice (second voice) including a startup word or a command, stores the input voice (first voice) in the startup word database unit 160 before entering the startup verification mode and stores the same first voice later. Allow this to be recognized as a starter. This will be described in detail with reference to S60a and S60b of FIGS. 5 and 7.

1, the components for providing the device 100 with a specific function are not shown. For example, when the device 100 is a TV, it may have a display panel or a power supply separately. If the device 100 is an air conditioner, it may have separate components that provide an air conditioning function. If the device 100 is a washing machine, it may have separate components that provide a washing function. If the device 100 is a refrigerator, it may have separate components that provide refrigeration and freezing functions.

The voice input unit 110 is a module that receives a voice of a user, and uses a microphone as an embodiment. In particular, the voice input unit 110 may be disposed integrally with the device 100 or may be disposed outside the device to convert the input voice into a file and provide the same to the device 100. The voice input unit 110 receives voice data from the microphone and transmits the voice data to the central controller 150 so that the central controller 150 can recognize the voice.

The user confirmation unit 120 confirms the user who spoke the voice. Identifying the user includes confirming that the user is within the space in which the device is currently located. In one embodiment, the user identification unit 120 may be a sensor for sensing whether a person is located around the device. The type of sensor may be a camera sensor photographing the surroundings of the device, a passive infrared sensor (PIR) sensor that detects whether a person exists, or a motion detection sensor that detects a person's movement. The user identification unit 120 of the present invention is not limited to a specific sensor.

In addition, other than sensing, the user identification unit 120 may confirm that the user exists in the space. In one embodiment, the user identification unit 120 confirms the controlled time of the peer device, which is the device 100 or another device arranged around the device, in the space where the device 100 is disposed. When the device or the peer device is controlled within a preset time, the user identification unit 120 may provide the central controller 150 with information that the user has been confirmed. As a result, the central controller 150 may determine that the user is confirmed around the apparatus 100 and determine the voice input by the voice input unit 110.

The device function control unit 130 controls the function of the device. It may be controlled by the central controller 150. The function of the device may vary depending on the type of device. The device function control unit 130 receives a signal for controlling the device by the user using the remote control, or the control interface unit 135 disposed on the device in the form of a predetermined button to provide an interface for the user to control the function. It includes more. The control interface 135 may further include a speaker that outputs a predetermined voice or sound to the outside.

The startup word database unit 160 stores data necessary for identifying and comparing whether or not the voice input unit 110 has received the startup word. In addition, the startup word database unit 160 may store data such that different voices of the same startup word may be startup words according to characteristics of the voice of the user who spoke the startup word. The command database unit 170 stores data necessary for checking and comparing whether the voice input unit 110 receives a command input from the voice input unit 110.

The startup word database unit 160 or the command database unit 170 may store voice file data of a preset startup word or command or data necessary for checking the voice file, and the stored data may be updated or added. In addition, the start word database unit 160 may store start words of other devices besides the corresponding device. This is illustrated in FIG. 10.

Although not shown in the drawing, the central controller 150 may include a separate storage space such as a memory card or a memory chip to increase the operation speed or temporarily store the input voice file for a predetermined period of time.

In summary, it is as follows. The central controller 150 analyzes the voice input from the voice input unit 110 and the result of the user confirming unit 120 confirming the existence of the user, and thus the input voice of the device 100 or a peer device adjacent to the device 100. Produces a start word determination result instructing subsequent commands to control a function. The start word determination result is a result of determining how the input voice corresponds to the start word.

The startup word instructs the follow-up of the command to switch the device 100 to the command input mode (mode switching or mode entry). According to the result of the startup word determination, the device 100 enters the command input mode, the startup word verification mode, or the normal mode. It can be either. The voice data input from the voice input unit 110 by the central controller 150 may be compared with the acoustic model stored in the start language database 160 to determine whether the input voice data corresponds to the start language.

When the device 100 switches (enters) to the command input mode, the device 100 may output through the built-in speaker or the like to flash the LED light that the user is ready to input a command. For example, after confirming the start word, the device 100 outputs a voice of "Please speak" to allow the user to confirm that the user is converted to a mode in which the command can be received.

In addition, in the normal mode by flashing the LED light is turned off, but flashes in the command input mode so that the user can confirm that the current state of the device 100 can receive a command. Further, according to another embodiment of the present invention, the user is asked "yes?" Even in the start word verification mode. As shown in FIG. 5, a voice may be output to request the user to input the starting word once more.

The start word determination result may be generated numerically. Compared with the data stored in the starting word database unit 160, the numerical value of the matching with the starting word data stored therein is numerically generated, and the result of the determination of the starting word in the case of a predetermined criterion (for example, 80% or 70%) or more. Can be calculated by starting word identification.

On the other hand, when compared with the data stored in the start word database unit 160, the numerical value matching the start word data stored therein is low (for example, 40% or less than 50%), the result of the start word determination is a start word inconsistency It can be calculated as

However, in the present specification, the central controller 150 controls the device to enter the starting word verification mode by determining the starting word ambiguity regarding the result between the starting word checking and the starting word inconsistency.

The start-up word verification mode means that when the inputted voice is not confirmed as the start-up word, but the device has some similarity, the device switches to a state in which the start-up language can be better input. That is, when the central control unit 150 generates the result of the determination of the starting word with the starting word ambiguity, the voice recognition parameter of the voice input diagram is controlled within a preset time to increase the sensitivity of the speech recognition or reduce the noise generated by the device. In one embodiment, switching to a mode such as reducing noise generated by neighboring peer devices.

If the control unit 150 determines that the user is similar to the starting word but is not the starting word based on the preset sensitivity parameter, the user identification unit 120 confirms the existence of the user and temporarily raises the sensitivity parameter when the user is confirmed. The recognition rate of the starting word can be increased by reducing the In this case, the sensitivity parameter is temporarily increased. This temporarily raises the sensitivity parameter for recognition of a starting word, and when receiving a command later, restores the sensitivity parameter so that the command can be correctly input.

For example, if the starting word is "Let's start" while the input voice is "Shizakaza", if the sensitivity parameter is increased to recognize the starting word that is uttered again, it increases the recognition rate of the starting word and the "Shizakaza" voice is recognized as the starting word. Can be.

However, in the case of the command, since the device 100 correctly instructs a specific operation, in order to receive the command after the start word is input, the sensitivity parameter may be restored.

In addition, when the central controller 150 confirms that the voice of "Shizakaza" is continuously recognized as the starting language by increasing the sensitivity parameter in the state where only the voice model of "Let's start" is stored in the starting language database unit 160 ( For example, when a command is input after recognizing "Shizakaza" as a startup language, the central control unit 150 recognizes the voice model "Shizakaza" as a new startup language and stores it in the startup language database unit 160. do. Let's take a closer look at this.

2 is a diagram illustrating an operation process of a device when a voice of a user is input according to an embodiment of the present invention. Figure 2 is composed of two flows, so that the speech recognition can be performed for a period of time suitable for the start word verification mode of verifying the start word input by adding a timer.

You can start in normal mode with no startup words or commands. First, a user voice is input to the voice input unit 110 (S1). In response to the input voice, the central controller 150 recognizes whether the input voice is a starting word (S2). Whether the user is a starting word may be determined according to the similarity by measuring the similarity by comparing the input voice with information stored in the starting word database unit 160 (such as a voice file or a word file for comparison).

When the degree of similarity is determined as a starting word, that is, when a speech comparison result derived by comparing the starting word and information stored in the starting word database unit 160 satisfies a parameter indicating a preset voice matching state (S3). The central control unit 150 confirms that the starting word is spoken and proceeds to the command input mode.

When the command input mode is first started in the normal mode, since the timer is not executed (S4), the components of the apparatus 100 perform the command recognition and terminate (S6). Terminating means that the device 100 enters a normal mode for inputting a new starting word or command. If the timer is in the running state (S4), the timer is released, the sensitivity is restored, and the operation is restored to the previous state (S5).

On the other hand, if the parameter is not satisfied at S3 but the voice comparison result derived from the result is close to or similar to the parameter indicating the preset voice matching state (S8), it may be determined whether to enter the starting-word verification mode. have. To this end, the user confirmation unit 120 checks whether a user exists (S9). If it is determined that the user exists, the central control unit 150 enters the start word verification mode and sets or resets the timer (S10).

This controls to increase the voice recognition sensitivity of the voice input unit 110 for a predetermined period (time set in the timer) to verify the start word, and to reduce the noise of the device or the surrounding devices (S20). To this end, the communicator 180 of the device 100 may transmit a message instructing peripheral devices to reduce noise.

In step S20, when the start word determination result determined by the central controller 150 is a ambiguity of the start word (start word ambiguity), the central controller 150 enters the start word verification mode based on the start word determination result. Starting word ambiguity means that the input voice is not accepted as the starting word, but the similarity is higher than a preset criterion.

For example, if the waveform of the voice coincides with 80% or more when it is recognized as the starting word, it may be determined that the starting word ambiguity is 60% to 79%. Alternatively, when the number of letters recognized as the same as the number N of the starting word is K or more, it may be confirmed as an ambiguous state and may enter the starting word verification mode.

 The central controller 150 controls the voice recognition parameter of the voice input unit 110 within a preset time to increase the sensitivity of the voice recognition. In this process, the voice input unit 110 may sense and receive a smaller sound than in the normal mode.

In addition, in S20, when the start word determination result determined by the central controller 150 is start word ambiguity, the central controller 150 enters the start word verification mode. That is, the communicator 180 which transmits and receives a message with the surrounding peer devices may transmit a message to the peer device indicating to reduce the noise generated by the peer device. According to an embodiment of the present invention, the air conditioner transmits a message so as to reduce the sound of a TV, which is a peer device disposed adjacent to each other, in the process of confirming the starting word.

After S20, the process returns to step S1 to wait for a user's voice input. In this process, the timer set in advance is continuously operating. If the timer expires (S12), it is determined that no more start words are entered, and the timer is released to restore normal sensitivity from the start word verification mode, the sensitivity is restored, and the device and other devices are stopped for noise reduction. The state that was in the mute state can be stopped and restored to the previous operation (S13).

 When the voice is input again from the user after S20 (S1), the central controller 150 may recognize the starting word more accurately in the state where the sensitivity is increased and the device noise is reduced (S2), and thus may operate accordingly.

If the start word is not input again in the voice input mode or a separate voice is not input until the time set as the timer elapses, the central controller 150 may switch the device 100 to the normal mode with the end of the timer. have.

In summary, the normal mode refers to a mode of the apparatus 100 capable of receiving a startup word. In this process, if the start word is sufficiently recognized, the state of the device 100 enters the command input mode. However, if the start word is incompletely recognized, the device 100 enters the start word verification mode and is fixed for the start word recognition. The noise of the device or peripheral device can be shut off for input of the starting word for a period of time.

3 is a diagram illustrating a mode that an apparatus according to an embodiment of the present invention may have. From the perspective of speech recognition, the device 100 has three modes. Normal mode (STATE_N) to wait for voice input because no separate voice is recognized, Command input mode (STATE_C) to receive a command by inputting a starting word, and waiting for the starting word to be input again because it is not clearly identified as a starting word. One embodiment includes the start word verification mode (STATE_R).

When the voice is input in the normal mode (STATE_N) (S31), it is checked whether it is a starting word, and according to the result (starting language judgment result), the process enters the command input mode (STATE_C) (S32), and the starting word verification mode (STATE_R). It is determined that the process of entering (S33) or the process is not a starting word and returns to the normal mode (STATE_N) (S34).

When the start word is determined as the start word, the central control unit 150 may switch the device to the command input mode and receive a command. In this process, the device 100 may notify the user of the command input mode. You can output the voice say "".

When the command is input in the command input mode (STATE_C), the command enters the normal mode (STATE_N) after executing the command. If the command is not input within a predetermined time, the command enters the normal mode (STATE_N) again (S35).

The start word verification mode (STATE_R) also enters the command input mode (STATE_C) when the start word is input again (S36), and if the command is not input within a predetermined time or the voice input afterward is not the start word, the normal mode (STATE_N) again. Enter (S37). While entering the start word verification mode (STATE_R), the device 100 may increase the sensitivity of the voice input unit 110 and reduce the noise to increase the start word input possibility.

In the start word verification mode (STATE_R), when the central control unit 150 determines that the starting word is a starting word ambiguity, the central control unit 150 controls the device function control unit 130 to reduce noise generated by the device. Can be controlled. In addition, peripheral devices can also be controlled to reduce noise.

2 and 3 summarized as follows. When the user inputs a voice, the voice input unit 110 provides the voice input to the central controller 150. The central controller 150 measures the similarity with the acoustic model stored in the starter database 160 and checks the starter if the set sensitivity parameter is satisfied, and enters the command input mode.

On the other hand, although the object is rejected (rejected), but the range of similarity is within a preset range, or the user is around, the user confirmation unit 120 can determine whether the user exists. Alternatively, user confirmation may be performed with a range of similarities.

If the user confirms that the user is in the vicinity of the device, for example, because the camera, PIR sensor, or peripherals are controlled by the user, the device or peripheral may be set for a specific time and the sensitivity parameter may be set Reduces noise. If the voice data comes back to the voice input unit 110 before the timer expires, the similar words are better recognized as the starting words due to the increased sensitivity and reduced noise. Change to the state. In addition, even when the timer expires, the sensitivity and operation of the device are restored to prevent misunderstanding of commands or starting words due to excessive sensitivity, and the devices and peripheral devices that were in the pause or noise canceling mode while operating were continuously operated to reduce noise. To control.

4 and 5 are views illustrating an operation process of a device including a PIR sensor by a user confirmation unit according to an embodiment of the present invention. 100A of FIG. 4 uses a washing machine as an embodiment. The user identification unit 120a includes a PIR sensor.

When configured as shown in FIG. 4, the washing machine 100a may operate in response to the starting word based on the process as shown in FIG. 5. The configuration of the washing machine 100a may include the components described with reference to FIG. 1.

As in S41a of FIG. 5, the user utters "LG trom" as a starting word for operating the washing machine 100a. The central controller 150 of the washing machine 100a recognizes whether the voice data input through the voice input unit 110 such as a microphone is a starting word and measures similarity (S42). As a result of measuring the similarity, the speech is rejected at the level of the sensitivity parameter set in the speech recognition but may be measured to be close to the starting word (S43a).

For example, it may be identified as an "lg drop" by ambient noise or by a user's language habit or pronunciation. Alternatively, when it is confirmed that only three letters among four letters constituting the starting word match as compared with the voice model stored in the starting word database unit 160, the central controller 150 accepts the starting word. ) Or before rejecting, it may be checked whether the user is present in the vicinity.

That is, the user confirmation unit 120a, that is, the PIR sensor checks the surroundings to confirm that the person is in the vicinity (S44a). If there is a person around the washing machine 100a (S45), the central control unit 150 resets the timer for a predetermined time (S46), changes the sensitivity parameter to high, and reduces the noise of the washing machine 100a. (S47a).

Changing the sensitivity parameter higher allows you to receive a smaller sound. Although the sensitivity parameter set in the washing machine 100a is appropriately set for the device, the sensitivity parameter is temporarily changed to a high value so as to receive an additional input of the starting word so that it can be well recognized when the starting word is input within a short time.

If the sensitivity parameter is changed to a high value, the false recognition rate may also be increased. In this specification, the timer is reset (S46) so that the sensitivity parameter is kept high only within a predetermined time. If it is determined in S45 that there is no person, the mobile station enters a normal mode (S53) and waits for a new voice to be recognized (S54).

In addition, when the washing machine 100a is in operation, the central controller 150 controls the device function controller 130 to allow the washing machine 100a to temporarily stop. Of course, the central controller 150 may omit the step of reducing the noise when the washing machine 100a is not operating. Alternatively, the central controller 150 may omit the step of reducing noise even when the washing machine 100a is performing an operation in which noise is not generated.

This means that the washing machine 100a enters the start word verification mode. In this process, the washing machine 100a asks the user to re-enter the mobile word. "Please say again" or "Yes?" You can also output a voice file called. If the person is not confirmed in S44a, it is determined that no starting word is input and the vehicle enters the normal mode.

Afterwards, the user again fires a word that can be judged as "lg trom" or similar (S48a). This includes the step of re-igniting the start word by the user since the user has uttered the start word but has not been informed that the device has been switched to the mode for receiving the command. At this time, even if the user's reproduced voice data is measured at a level similar to that of the first voice data, the probability of being accepted as a starting word increases because it is compared with the temporarily increased sensitivity.

That is, since the sensitivity is increased, the input voice is more likely to be recognized as a starting language than S42, and thus is recognized as a starting language (S49). As a result of recognizing the maneuver, the central controller 150 terminates the timer, restores the sensitivity to the previous sensitivity (S50), and enters the command input mode in which the command can be input (S51).

The central control unit 150 may output a voice file “tell me” to the speaker to inform the user of the entry into the command input mode. In order to increase the recognition rate of the command, the operation of the device may be restored after the command is input (S52a). Of course, it is possible to restore the operation of the device while simultaneously restoring to sensitivity.

In one embodiment when the step S52a is performed before the step S51, the operation of the washing machine 100a having reduced the noise may be performed before the step S52a in the case where the operation of the noise reduction operation is not significant (for example, an operation or a called operation). Can be. On the other hand, the central controller 150 may control to restore the operation after the command is input after the command input mode for an operation such as dehydration with a high noise.

In addition, if the starting word is no longer input after S47a, the user may enter the normal mode after a certain time (when the timer ends).

Meanwhile, after S52a, the start word input in S41a is stored in the start word database unit 160 so that the start word may be recognized even if it is later recognized as an “lG drop”, thereby increasing user convenience. Alternatively, instead of storing the start word directly as a start word of the start word database unit 160, the start word may be stored as a start word candidate and stored in the start word database unit 160 when the start word candidate is continuously recognized. That is, the central controller 150 enters the starting word verification mode based on the starting word determination result for the first voice input from S41a or S41b, and then the second voice input by the voice input unit (see S48a and S48b). When it is confirmed by the start word or command, the first voice is stored in the start word database unit 160. That is, the central controller 150 stores the first voice input before starting the start word verification mode in the start word database unit 160 to recognize the start word if the same voice is input later. .

4 and 5 illustrate the washing machine 100a as an embodiment, but may be applied to various home appliances. In addition, as an embodiment of the user identification unit, a human body (motion) sensor is presented to suggest a threshold control situation in the input of the maneuver, but other sensors may be used.

6 and 7 illustrate an operation process of a device including a camera sensor by a user confirmation unit according to another embodiment of the present invention. 100b of FIG. 6 illustrates an example of an air conditioner. The user identification unit 120b includes a camera sensor.

In the case of FIG. 6, the air conditioner 100b may operate in response to the starting word based on the process of FIG. 7. The configuration of the air conditioner 100b may include the components described with reference to FIG. 1. Since FIG. 7 is similar to the process of FIG. 5, the differences are mainly described. Parts different from FIG. 5 are S41b, S43b, S44b, S47b, S52b, S60b, and the like.

As in S41b of FIG. 7, the user utters "LG whissen" as a starter for operating the acorn 100b. The central controller 150 of the Aiken 100b recognizes whether the voice data input through the voice input unit 110 such as a microphone is a starting word and measures similarity (S42). As a result of measuring the similarity, the speech was rejected at the level of the sensitivity parameter set in the speech recognition but may be measured to be close to the starting word (S43b).

For example, it may be identified as an "lg hissen" by ambient noise or by a user's language habit or pronunciation. Alternatively, when it is confirmed that only three letters among four letters constituting the starting word match as compared with the voice model stored in the starting word database unit 160, the central controller 150 accepts the starting word. ) Or before rejecting, it may be checked whether the user is present in the vicinity.

That is, the user confirmation unit 120b, that is, the camera sensor checks the surroundings and confirms that the person is in the vicinity (S44b). If there is a person around the air conditioner 100b (S45), the central controller 150 resets the timer for a predetermined time (S46), changes the sensitivity parameter to high, and reduces the noise of the air conditioner 100b. (S47).

Changing the sensitivity parameter higher allows you to receive a smaller sound. Although the sensitivity parameter set in the air conditioner 100b is appropriately set for the device, the sensitivity parameter is temporarily changed to a high value in order to receive an additional input of the starting word, so that it can be well recognized when the starting word is input within a short time. In addition, the position of the user was confirmed using a camera sensor, and the central controller 150 may adjust the microphone of the voice input unit 110 to face the position of the user. On the other hand, if it is determined that there is no person in S45, the process proceeds to S53 and S54.

In addition, when the air conditioner 100b is in operation, the central controller 150 controls the apparatus function control unit 130 to reduce the air volume of the air conditioner 100b or switch to indirect wind. In one embodiment, the central controller 150 controls the air volume of the device 100b to receive a better input of the starting word. Of course, the central control unit 150 may omit the step of reducing the noise when the air conditioner (100b) is not in operation. Alternatively, the central controller 150 may omit the step of reducing the noise even when the air conditioner 100b is performing an operation in which noise is not generated.

This means that the Aiken 100b enters the start word verification mode and proceeds to steps S48b to S52b as shown in FIG. 5. In this process, Aiken 100b asks the user to re-enter the activation word, "Please say again" or "Yes?" It can also output a voice file, and if the person is not confirmed in S44, it is determined that the starting word is not input and enters the normal mode.

Afterwards, the user again fires a word that can be judged as "LG Whissen" or similar (S48b). This includes the step of re-igniting the start word by the user since the user has uttered the start word but has not been informed that the device has been switched to the mode for receiving the command. At this time, even if the user's speech data is measured at a level similar to that of the first voice data, the probability of being accepted as a starting word increases because the sensitivity is temporarily increased.

That is, since the sensitivity is increased, the input voice is more likely to be recognized as a starting language than S42. Accordingly, the input voice is recognized as a starting language (S49) and steps S50 to S52b are performed. It includes restoring the air volume previously reduced in S52b. By reducing the air volume of the air conditioner 100b, the signal-to-noise ratio (SNR) is relatively better, so the air conditioner 100b recognizes the starting word well.

Meanwhile, after S52b, the start word input in S41b is stored in the start word database unit 160 so that the start word may be recognized even if it is later recognized as an "lg hissen", thereby increasing user convenience. Alternatively, instead of storing the start word directly as a start word of the start word database unit 160, the start word may be stored as a start word candidate and stored in the start word database unit 160 when the start word candidate is continuously recognized. For example, when "LG Hissen" is input more than a predetermined number of times (for example, five times), it can be stored as a starting language.

In FIG. 4 to FIG. 7, the control of the device noise may be performed by operating in a soft, slow or low noise mode or pausing for a specific time when the device is in operation. Various devices may perform noise reduction according to the characteristics of each device. For example, in the case of an air conditioner, the voice recognition rate may be increased by weakening the wind or changing the direction of the wind toward the user. In the case of a washing machine can be paused or run slowly. In the case of a vacuum cleaner can weaken the suction power. In particular, the robot cleaner can stop the movement or reduce the movement speed. When controlling devices such as smart hubs, the volume can be reduced.

Further, temporarily increasing the sensitivity parameter to verify the start word may increase the recognition rate as well as the recognition rate, so that the start word verification mode may be maintained only within a short time based on a timer, thereby lowering the recognition rate.

Meanwhile, S60a and S60b of FIGS. 5 and 7 include a process of newly adding a start word. In other words, the voice input in S41a and S41b does not match the starting word. However, based on the result of the determination of the starting word, the central controller 150 enters the starting word verification mode, and when the voice input unit subsequently inputs the voice (voice received in the starting language verification mode) is a starting word such as S48a and S48b. In addition, the central controller 150 may store the voice (that is, the voice input from S41a and S41b) causing the entry into the start word verification mode in the starter database unit 160.

According to another embodiment of the present invention, when a command is input in the start word verification mode, the user determines that the start word has already been uttered and corresponds to the case where the command is uttered. Therefore, even if a command is input in the start word verification mode, the central controller 150 may store the voice (that is, the voice input from S41a and S41b) causing the entry into the start word verification mode in the start language database unit 160. .

Although not shown in the drawings, when the user controls the apparatuses 100a and 100b in the start word verification mode, that is, when the function of the apparatus is controlled externally, the user's intention was to control the apparatus. The central controller 150 may store a voice (ie, a voice input from S41a and S41b) that causes the entry into the start word verification mode in the starter database unit 160.

8 is a diagram illustrating a process of recognizing a start word in cooperation with peer devices according to an embodiment of the present invention. According to an embodiment of the present invention, while the device enters the start-up verification mode, other adjacent peer devices may be used to check whether the user is present and to temporarily reduce the noise of the other peer devices. You can send a message. Look in more detail.

In the device 100 for performing voice recognition, peer devices 100p and 100q are disposed in the vicinity. The voice input unit 110 of the apparatus 100 receives a voice (S61), and as a result of recognizing the voice, it is confirmed as a starting word ambiguity state (S62). As described above, although it is not matched to the maneuvering word, rejecting includes a case where the matching rate is higher than a predetermined criterion, and includes a case corresponding to some of the maneuver words or confirmed by similar pronunciation.

In the case of the start word ambiguity state, surrounding users may be checked as in the method of FIGS. 4 to 7. In addition, as shown in FIG. 8, the corresponding device 100 and the surrounding peer devices 100p and 100q may check whether the user is present and whether the user exists by checking the history controlled by the user. That is, if the device 100 and the surrounding peer devices 100p and 100q have been controlled by the user within a short time (for example, 1 minute or 3 minutes, etc.), the user may still be around. , Check this. The device 100 checks the history controlled by the user in the corresponding device (S63), and transmits a user confirmation request message to adjacent peer devices 100p and 100q (S64p and S64q).

The first peer device 100p checks whether the user is in the vicinity by using the camera sensor or the PIR sensor (S65p). When the first peer device 100p detects the user as a result of the check, the detected result is transmitted as a user check result message (S66p). Of course, if the user is not sensed by the first peer device 100p, the unsensed result is transmitted as a user confirmation result message (S66p).

On the other hand, the second peer device 100q checks the history of the user controlling the second peer device 100q (S65q). For example, when the second peer device 100q is a TV, when the user has a history of controlling the second peer device 100q, such as increasing a TV volume or changing a channel, the second peer device 100q is used. The user confirms that there is a user around, and transmits the confirmed result to the user confirmation result message (S66q).

When the device 100 determines that the user is in the vicinity by combining the results of S63, S66p, and S66q, the apparatus 100 enters the start word verification mode (S67).

8 is as follows. The user identification unit of the device 100 checks the time when the device is controlled or the peer devices 100p and 100q are controlled in the space where the device 100 is disposed, and the preset time (for example, 1 minute or 3 minutes). In the case where the device 100 or the peer devices 100p and 100q are controlled in FIG. 2, the device 100 may confirm that the user is in the vicinity. As a result, the central controller 150 of the apparatus 100 may generate a result of determining the starting word based on the user's confirmation in the space. If there is a user and the start word is similarly input, the device 100 may enter the start word verification mode.

FIG. 9 is a diagram illustrating a process of reducing noise or performing a task for inputting a start word, even in a peripheral device of a device that has entered a start word verification mode according to an embodiment of the present invention.

When the device 100 enters the starter verification mode (S71), the starter verification mode request message is transmitted to the adjacent peer devices 100p and 100q (S72p and S72q). This includes requesting that adjacent peer devices 100p and 100q also reduce noise, stop operation or perform voice input. As a result, the first peer device 100p reduces the noise of the device or stops the operation (S73p). The second peer device 100q increases the sensitivity of the user input unit so that the user can receive the spoken voice well (S73q).

The second peer device 100q transmits the input voice file to the device 100 (S74). In this case, the device 100 may separately receive a starting word and receive a voice input by the adjacent second peer device 100q as shown in S74. Using these, the device 100 confirms the starting word (S75). After confirming that the startup word is uttered, the apparatus 100 enters the command input mode (S76).

In this process, the starter verification mode end message is transmitted to adjacent peer devices 100p and 100q (S77p and S77q). A time interval may be provided between S76 and the time points at which the message is transmitted (S77p and S77q), in order to maintain S73p and S73q for a short time so that the command can be input more accurately in the command input mode. Thereafter, each of the peer devices 100p and 100q may perform a procedure of restoring noise and motion S78p and restoring sensitivity of the user input unit S78q to restore S73p and S73q.

10 is a diagram illustrating a process of performing voice recognition by a plurality of devices according to another embodiment of the present invention.

The first device 100a and the second device 100b can construct a database for each of the starting words. Also, in one embodiment, a starter database for other devices may also be established. In an embodiment, in the case of FIG. 4, the washing machine 100a uses the "LG Trom", which is a starting word for itself, but also starts the "LG whissen", which is a starting word for the air conditioner 100b of FIG. You can save it. When a start word of another device is input, the input start word may be transmitted to the corresponding device.

The first apparatus 100a, for example, the washing machine, is in the normal mode (S81). Similarly, the second device 100b, for example the air conditioner, is also in the normal mode (S82). In this state, the user speaks with the "lg trom" in the vicinity of the second device 100b, which is voice input from the second device 100b (S83). The "lg trom" input by the voice input unit 110 of the second device 100b is analyzed by the central control unit 150 of the second device 100b and the "LG whissen" which is the starting word of the second device 100b. It was confirmed to be different.

However, the central controller 150 of the second device 100b confirms that the start word of the first device 100a stored in the second device 100b matches the input voice. That is, the central controller 150 of the second device 100b checks the voice inputted as the starter of the first device as the voice recognition result (S84) and transmits the input voice file to the first device 100a (S85). . Of course, the start word recognition result can be transmitted in S85. Thereafter, when the first apparatus 100a checks the starting language based on the transmitted voice file (or the starting word recognition result), the first apparatus 100a switches to the command input mode (S87).

8 to 10 illustrate a process of recognizing a start word or entering a start word verification mode in cooperation with adjacent devices. According to an embodiment of the present invention, if the voice data of the user's spoken start word is rejected because it is similar to the start word based on the set voice recognition sensitivity parameter but is not determined to be the same, the corresponding device or adjacent devices (peer devices) By confirming that this user exists, the recognition rate can be increased while preventing false recognition. The user identification method may use a camera sensor, a PIR sensor, or may use a history in which the user controls the devices.

When the voice data similar to the starting word is input to the device, after confirming the user, if there is a person, the device may change the voice recognition sensitivity parameter to a high level so that the similar word may be recognized as the starting word. In addition, the device and the surrounding peer devices reduce the noise of the device (slow, weak, low-noise mode operation, pause, etc.) to secure the SNR to increase the speech recognition rate when the start-up word re-ignition.

If there is no user detected as a result of checking the user, if the user does not detect the starting word even though there is a user and tries to increase the recognition rate for a certain time, the user is restored to the existing value and operation. Does not increase misperception.

FIG. 11 is a diagram illustrating a process of controlling other adjacent devices to enter a command input mode by performing only voice recognition according to an embodiment of the present invention.

11 illustrates that the hub device 300 serves as a kind of command recognition hub. The hub device 300 may not optionally include the device function control unit 130 and the control interface unit 135 in the configuration of FIG. 1. In addition, the command database unit 170 may not be included.

The hub device 300 receives the voice and determines that the start word is ambiguous (starting word ambiguity state) as a result of the voice recognition (S90). This is a result calculated by comparing both the start word of the first device 100a and the start word of the second device 100b controlled by the hub device 300. Since the start word is ambiguously recognized, the hub device 300, the first device 100a, and the second device 100b all perform user verification (S91). Earlier, we looked at sensing people or checking control history to proceed with user verification.

If it is determined that there is a person, as a result of S91, the hub device 300 enters a startup word verification mode (S92) and transmits a startup word verification mode request message to the first device 100a and the second device 100b (S93a). S93b). This is as described with reference to S72p / S72q of FIG. 9. Thereafter, the hub device 300, the first device 100a, and the second device 100b all enter the start-up verification mode (S94), and when a new voice is input in this state, the hub device 300 has a high sensitivity parameter. In step S95, it is confirmed that the starting word input in step S1 is the starting word of the first apparatus 100a. The hub device 300 then notifies that the startup word verification mode has ended. That is, the hub device 300 transmits a start word verification mode end message to the first device 100a and the second device 100b (S96a and S96b), and each device ends the start word verification mode (S97). The operations of terminating the timer, removing noise, or restoring the sensitivity parameter of the user input unit have been described above with reference to S78p and S78q of FIG. 9.

The hub device 300 transmits a message instructing the first device 100a to switch to the command input mode since the starting word of the first device is confirmed (S98). As a result, the first device may enter the command input mode S99.

Of course, according to another embodiment of the present invention, after receiving a command, the hub device 300 may transmit a message instructing the first device 100a to perform a function corresponding to a specific command. This may be configured in various ways depending on the implementation.

In the case of applying the present invention, if the input start word does not match the original stored voice model but approaches the sensitivity parameter, it is possible to increase the recognition rate of the subsequent start word. In particular, in a noisy environment, the user may not recognize even if the user speaks the starting language loudly and accurately. In contrast, the devices may control the microphone to reduce the noise or increase the recognition rate of the starting language. In addition, to increase its accuracy, it is possible to check whether a person is present in various ways. If a person is nearby, the sensitivity parameter is changed for a certain time and the device noise is reduced so that once the maneuver is fired, it can be accepted a second time, even if it has the previous level of similarity.

Accordingly, as shown in FIG. 3, the same starter word may be rejected in the normal mode but accepted in the starter verification mode. In addition, since the recognition rate of the starting word is increased only within a certain time, it is possible to prevent an increase in the false recognition rate in recognizing another command.

In addition, the operation of the devices can be controlled for a short time to prevent the noise of the device or the surroundings in the start word verification mode. This increases the speech recognition rate and does not affect the operation of the device, thereby increasing the possibility of starting the device. In particular, if the user speaks more than two times, the recognition rate is increased, so that the user can easily control the device by voice. In addition, even if the voice file is not stored in the starter database, the voice data generated by entering the starter verification mode for a certain period of time is stored again as the starter language, so that the starter database can be configured in response to user convenience with various pronunciation tendencies. have.

Although all components constituting the embodiments of the present invention have been described as being combined or operating in combination, the present invention is not necessarily limited to these embodiments, and all of the components are within the scope of the present invention. It can also be combined to operate selectively. In addition, although all of the components may be implemented in one independent hardware, each or all of the components may be selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program includes a storage medium including a magnetic recording medium, an optical recording medium and a semiconductor recording element. In addition, the computer program for implementing an embodiment of the present invention includes a program module transmitted in real time through an external device.

In the above description, the embodiment of the present invention has been described, but various changes and modifications can be made at the level of ordinary skill in the art. Therefore, it will be understood that such changes and modifications are included within the scope of the present invention without departing from the scope of the present invention.

100: device 110: voice input unit
120: user confirmation unit 150: central control unit
160: startup word database unit

Claims (16)

In a device for improving speech recognition,
A voice input unit for receiving a voice of a user;
A user confirmation unit for confirming a user who uttered the voice;
A central controller configured to analyze a confirmation result of the input voice and the user confirmation unit to generate a start word determination result instructing the voice to follow a command for controlling a function of the device or a peer device adjacent to the device; And
A device function control unit for controlling a function of the device based on the control of the central controller;
The central controller enters into any one of a command input mode, a starting word verification mode, or a general mode based on a result of the determination of the starting word, and the central controller sets a timer when entering the starting word verification mode, thereby recognizing the voice input unit. Control parameters to increase the sensitivity of speech recognition to recognize similar words in the starting word
When the timer expires, the central controller restores the sensitivity of the speech recognition.
The normal mode is a standby state because no starting word or command is input,
The sensitivity of the speech recognition in the startup word verification mode is higher than the sensitivity of the speech recognition in the normal mode, so that the startup word rejected in the normal mode is accepted in the startup word verification mode.
The method of claim 1,
And increase the false recognition rate of speech in the start word verification mode.
The method of claim 1,
And the central controller enters a startup word verification mode based on the result of the determination of the starting word, and the central controller controls a device function control unit to reduce noise generated in the device.
The method of claim 1,
Further comprising a communication unit for transmitting and receiving a message between the device and the peer device,
The central controller enters a startup language verification mode based on a result of the determination of the startup language, and the communication unit transmits a message to the peer device instructing to reduce noise generated in the peer device. .
The method of claim 1,
The user identification unit further comprises a sensor for sensing whether a person is located around the device, the apparatus for improving speech recognition.
The method of claim 1,
When the user or the peer device is controlled within a preset time by checking the time the device or the peer device is controlled in the space where the device is disposed
And the central controller generates the starting word determination result based on the user's confirmation in the space.
The method of claim 1,
And the central controller switches the device to a command input mode based on a result of the determination of the starting word.
The method of claim 1,
Further comprising a startup word database unit for storing the startup word,
The central controller enters a starting word verification mode based on a result of determining a starting word for the first voice input in the normal mode, and when the second voice input after the voice input unit is a starting word or a command, the central And a controller is configured to store the first voice input before entering the starting word verification mode in the starting word database unit.
In the apparatus for performing speech recognition,
Receiving a voice of a user by a voice input unit;
Confirming, by a user confirmation unit, the user who spoke the voice; And
Analyzing, by the central controller, the verification result of the input voice and the verification result of the user verification unit, and generating a start word determination result instructing the voice to follow a command for controlling a function of the device or a peer device adjacent to the device; ,
The central controller enters into any one of a command input mode, a starting word verification mode, or a general mode based on a result of the determination of the starting word, and the central controller sets a timer when entering the starting word verification mode, thereby recognizing the voice input unit. Controlling a parameter to increase the sensitivity of the speech recognition, recognizing a similar word of the starting word as the starting word, and when the timer ends, the central controller restores the sensitivity of the speech recognition.
The normal mode is a standby state because no starting word or command is input,
The sensitivity of the speech recognition in the startup word verification mode is higher than the sensitivity of the speech recognition in the normal mode, so that the startup word rejected in the normal mode is accepted in the startup word verification mode.
The method of claim 9,
And a false recognition rate of speech in the start word verification mode is increased.
The method of claim 9,
Controlling the apparatus so that the central control unit enters a starting word verification mode based on a result of the starting word determination; And
And the central controller further controlling a device function control unit controlling a function of a device to reduce noise generated in the device.
The method of claim 9,
The device further includes a communication unit for transmitting and receiving messages between the device and the peer device,
Controlling the apparatus so that the central control unit enters a starting word verification mode based on a result of the starting word determination; And
And the communication unit sends a message to the peer device instructing to reduce noise generated by the peer device.
The method of claim 9,
The user identification unit further comprises the step of sensing the location of the person around the device using a sensor, the method of improving speech recognition.
The method of claim 9,
Confirming, by the user identification unit, a time at which the device or the peer device is controlled in a space where the device is disposed;
Confirming, by the central controller, that the device or the peer device is controlled within a preset time based on the confirmed result; And
And said central controller controlling said device to enter a start word verification mode.
The method of claim 9,
And switching the device to a command input mode based on a result of the determination of the starting word.
The method of claim 9,
Further comprising a startup word database unit for storing the startup word,
Controlling, by the central controller, the voice input unit to enter a starting word verification mode based on a result of determining the starting word for the first voice received in the normal mode;
Receiving a voice including a voice word or a command by the voice input unit; And
The central control unit further comprises the step of storing the first voice input before entering the startup language verification mode, the startup language database unit, speech recognition.

Images