JP2022033258A - Speech control apparatus, operation method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech control apparatus for preventing false detection of a keyword and a method of operating the same.

SOLUTION: The method according to the present disclosure includes the steps of: receiving an audio signal corresponding to a surrounding sound; generating audio stream data; determining a first section in which a candidate keyword corresponding to a predetermined keyword is detected, from the audio stream data; extracting a first speaker feature vector for identifying a speaker in the first section and a second speaker feature vector for identifying a speaker in a second section adjacent to the first section from the audio stream data; and determining whether or not the predetermined keyword is included in the first section based on similarity between the first speaker feature vector and the second speaker feature vector.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a voice control device, and more particularly to a voice control device capable of preventing keyword misrecognition, an operation method of the voice control device, a computer program, a recording medium, and the like.

A voice recognition function is installed and controlled by voice in order to improve operability while improving the performance of computer devices such as portable communication devices, desktop PCs (personal computers), tablet PCs, and entertainment systems. Electronic devices are on the market. The voice recognition function has an advantage that the device can be easily controlled by recognizing a user's voice without using a separate button operation or touching a touch module.

According to such a voice recognition function, for example, in a portable communication device such as a smartphone, it is possible to perform a call function or compose a text message without pressing a separate button. You can easily set various functions such as directions, Internet search, and alarm settings. However, if such a voice control device erroneously recognizes the user's voice, a problem may occur in which an undesired operation is performed.

Korean Patent Publication No. 10-2017-0028628

An object to be solved by the present invention is to provide a voice control device, an operation method of the voice control device, a computer program, a recording medium, and the like, which can prevent erroneous recognition of keywords.

As a technical means for achieving the above-mentioned technical problems, the first aspect of the present disclosure is from an audio processing unit that receives an audio signal corresponding to an ambient sound and generates audio stream data, and from the audio stream data. , A keyword detection unit that detects a candidate keyword corresponding to a predetermined keyword and determines a start point and an end point of a first section corresponding to the first audio data in which the candidate keyword is detected, and the first item. The first speaker feature vector related to one audio data is extracted, and in the audio stream data, the second speaker feature vector related to the second audio data corresponding to the second section whose end point is the start point of the first section is used. Based on the degree of similarity between the speaker feature vector extraction unit to be extracted and the first speaker feature vector and the second speaker feature vector, whether or not the keyword is included in the first audio data is determined. A voice control device including a wake-up determination unit for determination can be provided.

Further, the second aspect of the present disclosure is a stage of receiving an audio signal corresponding to an ambient sound and generating audio stream data, and detecting a candidate keyword corresponding to a predetermined keyword from the audio stream data, and the audio. In the stream data, a step of determining the start point and the end point of the first section corresponding to the first audio data in which the candidate keyword is detected, and a step of extracting the first speaker feature vector related to the first audio data. In the audio stream data, a step of extracting a second speaker feature vector related to the second audio data corresponding to the second section whose end point is the start point of the first section, the first speaker feature vector, and the first speaker. 2. A voice control device including a step of determining whether or not the keyword is included in the first audio data based on the similarity with the speaker feature vector, and determining whether or not to wake up. It is possible to provide a method of operation.

Further, the third aspect of the present disclosure can provide a computer program including a command word for causing the processor of the voice control device to execute the operation method according to the second aspect.

Further, the fourth aspect of the present disclosure can provide a computer-readable recording medium in which the computer program according to the third aspect is recorded.

According to various embodiments of the present invention, the possibility of erroneously recognizing a keyword is reduced, so that a malfunction of the voice control device is prevented.

It is a drawing which illustrated the example of the network environment by one Embodiment. It is a block diagram for demonstrating the internal structure of an electronic device and a server by one Embodiment. It is a figure which illustrated the example of the functional block which can include the processor of the voice control apparatus by one Embodiment. It is a flowchart which illustrated an example of the operation method which a voice control apparatus can perform by one Embodiment. It is a flowchart which illustrates the example of the operation method which a voice control apparatus can perform by another embodiment. It is a figure which illustrates the example in which a single instruction keyword is uttered when the voice control device by one Embodiment executes the operation method of FIG. It is a figure which illustrates the example in which a general dialogue voice is uttered when the voice control device by one Embodiment executes the operation method of FIG. It is a flowchart which illustrated an example of the operation method which a voice control apparatus can perform by still another Embodiment. It is a figure which illustrates the example in which a wake-up keyword and a natural language voice command are uttered when the voice control device according to one embodiment executes the operation method of FIG. 7. It is a figure which illustrates the example in which a general dialogue voice is uttered when the voice control device by one Embodiment executes the operation method of FIG. 7.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the invention in the technical field to which the present invention belongs. However, the present invention is embodied in various different forms and is not limited to the embodiments described herein. In the drawings, in order to clearly explain the present invention, parts unrelated to the description are omitted, and similar parts are designated by similar drawing reference numerals throughout the specification.

In the entire specification, when one part is "connected" to another part, it is not only when it is "directly connected", but another element is sandwiched between them and "electrically". Including the case of being "concatenated to". Also, when a part "contains" a component, it does not exclude other components, but may further include other components, unless otherwise stated. ..

In the present specification, terms such as "in a partial embodiment" or "in one embodiment" appearing in various places do not necessarily indicate the same embodiment.

Some embodiments are shown in functional block configurations and various processing steps. Some or all of such functional blocks are also embodied by a diverse number of hardware and / or software configurations that perform a particular function. For example, the functional blocks of the present disclosure may be embodied by one or more microprocessors, or may also be embodied by a circuit configuration for a given function. Also, for example, the functional blocks of the present disclosure are embodied in various programming or scripting languages. The functional block is also embodied by an algorithm executed by one or more processors. Also, the present disclosure can employ prior art for electronic environment setting, signal processing, and / or data processing and the like. Terms such as "module" and "configuration" are generic, mechanical and not limited to physical configurations.

Also, the connecting lines or connecting members between the components illustrated in the drawings merely illustrate functional connection and / or physical or circuit connection. In a real device, various functional connections, physical connections or circuit connections that can be replaced or added indicate the connections between the components.

In the present disclosure, the keyword refers to voice information that can wake up a specific function of the voice control device. The keyword is both a single command keyword and a wake-up keyword based on the user's voice signal. The wake-up keyword is a voice-based keyword that can convert a voice control device in a sleep mode to a wake-up mode, and is also a voice keyword such as "clover" or "high computer". After uttering the wake-up keyword, the user can utter a command in a natural language form for instructing a function or operation desired to be performed by the voice control device. It should be noted that "Clover", which appears as a mere example of the wake-up keyword in the following explanation, is a registered trademark and is different from "Clover" in "four-leaf clover". In that case, the voice control device can voice-recognize the voice command in the natural language form and perform a function or an operation corresponding to the voice-recognized result. The single command keyword is also a voice keyword that can directly control the operation of the voice control device, for example, when music is being played, such as "stop". The wake-up keywords referred to in this disclosure are referred to by terms such as wake-up word, hot word, and trigger word.

In the present disclosure, the candidate keywords include words whose pronunciation is similar to that of the keywords. For example, when the keyword is "clover", the candidate keyword is also "clover", "global", "club" and the like. The candidate keyword is defined as one detected as a keyword from the audio data by the keyword detection unit of the voice control device. The candidate keyword is also the same as the keyword, but is also another word having a pronunciation similar to the keyword. In general, the voice control device may wake up by erroneously recognizing the keyword even when the user utters a sentence containing a term corresponding to the candidate keyword. The voice control device according to the present disclosure reacts even when the above-mentioned candidate keyword is detected from the voice signal, but it is possible to prevent wake-up by the candidate keyword.

In the present disclosure, the voice recognition function means converting a user's voice signal into a character string (or text). The user's voice signal may include voice instructions. The voice command can perform a specific function of the voice control device.

In the present disclosure, the voice control device refers to an electronic device equipped with a voice control function. Electronic devices equipped with voice control functions are also independent electronic devices such as smart speakers or artificial intelligence speakers. Further, the electronic device equipped with the voice control function is not only a computer device equipped with the voice control function, for example, a desktop PC (personal computer), a notebook personal computer, etc., but also a portable computer device, for example. , Smartphones, etc. In that case, a program or application for performing the voice control function is installed in the computer device. Further, the electronic device equipped with the voice control function is also an electronic product that mainly performs a specific function, for example, a smart TV, a smart refrigerator, a smart air conditioner, a smart navigation, etc., and is also an infotainment system for an automobile. Not only that, but things Internet devices that are controlled by voice also fall under this category.

In the present disclosure, a particular function of a voice control device may include, but is not limited to, executing, for example, an application installed in the voice control device. For example, when the voice control device is a smart speaker, the specific function of the voice control device may include music playback, Internet shopping, voice information provision, control of an electronic device or a mechanical device connected to the smart speaker, and the like. For example, if the voice control device is a smartphone, the application execution may include making a call, finding a way, searching the Internet or setting an alarm. For example, if the voice control device is a smart television, the application execution may include program search, channel search, and the like. If the voice control device is a smart oven, the application execution may include cooking method search and the like. When the voice control device is a smart refrigerator, the application execution may include checking the refrigerated and frozen states, setting the temperature, and the like. When the voice control device is a smart vehicle, the application execution may include automatic start, autonomous driving, automatic parking, and the like. Application execution in the present disclosure is not limited to the above.

In the present disclosure, the keyword may have a word form or a spherical form. In the present disclosure, the speech command uttered after the wake-up keyword can have a sentence form, a word form, or a spherical form in a natural language form.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

FIG. 1 is a drawing illustrating an example of a network environment according to an embodiment. The network environment illustrated in FIG. 1 is schematically illustrated to include a plurality of electronic devices 100a to 100f, a server 200, and a network 300.

The electronic devices 100a to 100f are exemplary electronic devices controlled by voice. Each of the electronic devices 100a to 100f can perform a specific function in addition to the voice recognition function. Examples of electronic devices 100a to 100f include smart speakers or artificial intelligence speakers, smartphones, mobile phones, navigation systems, computers, notebook computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), etc. There are tablet PCs, smart electronic products, etc. The electronic devices 100a to 100f can communicate with the server 200 and / or other electronic devices 100a to 100f via the network 300 by using a wireless or wired communication method. However, the present invention is not limited to this, and each of the electronic devices 100a to 100f is not connected to the network 300 and can operate independently. The electronic devices 100a to 100f are also collectively referred to as the electronic device 100.

The communication method of the network 300 is not limited, and is not limited to a communication method utilizing a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 300 can include, as well as an electronic device 100a. It may also include short-range wireless communication between 100f and 100f. For example, the network 300 includes a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), a WAN (wide area network), a BBN (broadband network), the Internet, and the like. It may include any one or more of the networks. Also, the network 300 may include any one or more of network topologies including bus networks, star networks, ring networks, mesh networks, star bus networks, tree networks, hierarchical networks, and the like. It is not limited to.

The server 200 is also embodied by a computer device that communicates with electronic devices 100a to 100f via a network 300 and performs a voice recognition function, or a plurality of computer devices. The server 200 is distributed in the form of a cloud and can provide instructions, codes, files, contents, and the like.

For example, the server 200 receives the audio file provided from the electronic devices 100a to 100f, converts the audio signal in the audio file into a character string (or text), and converts the converted character string (or text) into a character string (or text). , Can be provided for electronic devices 100a to 100f. Further, the server 200 can provide the electronic devices 100a to 100f connected via the network 300 with a file for installing an application for performing the voice control function. For example, the second electronic device 100b can use the file provided by the server 200 to install an application. The second electronic device 100b connects to the server 200 under the control of an installed operating system (OS) and / or at least one program (for example, an installed voice control application), and is a voice recognition service provided by the server 200. Will be provided.

FIG. 2 is a block diagram for explaining an internal configuration of an electronic device and a server according to an embodiment.

The electronic device 100 is one of the electronic devices 100a to 100f of FIG. 1, and the electronic devices 100a to 100f can have at least the internal configuration shown in FIG. Although the electronic device 100 is shown to be connected to a server 200 that performs a voice recognition function via a network 300, it is an example, and the electronic device 100 independently performs a voice recognition function. It can also be accomplished. The electronic device 100 is an electronic device controlled by voice, and is also called a voice control device 100. The voice control device 100 is embodied in a smart speaker or an artificial intelligence speaker, a computer device, a portable computer device, a smart home appliance, or the like, or is connected to them by wire and / or wirelessly.

The electronic device 100 and the server 200 may include memories 110, 210, processors 120, 220, communication modules 130, 230, and input / output interfaces 140, 240. The memories 110 and 210 are computer-readable recording media, and include a permanent mass storage device such as a RAM (random access memory), a ROM (read-only memory), and a disk drive. Can include. Further, the operation system and at least one program code (for example, a code for a voice control application, a voice recognition application, etc. installed and driven in the electronic device 100) are stored in the memories 110 and 210. Such software components are loaded into memory 110, 210 via communication modules 130, 230, which are not computer readable recording media. For example, at least one program is loaded into memory 110, 210 based on the program installed by the developer, or the file distribution system that distributes the application installation files, by the files provided over the network 300.

Processors 120, 220 are configured to process instructions in a computer program by performing basic arithmetic, logic, and input / output operations. The instructions are also provided to processors 120, 220 by memory 110, 210 or communication modules 130, 230. For example, the processors 120, 220 are also configured to execute instructions received by a program code stored in a recording device such as memories 110, 210.

The communication modules 130 and 230 can provide a function for the electronic device 100 and the server 200 to communicate with each other via the network 300, and provide a function for communicating with other electronic devices 100b to 100f. be able to. As an example, a request (for example, a voice recognition service request) generated by a processor 120 of an electronic device 100 by a program code stored in a recording device such as a memory 110 is controlled by a communication module 130 via a network 300. Is transmitted to the server 200. On the contrary, the character string (text) which is the voice recognition result provided by the control of the processor 220 of the server 200 passes through the communication module 230 and the network 300, and via the communication module 130 of the electronic device 100, the electronic device 100. Is received by. For example, the voice recognition result of the server 200 received via the communication module 130 is transmitted to the processor 120 and the memory 110. The server 200 can transmit control signals, instructions, contents, files, etc. to the electronic device 100, and the control signals, instructions, etc. received via the communication module 130 are transmitted to the processor 120, the memory 110, and the contents. And files are also stored in a separate recording medium that may be further included in the electronic device 100.

The input / output interfaces 140 and 240 are also means for interfacing with the input / output device 150. For example, the input device includes not only the microphone 151 but also a device such as a keyboard or a mouse, and the output device is not only the speaker 152 but also a state indicator LED (light emitting diode) indicating a state and a communication session of the application. Can include devices such as displays in. As another example, the input / output device 150 may include a device that integrates functions for input and output, such as a touch screen.

The microphone 151 can convert ambient sounds into electrical audio signals. The microphone 151 is not directly mounted in the electronic device 100, but is mounted in an external device (for example, a smart watch) that is communicably connected, and the generated external signal is transmitted to the electronic device 100 by communication. FIG. 2 is shown so that the microphone 151 is included inside the electronic device 100, but according to another embodiment, the microphone 151 is included in a separate device and is not the electronic device 100. , It is also embodied in the form of being connected by wired communication or wireless communication.

In other embodiments, the electronic device 100 and the server 200 may include more components than the components of FIG. For example, the electronic device 100 may be configured to include at least a portion of the above-mentioned input / output devices 150, or may include a transceiver, a GPS (global position system) module, a camera, various sensors, a database, and the like. Other components may be further included.

FIG. 3 is a drawing illustrating an example of a functional block that can be included in the processor of the voice control device according to the embodiment, and FIG. 4 is an example of an operation method that the voice control device can perform according to the embodiment. It is a flowchart illustrated.

As shown in FIG. 3, the processor 120 of the voice control device 100 includes an audio processing unit 121, a keyword detection unit 122, a speaker feature vector extraction unit 123, a wake-up determination unit 124, a voice recognition unit 125, and a functional unit 126. Can include. At least a portion of such processors 120 and functional blocks 121-126 can control the voice control device 100 to perform the steps (S110-S190) included in the operation method illustrated in FIG. .. For example, at least a part of the processor 120 and the functional blocks 121 to 126 of the processor 120 are to execute the operation system code included in the memory 110 of the voice control device 100 and the instruction by at least one program code. Is also embodied.

A part or all of the functional blocks 121 to 126 illustrated in FIG. 3 is also embodied in a hardware configuration and / or a software configuration that performs a specific function. The function performed by the functional blocks 121 to 126 illustrated in FIG. 3 is embodied by one or more microprocessors, or is also embodied by a circuit configuration for that function. Some or all of the functional blocks 121 to 126 illustrated in FIG. 3 are also software modules composed of various programming or scripting languages executed by the processor 120. For example, the audio processing unit 121 and the keyword detection unit 122 are embodied by a digital signal processor (DSP), and the speaker feature vector extraction unit 123, the wake-up determination unit 124, and the voice recognition unit 125 are also embodied by a software module. Will be done.

The audio processing unit 121 receives an audio signal corresponding to the ambient sound and generates audio stream data. The audio processing unit 121 can receive an audio signal corresponding to the ambient sound from an input device such as the microphone 151. The microphone 151 is included in a peripheral device connected to the voice control device 100 by communication, and the audio processing unit 121 can receive the audio signal generated by the microphone 151 by communication. The peripheral sound includes not only the voice spoken by the user but also the background sound. Therefore, the audio signal includes not only the audio signal but also the background sound signal. The background sound signal corresponds to noise in keyword detection and voice recognition.

The audio processing unit 121 can generate audio stream data corresponding to continuously received audio signals. The audio processing unit 121 can filter and digitize the audio signal to generate audio stream data. The audio processing unit 121 can filter the audio signal to remove the noise signal and amplify the audio signal as compared with the background sound signal. Further, the audio processing unit 121 can also remove the echo of the audio signal from the audio signal.

The audio processing unit 121 can always operate in order to receive the audio signal even when the voice control device 100 operates in the sleep mode. The audio processing unit 121 can operate at a low operating frequency when the voice control device 100 operates in the sleep mode, and can operate at a high operating frequency when the voice control device 100 operates in the normal mode.

The memory 110 can temporarily store the audio stream data generated by the audio processing unit 121. The audio processing unit 121 can use the memory 110 to buffer the audio stream data. In the memory 110, not only the audio data including the keyword but also the audio data before the keyword is detected is stored together. In order to save the latest audio data in the memory 110, the earliest audio data stored in the memory 110 is deleted. If the size allocated to the memory 110 is the same, the audio data for the same period is always stored. It is desirable that the period corresponding to the audio data stored in the memory 110 is longer than the time for uttering the keyword.

According to another embodiment of the present invention, the memory 110 can extract and store the speaker feature vector related to the audio stream generated by the audio processing unit 121. At that time, the speaker feature vector is extracted and stored for an audio stream of a specific length. As mentioned above, the earliest saved speaker feature vector is deleted in order to save the speaker feature vector associated with the recently generated audio stream.

The keyword detection unit 122 detects a candidate keyword corresponding to a defined (that is, a predetermined) keyword from the audio stream data generated by the audio processing unit 121. The keyword detection unit 122 can detect a candidate keyword corresponding to a defined keyword from the audio stream data temporarily stored in the memory 110. A plurality of predefined keywords may exist, and the plurality of defined keywords are stored in the keyword storage location 110a. The keyword storage 110a may be included in the memory 110.

The candidate keyword means the audio stream data detected as a keyword by the keyword detection unit 122. The candidate keyword may be the same as the keyword, or may be another word pronounced similar to the keyword. For example, when the keyword is "clover", the candidate keyword may be "global". That is, when the user utters a sentence including "global", the keyword detection unit 122 may misidentify "global" as "clover" and detect it from the audio stream data. The "global" detected in this way corresponds to the candidate keyword.

The keyword detection unit 122 can compare the audio stream data with the known keyword data and calculate the possibility that the audio stream data includes the voice corresponding to the keyword. The keyword detector 122 can extract audio features such as filter bank energy or Mel-frequency cepstram coefficients (MFCC) from the audio stream data. The keyword detector 122 can use a classification window to process such audio features, for example, using a support vector machine or a neural network. .. Based on the processing of the audio feature, the keyword detection unit 122 can calculate the possibility that the keyword is included in the audio stream data. When the possibility is higher than the set reference value (that is, a predetermined reference value), the keyword detection unit 122 detects the candidate keyword by determining that the keyword is included in the audio stream data. Can be done.

The keyword detection unit 122 generates an artificial neural network (for example, a neural network for artificial intelligence) using a voice sample corresponding to the keyword data, and uses the generated neural network to generate a keyword from the audio stream data. Is trained to detect. The keyword detection unit 122 can calculate the probability of phonemes constituting the keyword or the overall probability of the keyword for each frame in the audio stream data. The keyword detection unit 122 can output the probability sequence corresponding to each phoneme or the probability of the keyword itself from the audio stream data. Based on the sequence or probability, the keyword detection unit 122 can calculate the possibility that the keyword is included in the audio stream data, and if the possibility is equal to or higher than the set reference value, the candidate keyword is detected. It can be judged that it was. The above-mentioned method is exemplary, and the operation of the keyword detection unit 122 is embodied through various methods.

Further, the keyword detection unit 122 extracts audio features for each frame in the audio stream data, so that the audio data of the frame may correspond to human voice and the background sound. Can be calculated. The keyword detection unit 122 can compare the possibility of corresponding to human voice and the possibility of corresponding to background sound, and can determine that the audio data of the frame corresponds to human voice. For example, in the keyword detection unit 122, when the possibility that the audio data of the frame corresponds to human voice is higher than the possibility that it corresponds to the background sound exceeds the set reference value, the audio data of the frame is human. It can be judged that it corresponds to the voice of.

The keyword detection unit 122 can specify the section in which the candidate keyword is detected from the audio stream data, and can determine the start point and the end point of the section in which the candidate keyword is detected. The section in which the candidate keyword is detected from the audio stream data is regarded as the keyword detection section, the current section, or the first section. The audio data corresponding to the first section in the audio stream data is referred to as the first audio data. The keyword detection unit 122 can determine the end of the section in which the candidate keyword is detected as the end point. According to another example, after the candidate keyword is detected, the keyword detection unit 122 waits until a silent sound for a set time (for example, 0.5 seconds) is generated, and then a silent sound section is set in the first section. The end point of the first section can be determined so that it is included, or the end point of the first section can be determined so that the silent period is not included.

The speaker feature vector extraction unit 123 reads the second audio data corresponding to the second section from the memory 110 in the audio stream data temporarily stored in the memory 110. The second section is the previous section of the first section, and the end point of the second section is also the same as the start point of the first section. The second section is the previous section. The length of the second section is also variably set by the keyword corresponding to the detected candidate keyword. According to another example, the length of the second section is also fixedly set. According to yet another example, the length of the second interval is adaptively variable so that the keyword detection performance is optimized. For example, when the audio signal output by the microphone 151 is "four-leaf clover" and the candidate keyword is "clover", the second audio data corresponds to the voice "four-leaf clover".

The speaker feature vector extraction unit 123 extracts the first speaker feature vector of the first audio data corresponding to the first section and the second speaker feature vector of the second audio data corresponding to the second section. .. The speaker feature vector extraction unit 123 can extract a speaker feature vector that is robust to speaker recognition from audio data. The speaker feature vector extraction unit 123 converts the voice signal in the time domain into the signal in the frequency domain, and transforms the frequency energy of the converted signal so as to be different from each other. The speaker feature vector can be extracted. For example, the speaker feature vector is extracted based on the Mel Frequency Cepstrum Coefficient (MFCC) or filter bank energy, but is not limited to them, and the speaker feature vector can be obtained from audio data in various ways. Can be extracted.

The speaker feature vector extraction unit 123 can generally operate in the sleep mode. When the keyword detection unit 122 detects a candidate keyword from the audio stream data, the speaker feature vector extraction unit 123 can wake up. When the keyword detection unit 122 detects a candidate keyword from the audio stream data, the keyword detection unit 122 can transmit a wake-up signal to the speaker feature vector extraction unit 123. The speaker feature vector extraction unit 123 is waked up in response to the wakeup signal indicating that the candidate keyword has been detected in the keyword detection unit 122.

According to one embodiment, the speaker feature vector extraction unit 123 extracts a frame feature vector for each frame of the audio data, normalizes and averages the extracted frame feature vector, and represents the audio data. The speaker feature vector can be extracted. L2 normalization is used to normalize the extracted frame feature vectors. The averaging of the extracted frame feature vectors is achieved by calculating the average of the normalized frame feature vectors generated by normalizing the extracted frame feature vectors for each of the frames in the audio data. To.

For example, the speaker feature vector extraction unit 123 extracts the first frame feature vector for each frame of the first audio data, normalizes and averages the extracted first frame feature vector, and obtains the first audio data. The representative first speaker feature vector can be extracted. Further, the speaker feature vector extraction unit 123 extracts the second frame feature vector for each frame of the second audio data, normalizes and averages the extracted second frame feature vector, and obtains the second audio data. A representative second speaker feature vector can be extracted.

According to another embodiment, the speaker feature vector extraction unit 123 does not extract the frame feature vector for all the frames in the audio data, but extracts the frame feature vector for some frames in the audio data. Can be extracted. The partial frame is selected as a voice frame in a frame in which the audio data of the frame is likely to be the voice data of the user. The selection of such a voice frame is made by the keyword detection unit 122. The keyword detection unit 122 can calculate a first probability of human voice and a second probability of background sound for each frame of audio stream data. The keyword detection unit 122 can determine a frame in which the first probability that the audio data of each frame is human voice is higher than the second probability that the audio data is the background sound exceeds the set reference value as the voice frame. The keyword detection unit 122 can store a flag or a bit indicating whether or not the frame is an audio frame in the memory 110 in association with each frame of the audio stream data.

When the speaker feature vector extraction unit 123 reads the first audio data and the second audio data from the memory 110, it can know whether or not the frame is a voice frame by reading the flag or the bit together. can.

The speaker feature vector extraction unit 123 extracts the frame feature vector for each of the frames determined as the voice frame among the frames in the audio data, normalizes and averages the extracted first frame feature vector, and obtains the audio data. A representative speaker feature vector can be extracted. For example, the speaker feature vector extraction unit 123 extracts the first frame feature vector for each of the frames determined to be the audio frame in the frames in the first audio data, and normalizes the extracted first frame feature vector. And averaged, the first speaker feature vector representing the first audio data can be extracted. Further, the speaker feature vector extraction unit 123 extracts the second frame feature vector for each of the frames determined to be the audio frame in the frames in the second audio data, and normalizes the extracted second frame feature vector. And averaged, a second speaker feature vector representing the second audio data can be extracted.

Whether the wake-up determination unit 124 includes the keyword in the first audio data based on the degree of similarity between the first speaker feature vector and the second speaker feature vector extracted by the speaker feature vector extraction unit 123. It is determined whether or not, that is, whether or not the keyword is included in the audio signal in the first section. The wake-up determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with the set reference value, and if the similarity is equal to or less than the reference value, the first audio in the first section. It can be determined that the keyword is included in the data.

A typical case where the voice control device 100 erroneously recognizes a keyword is a case where a word having a pronunciation similar to the keyword is located in the middle of the voice in the user's voice. For example, when the keyword is "clover", the voice control device 100 is waked up in response to the "clover" even when the user asks another person "how to find the four-leaf clover", and the user intends to do so. You may perform actions that you have not done. Furthermore, even when the announcer says "The market capitalization of JN Global is ..." in TV news, the voice control device 100 may be woken up in response to "global". In order to prevent such misrecognition of the keyword from occurring, according to one embodiment, the voice control device 100 uses the voice control device 100 only when the word having a pronunciation similar to the keyword is located at the earliest position of the voice. react. Also, in an environment with a lot of ambient background noise or in an environment where other people are discussing, even if the user utters the voice corresponding to the keyword first, the user will be affected by the ambient background noise and the dialogue of other people. It may not be perceived that the voice corresponding to the keyword was uttered first. According to one embodiment, the voice control device 100 extracts the first speaker feature vector in the section in which the candidate keyword is detected and the second speaker feature vector in the previous section, and uses the first speaker feature vector. When the second speaker feature vectors are different from each other, it can be determined that the user utters the voice corresponding to the keyword first.

For such a determination, the wake-up determination unit 124 determines that the user corresponds to the keyword when the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value. Can be judged to have been uttered first. That is, the wake-up determination unit 124 can determine that the keyword is included in the first audio data in the first section, and can wake up some functions of the voice control device 100. The high degree of similarity between the first speaker feature vector and the second speaker feature vector means that the person who emitted the voice corresponding to the first audio data and the person who emitted the voice corresponding to the second audio data. Is likely to be the same.

When the second audio data corresponds to the silent sound, the speaker feature vector extraction unit 123 can extract the second speaker feature vector corresponding to the silent sound from the second audio data. Since the speaker feature vector extraction unit 123 extracts the first speaker feature vector corresponding to the user's voice from the first audio data, the degree of similarity between the first speaker feature vector and the second speaker feature vector is high. ,Low.

The voice recognition unit 125 can receive the third audio data corresponding to the third section in the audio stream data generated by the audio processing unit 121, and can recognize the third audio data by voice. According to another example, the voice recognition unit 125 transmits the third audio data to the outside so that the third audio data is voice-recognized by an external device (for example, the server 200), and receives the voice recognition result. be able to.

The functional unit 126 can perform a function corresponding to the keyword. For example, when the voice control device 100 is a smart speaker, the function unit 126 includes a music playback unit, a voice information providing unit, a peripheral device control unit, and the like, and can perform a function corresponding to the detected keyword. When the voice control device 100 is a smartphone, the functional unit 126 can perform a function corresponding to the detected keyword, including a telephone connection unit, a character transmission / reception unit, an Internet search unit, and the like. The functional unit 126 is variously configured depending on the type of the voice control device 100. The functional unit 126 comprehensively shows a functional block for performing various functions that can be performed by the voice control device 100.

The voice control device 100 illustrated in FIG. 3 is shown to include a voice recognition unit 125, which is exemplary, and the voice control device 100 does not include the voice recognition unit 125, FIG. The server 200 illustrated in 1 can perform the voice recognition function instead. In that case, as shown in FIG. 1, the voice control device 100 is connected to the server 200 that performs the voice recognition function via the network 300. The voice control device 100 can provide a voice file including a voice signal requiring voice recognition to the server 200, and the server 200 performs voice recognition for the voice signal in the voice file and corresponds to the voice signal. You can generate a string. The server 200 can transmit the generated character string to the voice control device 100 via the network 300. However, in the following, it is assumed that the voice control device 100 includes a voice recognition unit 125 that performs a voice recognition function.

The processor 120 can load the program code stored in the program file for the operation method into the memory 110. For example, a program is installed in the voice control device 100 by a program file. At that time, when the program installed in the voice control device 100 is executed, the processor 120 can load the program code into the memory 110. At that time, at least a part of the audio processing unit 121, the keyword detection unit 122, the speaker feature vector extraction unit 123, the wake-up determination unit 124, the voice recognition unit 125, and the functional unit 126 included in the processor 120 are stored in the memory 110. It is also embodied to execute the instruction by the corresponding code in the loaded program code and execute the step (S110 to S190) of FIG.

After that, it is understood that the functional blocks 121 to 126 of the processor 120 controlling the voice control device 100 means that the processor 120 controls other components of the voice control device 100. For example, the processor 120 can control the communication module 130 included in the voice control device 100, and the voice control device 100 can control the voice control device 100 so that the voice control device 100 communicates with, for example, the server 200.

In the stage (S110), the processor 120, for example, the audio processing unit 121 receives the audio signal corresponding to the ambient sound. The audio processing unit 121 can continuously receive an audio signal corresponding to the ambient sound. The audio signal is also an electrical signal generated by an input device such as the microphone 151 in response to ambient sound.

In the step (S120), the processor 120, for example, the audio processing unit 121, generates audio stream data based on the audio signal from the microphone 151. The audio stream data corresponds to the continuously received audio signal. The audio stream data is also data generated by filtering and digitizing an audio signal.

In the stage (S130), the processor 120, for example, the audio processing unit 121 temporarily stores the audio stream data generated in the stage (S120) in the memory 110. The memory 110 has a limited size, and a part of the audio stream data corresponding to the audio signal for a certain period of time from the present is temporarily stored in the memory 110. When new audio stream data is generated, the oldest audio stream data saved in the memory 110 is deleted, and the new audio stream data is saved in the space freed by the deletion in the memory 110. Will be done.

In the stage (S140), the processor 120, for example, the keyword detection unit 122 detects a candidate keyword corresponding to the defined keyword from the audio stream data generated in the stage (S120). The candidate keyword is a word having a pronunciation similar to that of the defined keyword, and refers to a word detected as a keyword by the keyword detection unit 122 in the step (S140).

In the stage (S150), the processor 120, for example, the keyword detection unit 122 identifies the keyword detection section in which the candidate keyword is detected from the audio stream data, and determines the start point and the end point of the keyword detection section. The keyword detection section is the current section. The data corresponding to the current section in the audio stream data is regarded as the first audio data.

In the stage (S160), the processor 120, for example, the speaker feature vector extraction unit 123 reads the second audio data corresponding to the previous section from the memory 110. The previous section is the section immediately before the current section, and the end point of the previous section is also the same as the start point of the current section. The speaker feature vector extraction unit 123 can also read the first audio data from the memory 110.

In the stage (S170), the processor 120, for example, the speaker feature vector extraction unit 123 extracts the first speaker feature vector from the first audio data and extracts the second speaker feature vector from the second audio data. The first speaker feature vector is an index for identifying the voice speaker corresponding to the first audio data, and the second speaker feature vector identifies the voice speaker corresponding to the second audio data. It is an index for. The processor 120, for example, the wake-up determination unit 124 determines whether or not the keyword is included in the first audio data based on the similarity between the first speaker feature vector and the second speaker feature vector. be able to. When the wake-up determination unit 124 determines that the first audio data contains a keyword, the wake-up determination unit 124 can wake up some components of the voice control device 100.

In the step (S180), the processor 120, for example, the wake-up determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with the set reference value.

When the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, the wake-up determination unit 124 has the speaker of the first audio data in the current section and the second speaker in the previous section. Since the speakers of the audio data are different from each other, it can be determined that the first audio data contains the keyword. In that case, as in step (S190), the processor 120, for example, the wakeup determination unit 124, can wake up some components of the voice control device 100.

However, when the similarity between the first speaker feature vector and the second speaker feature vector is higher than the set reference value, the wakeup determination unit 124 is the speaker of the first audio data in the current section and the first speaker in the previous section. 2 Since the speakers of the audio data are the same as each other, it is judged that the keyword is not included in the first audio data, and the wake-up cannot be proceeded. In that case, the process proceeds to the stage (S110), and the audio signal corresponding to the ambient sound is received. In the stage (S110), the audio signal reception is also continued when performing the stage (S120-S190).

A plurality of defined keywords are stored in the keyword storage 110a of FIG. Such keywords are both wake-up keywords and single command keywords. The wake-up keyword is for wake-up a part of the functions of the voice control device 100. Generally, the user utters the desired natural speech command after uttering the wake-up keyword. The voice control device 100 can voice-recognize a natural language voice command and perform an operation and a function corresponding to the natural language voice command.

The single command keyword is for the voice control device 100 to directly perform a specific operation or function, and is also a defined simple word such as "play" or "stop". When the voice control device 100 receives the single command keyword, the voice control device 100 can wake up the function corresponding to the single command keyword and perform the function.

Hereinafter, the case where the candidate keyword corresponding to the single instruction keyword is detected from the audio stream data and the case where the candidate keyword corresponding to the wake-up keyword is detected from the audio stream data will be described.

FIG. 5 is a flowchart illustrating an example of an operation method that can be performed by the voice control device according to another embodiment.

FIG. 6A illustrates an example in which a single instruction keyword is spoken when the voice control device according to one embodiment executes the operation method of FIG. 5, and FIG. 6B shows an example in which the voice control device according to one embodiment is a voice control device according to FIG. An example in which a general dialogue voice is uttered when executing the operation method of is illustrated.

The operation method of FIG. 5 includes a step that is substantially the same as the operation method of FIG. Of the stages of FIG. 5, the stages that are substantially the same as the stages of FIG. 4 will not be described in detail. 6A and 6B show an audio signal corresponding to the audio stream data and a user's voice corresponding to the audio signal. FIG. 6A shows an audio signal corresponding to the voice "stop", and FIG. 6B shows an audio signal corresponding to the voice "stop here".

Referring to FIG. 5 together with FIGS. 6A and 6B, in step (S210), the processor 120, for example, the audio processing unit 121, receives an audio signal corresponding to the ambient sound.

In step (S220), the processor 120, for example, the audio processing unit 121, generates audio stream data based on the audio signal from the microphone 151.

In the stage (S230), the processor 120, for example, the audio processing unit 121 temporarily stores the audio stream data generated in the stage (S220) in the memory 110.

In the stage (S240), the processor 120, for example, the keyword detection unit 122 detects the candidate keyword corresponding to the defined single instruction keyword from the audio stream data generated in the stage (S220). The single command keyword is also a voice keyword that can directly control the operation of the voice control device 100. For example, the single command keyword is also a word such as "stop", as illustrated in FIG. 6A. In that case, the voice control device 100 is playing music or moving images, for example.

In the example of FIG. 6A, the keyword detection unit 122 can detect the candidate keyword "stop" from the audio signal. In the example of FIG. 6B, the keyword detection unit 122 can detect the candidate keyword "stop", which is a word having a pronunciation similar to the keyword "stop", from the audio signal.

In the stage (S250), the processor 120, for example, the keyword detection unit 122 identifies the keyword detection section in which the candidate keyword is detected from the audio stream data, and determines the start point and the end point of the keyword detection section. The keyword detection section is the current section. In the audio stream data, the data corresponding to the current section is regarded as the first audio data.

In the example of FIG. 6A, the keyword detection unit 122 can identify the section in which the candidate keyword "stop" is detected as the current section, and determine the start point and the end point of the current section. The audio data corresponding to the current section is referred to as the first audio data AD1.

In the example of FIG. 6B, the keyword detection unit 122 can identify the section in which the candidate keyword "stop" is detected as the current section, and determine the start point and the end point of the current section. The audio data corresponding to the current section is referred to as the first audio data AD1.

Further, in the stage (S250), the processor 120, for example, the keyword detection unit 122 determines which of the wake-up keyword and the single instruction keyword the detected candidate keyword corresponds to. be able to. In the examples of FIGS. 6A and 6B, the keyword detection unit 122 can determine that the detected candidate keywords, that is, "stop" and "stop" are candidate keywords corresponding to the single command keywords.

In the stage (S260), the processor 120, for example, the speaker feature vector extraction unit 123 reads the second audio data corresponding to the previous section from the memory 110. The previous section is the section immediately before the current section, and the end point of the previous section is also the same as the start point of the current section. The speaker feature vector extraction unit 123 can also read the first audio data from the memory 110.

In the example of FIG. 6A, the speaker feature vector extraction unit 123 can read the second audio data AD2 corresponding to the previous section, which is the section immediately immediately before the current section, from the memory 110. In the example of FIG. 6B, the speaker feature vector extraction unit 123 can read the second audio data AD2 corresponding to the previous section, which is the section immediately before the current section, from the memory 110. In the example of FIG. 6B, the second audio data AD2 corresponds to the voice "kode". The length of the previous interval is also variably set by the detected candidate keyword.

In the stage (S270), the processor 120, for example, the speaker feature vector extraction unit 123 receives the third audio data corresponding to the next section after the current section from the audio processing unit 121. The next section is the section immediately following the current section, and the start point of the next section is also the same as the end point of the current section.

In the example of FIG. 6A, the speaker feature vector extraction unit 123 can receive the third audio data AD3 corresponding to the next section immediately after the current section from the audio processing unit 121. In the example of FIG. 6B, the speaker feature vector extraction unit 123 can receive the third audio data AD3 corresponding to the next section immediately after the current section from the audio processing unit 121. In the example of FIG. 6B, the third audio data AD3 corresponds to the voice "to". The length of the next interval is also variably set by the detected candidate keywords.

In the stage (S280), the processor 120, for example, the speaker feature vector extraction unit 123 extracts the first speaker feature vector or the third speaker feature vector from the first audio data 1 to the third audio data, respectively. Each of the first speaker feature vector and the third speaker feature vector is an index for identifying the speaker of the voice corresponding to the first audio data 1 to the third audio data. The processor 120, for example, the wake-up determination unit 124, is based on the similarity between the first speaker feature vector and the second speaker feature vector, and the similarity between the first speaker feature vector and the third speaker feature vector. , It can be determined whether or not the single command keyword is included in the first audio data. When the wake-up determination unit 124 determines that the first audio data includes a single command keyword, the wake-up determination unit 124 can wake up some components of the voice control device 100.

In the example of FIG. 6A, the first speaker feature vector corresponding to the first audio data AD1 is an index for identifying the speaker who utters the voice "stop". Since the second audio data AD2 and the third audio data AD3 are substantially silent, the second speaker feature vector and the third speaker feature vector can have a vector corresponding to the silent sound. Therefore, the degree of similarity between the first speaker feature vector and the second speaker feature vector and the third speaker feature vector is low.

As another example, when another person who is not the speaker who uttered the voice "stop" utters the voice in the previous section and the next section, the second speaker feature vector and the third speaker feature vector are described above. Since it has a vector corresponding to another person, the degree of similarity between the first speaker feature vector, the second speaker feature vector, and the third speaker feature vector is low.

In the example of FIG. 6B, one person uttered "Stop here". Therefore, the first speaker feature vector extracted from the first audio data AD1 corresponding to "stop", the second speaker feature vector extracted from the second audio data AD2 corresponding to "kode", and the "speaker feature vector". Since the third speaker feature vector extracted from the third audio data AD3 corresponding to "te" is a vector for identifying speakers that are substantially the same, the first speaker feature vector or the first speaker feature vector. The degree of similarity with the three-speaker feature vector is high.

In the stage (S290), the processor 120, for example, the wake-up determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with the set reference value, and compares the similarity between the first speaker feature vector and the first speaker feature vector. And the third speaker feature vector are compared with the set reference value. In the wake-up determination unit 124, the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, and the similarity between the first speaker feature vector and the third speaker feature vector is If it is less than or equal to the set reference value, the speaker of the first audio data in the current section is different from the speaker of the second audio data in the previous section and the speaker of the third audio data in the next section. 1 It can be determined that the audio data includes a single command keyword. In that case, as in the step (S300), the processor 120, for example, the wake-up determination unit 124, provides the single instruction keyword to the functional unit 126, and the functional unit 126 uses the wake-up determination unit 124 for the first audio data. In response to the determination that the single instruction keyword is included, the function corresponding to the single instruction keyword can be performed.

In the example of FIG. 6A, the first speaker feature vector is a vector corresponding to the speaker who utters "stop", and the second speaker feature vector and the third speaker feature vector are vectors corresponding to silence. Therefore, the degree of similarity between the first speaker feature vector, the second speaker feature vector, and the third speaker feature vector is lower than the set reference value. In that case, the wake-up determination unit 124 can determine that the first audio data AD1 includes the single instruction keyword "stop". In that case, the functional unit 126 is waked up in response to the determination, and can perform an operation or a function corresponding to the single command keyword "stop". For example, if the voice control device 100 is playing music, the functional unit 126 can stop the music playback in response to the single command keyword "stop".

However, in the wake-up determination unit 124, the similarity between the first speaker feature vector and the second speaker feature vector is higher than the set reference value, or the first speaker feature vector and the third speaker feature vector If the similarity of is higher than the set reference value, the speaker of the first audio data in the current section is the same as the speaker of the second audio data in the previous section or the speaker of the third audio data in the next section. Therefore, it is judged that the keyword is not included in the first audio data, and the wake-up cannot be proceeded. In that case, the process proceeds to the stage (S210), and the audio signal corresponding to the ambient sound is received.

In the example of FIG. 6B, since one person uttered "Stop here", the similarity between the first speaker feature vector and the third speaker feature vector is high. Since the utterance "stop here" in the example of FIG. 6B does not include a keyword for controlling or wake-up the voice control device, the wake-up determination unit 124 uses the first audio data AD1. It is determined that the single command keyword is not included in the function unit 126, and the function unit 126 is prevented from performing the function or operation corresponding to "stop" or "stop".

According to general technology, it is technically possible for a voice control device to detect the voice "stop" among the utterances "stop here" and perform a function or operation corresponding to "stop". Is possible. Such functions and operations are not intended by the user, and the user feels inconvenience when using the voice control device. However, according to one embodiment, the voice control device 100 can accurately recognize the single command keyword from the voice of the user, and therefore, unlike the general technique, does not perform a malfunction.

FIG. 7 is a flowchart illustrating an example of an operation method that can be performed by the voice control device according to still another embodiment.

FIG. 8A illustrates an example in which a wake-up keyword and a natural language voice command are uttered when the voice control device according to the embodiment executes the operation method of FIG. 7, and FIG. 8B shows a voice control device according to the embodiment. However, when the operation method of FIG. 7 is executed, an example in which a general dialogue voice is uttered is illustrated.

The operation method of FIG. 7 includes a step that is substantially the same as the operation method of FIG. Of the stages of FIG. 7, the stages that are substantially the same as the stages of FIG. 4 will not be described in detail. 6A and 6B show an audio signal corresponding to the audio stream data and a user's voice corresponding to the audio signal. FIG. 8A illustrates the audio signal corresponding to the wake-up keyword "clover" and the natural voice command "tell me the weather tomorrow", and FIG. 8B shows the dialogue "how to find the four-leaf clover". The audio signal corresponding to the voice is illustrated.

Referring to FIG. 7 together with FIGS. 8A and 8B, in step (S410), the processor 120, for example, the audio processing unit 121, receives an audio signal corresponding to the ambient sound. In step (S420), the processor 120, for example, the audio processing unit 121, generates audio stream data based on the audio signal from the microphone 151. In the stage (S430), the processor 120, for example, the audio processing unit 121 temporarily stores the audio stream data generated in the stage (S120) in the memory 110.

In the stage (S440), the processor 120, for example, the keyword detection unit 122 detects a candidate keyword corresponding to the defined wake-up keyword from the audio stream data generated in the stage (S420). The wake-up keyword is a voice-based keyword capable of converting a voice control device in a sleep mode into a wake-up mode. For example, wake-up keywords are also voice keywords such as "clover" and "high computer".

In the example of FIG. 8A, the keyword detection unit 122 can detect the candidate keyword "clover" from the audio signal. In the example of FIG. 8B, the keyword detection unit 122 can detect the candidate keyword "clover", which is a word having a pronunciation similar to the keyword "clover", from the audio signal.

In the stage (S450), the processor 120, for example, the keyword detection unit 122 identifies the keyword detection section in which the candidate keyword is detected from the audio stream data, and determines the start point and the end point of the keyword detection section. The keyword detection section is the current section. The data corresponding to the current section in the audio stream data is regarded as the first audio data.

In the example of FIG. 8A, the keyword detection unit 122 can identify the section in which the candidate keyword "clover" is detected as the current section, and determine the start point and the end point of the current section. The audio data corresponding to the current section is referred to as the first audio data AD1. In the example of FIG. 8B, the keyword detection unit 122 can identify the section in which the candidate keyword "clover" is detected as the current section, and determine the start point and the end point of the current section. The audio data corresponding to the current section is referred to as the first audio data AD1.

Further, in the step (S450), the processor 120, for example, the keyword detection unit 122 determines whether the detected candidate keyword is a candidate keyword corresponding to which of the wake-up keyword and the single instruction keyword. Can be done. In the examples of FIGS. 8A and 8B, the keyword detection unit 122 can determine that the detected candidate keywords, that is, "clover" and "clover" are candidate keywords corresponding to the wake-up keywords.

In the stage (S460), the processor 120, for example, the speaker feature vector extraction unit 123 reads the second audio data corresponding to the previous section from the memory 110. The previous section is the section immediately before the current section, and the end point of the previous section is also the same as the start point of the current section. The speaker feature vector extraction unit 123 can also read the first audio data from the memory 110.

In the example of FIG. 8A, the speaker feature vector extraction unit 123 can read the second audio data AD2 corresponding to the previous section, which is the section immediately immediately before the current section, from the memory 110. In the example of FIG. 8B, the speaker feature vector extraction unit 123 can read the second audio data AD2 corresponding to the previous section, which is the section immediately immediately before the current section, from the memory 110. In the example of FIG. 8B, the second audio data AD2 corresponds to the voice ", four leaves". The length of the previous interval is also variably set by the detected candidate keyword.

In the stage (S470), the processor 120, for example, the speaker feature vector extraction unit 123 extracts the first speaker feature vector and the second speaker feature vector from the first audio data and the second audio data, respectively. The processor 120, for example, the wake-up determination unit 124 determines whether or not the wake-up keyword is included in the first audio data based on the similarity between the first speaker feature vector and the second speaker feature vector. You can judge. When the wake-up determination unit 124 determines that the first audio data includes the wake-up keyword, the wake-up determination unit 124 can wake up some components of the voice control device 100.

In the example of FIG. 8A, the first speaker feature vector corresponding to the first audio data AD1 is an index for identifying the speaker who utters the voice "clover". Since the second audio data AD2 is substantially silent, the second speaker feature vector can have a vector corresponding to the silent sound. Therefore, the degree of similarity between the first speaker feature vector and the second speaker feature vector is low.

As another example, when another person who is not the speaker who uttered the voice "clover" in the previous section utters the voice, the second speaker feature vector has a vector corresponding to the other person, so that the first The similarity between the speaker feature vector and the second speaker feature vector is low.

In the example of FIG. 8B, one person uttered, "How can I find a four-leaf clover?" Therefore, the first speaker feature vector extracted from the first audio data AD1 corresponding to the "clover" and the second speaker feature vector extracted from the second audio data AD2 corresponding to the "four-leaf" will be either. Is a vector for identifying speakers who are substantially the same, so that the degree of similarity between the first speaker feature vector and the second speaker feature vector is high.

In the stage (S480), the processor 120, for example, the wake-up determination unit 124 compares the similarity between the first speaker feature vector and the second speaker feature vector with the set reference value. When the similarity between the first speaker feature vector and the second speaker feature vector is higher than the set reference value, the wake-up determination unit 124 determines that the speaker of the first audio data in the current section and the second audio in the previous section. Since the speakers of the data are the same as each other, it is judged that the keyword is not included in the first audio data, and the wake-up cannot be proceeded. In that case, the process proceeds to step (S410), and the processor 120, for example, the audio processing unit 121 receives the audio signal corresponding to the ambient sound.

In the example of FIG. 8B, since one person uttered "four-leaf clover ...", the degree of similarity between the first speaker feature vector and the second speaker feature vector is high. In the example of FIG. 8B, the person who utters "four-leaf clover" determines that there is no intention to wake up the voice control device 100, and the wake-up determination unit 124 includes the wake-up keyword in the first audio data AD1. It is determined that this is not the case, and the voice control device 100 is not waked up.

When the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, the wake-up determination unit 124 is the speaker of the first audio data in the current section and the first in the previous section. Since the speakers of the two audio data are different from each other, it can be determined that the first audio data contains the keyword. In that case, the wake-up determination unit 124 can wake up some components of the voice control device 100. For example, the wake-up determination unit 124 can wake up the voice recognition unit 125.

In the example of FIG. 8A, the first speaker feature vector is a vector corresponding to the speaker who utters "clover", and the second speaker feature vector is a vector corresponding to the silent sound. The degree of similarity between the person feature vector and the second speaker feature vector is lower than the set reference value. In that case, the wake-up determination unit 124 can determine that the first audio data AD1 includes the wake-up keyword "clover". In that case, the voice recognition unit 125 is waked up to recognize the natural language voice command.

In the stage (S490), the processor 120, for example, the voice recognition unit 125 receives the third audio data corresponding to the next section after the current section from the audio processing unit 121. The next section is the section immediately following the current section, and the start point of the next section is also the same as the end point of the current section.

The voice recognition unit 125 can determine the end point of the next section when a silent sound having a preset length is detected in the third audio data. The voice recognition unit 125 can recognize the third audio data by voice. The voice recognition unit 125 can recognize the third audio data by voice by various methods. According to another example, in order to obtain the voice recognition result of the third audio data, the voice recognition unit 125 sends the third audio data to an external device, for example, the server 200 having the voice recognition function shown in FIG. Can be transmitted. The server 200 receives the third audio data and recognizes the third audio data by voice to generate a character string (text) corresponding to the third audio data, and the generated character string (text) is voiced. As a recognition result, it can be transmitted to the voice recognition unit 125.

In the example of FIG. 8A, the third audio data in the next section is a natural language voice command such as "tell me the weather tomorrow". The voice recognition unit 125 can directly recognize the third audio data and generate a voice recognition result, or can transmit the third audio data to the outside (for example, the server 200) so that the third audio data is recognized by voice. ..

In the stage (S500), the processor 120, for example, the functional unit 126 can perform a function corresponding to the voice recognition result of the third audio data. In the example of FIG. 8A, the functional unit 126 is also a voice information providing unit that searches for tomorrow's weather and provides a result, and the functional unit 126 searches for tomorrow's weather using the Internet and sends the result to the user. Can be provided. The functional unit 126 can also provide the search result of tomorrow's weather as voice using the speaker 152. The functional unit 126 responds to the voice recognition result of the third audio data and wakes up.

The embodiments according to the present invention described above are embodied in the form of a computer program executed on a computer via various components, and such a computer program is recorded on a computer-readable medium. .. At that time, the medium is also a continuous storage of a computer-executable program, or a temporary storage for execution or download. The medium is also a variety of recording or storage means in the form of a single piece or a combination of several pieces of hardware, but is not limited to a medium directly connected to a computer system, and is not limited to a network. It also exists in a distributed manner on top. Examples of such media are magnetic media such as hard disks, floppy discs and magnetic tapes; optical recording media such as CD-ROMs (compact disc read only memory) and DVDs (digital versatile discs); floptical discs. Includes magnetic and optical medium (disk); and ROM (read-only memory), RAM (random access memory), flash memory, etc., and is configured to store program command words. It is also a thing. Further, as an example of other media, a recording medium or a recording medium managed by an app store that distributes applications, a site that supplies or distributes various other software, a server, or the like can be mentioned.

As used herein, a "part", "module", etc. may also be a hardware component such as a processor or circuit, and / or a software component executed by a hardware configuration such as a processor. be. For example, "parts", "modules", etc. are components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, program code segments, etc. It is also embodied by drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays and variables.

The above description of the present invention is for illustration purposes only, and if a person skilled in the art to which the present invention belongs, other specific examples without changing the technical idea or essential features of the present invention. It can be understood that it can be easily transformed into a typical form. Therefore, it should be understood that the embodiments described above are exemplary in all respects and are not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, the components described as distributed may also be implemented in a combined form.

The scope of the present invention is shown by the scope of claims, rather than the above-mentioned detailed description, and the meaning and scope of the claims, and all modifications or variants derived from the concept of equality thereof, are described in the present invention. It must be construed as being included in the scope of the invention.

The voice control device for preventing erroneous keyword recognition and the operation method thereof according to the present invention can be effectively applied to, for example, a technical field related to voice recognition.

(Appendix 1)
An audio processing unit that receives audio signals corresponding to ambient sounds and generates audio stream data,
Keyword detection that detects a candidate keyword corresponding to a predetermined keyword from the audio stream data and determines the start point and the end point of the first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data. Department and
The first speaker feature vector related to the first audio data is extracted, and the second speaker feature related to the second audio data corresponding to the second section whose end point is the start point of the first section in the audio stream data. Speaker feature vector extractor that extracts the vector,
A voice including a wake-up determination unit for determining whether or not the keyword is included in the first audio data based on the degree of similarity between the first speaker feature vector and the second speaker feature vector. Control device.
(Appendix 2)
When the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, the wake-up determination unit includes the keyword in the first audio data. The voice control device according to Appendix 1, wherein the voice control device is characterized in that.
(Appendix 3)
Further including a keyword storage for storing a plurality of keywords including the predetermined keyword,
The voice control device according to Appendix 1, wherein each of the keywords is a wake-up keyword or a single command keyword.
(Appendix 4)
When the candidate keyword corresponding to the single command keyword is detected from the audio stream data by the keyword detection unit.
The speaker feature vector extraction unit receives the third audio data corresponding to the third section starting from the end point of the first section in the audio stream data, and the third speaker feature of the third audio data. Extract the vector,
The wake-up determination unit is based on the degree of similarity between the first speaker feature vector and the second speaker feature vector, and the similarity between the first speaker feature vector and the third speaker feature vector. The voice control device according to Appendix 3, wherein it is determined whether or not the single command keyword is included in the first audio data.
(Appendix 5)
In the wake-up determination unit, the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than a predetermined reference value, and the first speaker feature vector and the third speaker feature vector The voice control device according to Appendix 4, wherein it is determined that the first audio data includes the single command keyword when the degree of similarity with the above is equal to or less than a predetermined reference value.
(Appendix 6)
When the candidate keyword corresponding to the wake-up keyword is detected from the audio stream data by the keyword detection unit,
In response to the determination by the wakeup determination unit that the wakeup keyword is included in the first audio data, the wakeup is performed, and in the audio stream data, in the third section starting from the end point of the first section. Addendum, which further includes a voice recognition unit that receives the corresponding third audio data and recognizes the third audio data by voice, or transmits the third audio data to the outside so that the third audio data is recognized by voice. 3. The voice control device according to 3.
(Appendix 7)
The voice control device according to Appendix 6, wherein the second section is variably determined by the wake-up keyword.
(Appendix 8)
The speaker feature vector extraction unit
The first frame feature vector is extracted for each frame of the first audio data, the extracted first frame feature vector is normalized and averaged, and the first speaker feature vector representing the first audio data is expressed. Extract and
The second frame feature vector is extracted for each frame of the second audio data, the extracted second frame feature vector is normalized and averaged, and the second speaker feature vector representing the second audio data is expressed. The voice control device according to Appendix 1, wherein the voice control device is characterized in that.
(Appendix 9)
The keyword detection unit calculates a first probability of human voice and a second probability of background sound for each frame of the audio stream data, and the first probability is predetermined from the second probability. A frame higher than the standard value is determined as an audio frame, and
The speaker feature vector extraction unit
In the frame in the first audio data, the first frame feature vector is extracted for each of the frames determined to be the audio frame, the extracted first frame feature vector is normalized and averaged, and the first audio data is described. The first speaker feature vector representing the above is extracted,
In the frame in the second audio data, the second frame feature vector is extracted for each of the frames determined to be the audio frame, the extracted second frame feature vector is normalized and averaged, and the second audio data is described. The voice control device according to Appendix 1, wherein the second speaker feature vector representing the above is extracted.
(Appendix 10)
The voice control device according to Appendix 1, wherein the speaker feature vector extraction unit is waked up in response to detection of the candidate keyword by the keyword detection unit.
(Appendix 11)
At the stage of receiving the audio signal corresponding to the ambient sound and generating the audio stream data,
A step of detecting a candidate keyword corresponding to a predetermined keyword from the audio stream data and determining a start point and an end point of a first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data. ,
The stage of extracting the first speaker feature vector related to the first audio data, and
In the audio stream data, a step of extracting a second speaker feature vector related to the second audio data corresponding to the second section whose end point is the start point of the first section, and
Based on the degree of similarity between the first speaker feature vector and the second speaker feature vector, it is determined whether or not the keyword is included in the first audio data, and whether or not to wake up is determined. And how the voice control device operates, including.
(Appendix 12)
The stage of deciding whether or not to wake up is
A step of comparing the degree of similarity between the first speaker feature vector and the second speaker feature vector with a predetermined reference value, and
When the degree of similarity is equal to or less than the predetermined reference value, it is determined that the keyword is included in the first audio data and wakes up.
The voice according to Appendix 11, characterized in that, when the similarity exceeds the predetermined reference value, the first audio data includes a step of determining that the keyword is not included and not wake-up. How to operate the control device.
(Appendix 13)
When the detected candidate keyword is the candidate keyword corresponding to the single instruction keyword,
In the audio stream data, the stage of receiving the third audio data corresponding to the third section starting from the end point of the first section, and
At the stage of extracting the third speaker feature vector of the third audio data,
The degree of similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than a predetermined reference value, and the degree of similarity between the first speaker feature vector and the third speaker feature vector is The method of operating the voice control device according to Appendix 11, wherein when the value is equal to or less than a predetermined reference value, the first audio data further includes a step of determining that the single command keyword is included. ..
(Appendix 14)
The voice control device according to Appendix 13, further comprising a step of performing a function corresponding to the single command keyword in response to a determination that the first audio data includes the single command keyword. How it works.
(Appendix 15)
When the detected keyword is the candidate keyword corresponding to the wake-up keyword,
In response to the determination that the wake-up keyword is included in the first audio data, the audio stream data receives the third audio data corresponding to the third section starting from the end point of the first section. And the stage to do
The method of operating the voice control device according to Appendix 11, further comprising a step of recognizing the third audio data by voice or transmitting the third audio data to the outside so that the third audio data is recognized by voice. ..
(Appendix 16)
The stage of extracting the first speaker feature vector and the second speaker feature vector is
The stage of extracting the first frame feature vector for each frame of the first audio data, and
A step of normalizing and averaging the extracted first frame feature vector and extracting the first speaker feature vector representing the first audio data.
The stage of extracting the second frame feature vector for each frame of the second audio data, and
The voice according to Appendix 11, which comprises a step of normalizing and averaging the extracted second frame feature vector and extracting the second speaker feature vector representing the second audio data. How to operate the control device.
(Appendix 17)
For each frame of the audio stream data, a first probability of human voice and a second probability of background sound are calculated, and the first probability is higher than the second probability by exceeding a predetermined reference value. Including the step of deciding the frame as an audio frame,
The stage of extracting the first speaker feature vector and the second speaker feature vector is
In the frame in the first audio data, the step of extracting the first frame feature vector for each of the frames determined to be the audio frame, and
A step of normalizing and averaging the extracted first frame feature vector and extracting the first speaker feature vector representing the first audio data.
In the frame in the second audio data, the step of extracting the second frame feature vector for each of the frames determined to be the audio frame, and
The voice according to Appendix 11, which comprises a step of normalizing and averaging the extracted second frame feature vector and extracting the second speaker feature vector representing the second audio data. How to operate the control device.
(Appendix 18)
A computer program including a command word that causes a processor of a voice control device to execute the operation method according to any one of Supplementary note 11 to 17.
(Appendix 19)
A recording medium on which the computer program described in Appendix 18 is recorded.

100 Voice control device (electronic device)
110 Memory 120 Processor 121 Audio processing unit 122 Keyword detection unit 123 Speaker feature Vector extraction unit 124 Wake-up judgment unit 125 Speech recognition unit 126 Function unit

Claims (9)

An audio processing unit that receives audio signals corresponding to ambient sounds and generates audio stream data,
A keyword detection unit that detects a candidate keyword corresponding to a predetermined keyword from the audio stream data and determines a first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data.
In the audio stream data, the second audio data corresponding to the second section, which is the previous section of the first section, is determined, and the first speaker feature vector related to the first audio data and the second audio data are used. A speaker feature vector extraction unit that extracts the related second speaker feature vector,
A voice control device including a wake-up determination unit for determining whether or not the keyword is included in the first audio data based on the degree of similarity between the first speaker feature vector and the second speaker feature vector. .. When the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, the wake-up determination unit includes the keyword in the first audio data. The voice control device according to claim 1, wherein the voice control device is characterized in that. When the candidate keyword corresponding to the single command keyword is detected from the audio stream data by the keyword detection unit,
The speaker feature vector extraction unit extracts a third speaker feature vector related to the third audio data corresponding to the third section, which is the section next to the first section, from the audio stream data.
The wake-up determination unit is based on the degree of similarity between the first speaker feature vector and the second speaker feature vector, and the similarity between the first speaker feature vector and the third speaker feature vector. The voice control device according to claim 1, wherein it is determined whether or not the single command keyword is included in the first audio data. In the wake-up determination unit, the similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, and the first speaker feature vector and the third speaker feature vector The voice control device according to claim 3, wherein it is determined that the first audio data includes the single command keyword when the degree of similarity with the first audio data is equal to or less than the set reference value. When the candidate keyword corresponding to the wake-up keyword is detected from the audio stream data by the keyword detection unit,
In response to the determination by the wakeup determination unit that the wakeup keyword is included in the first audio data, the wakeup is performed, and in the audio stream data, the third section, which is the next section after the first section, is used. The voice control device according to claim 1, further comprising a voice recognition unit that recognizes the corresponding third audio data by voice or transmits the third audio data to the outside so that the third audio data is recognized by voice. At the stage of receiving the audio signal corresponding to the ambient sound and generating the audio stream data,
A step of detecting a candidate keyword corresponding to a predetermined keyword from the audio stream data and determining a first section corresponding to the first audio data in which the candidate keyword is detected in the audio stream data.
The stage of extracting the first speaker feature vector related to the first audio data, and
In the audio stream data, a step of extracting a second speaker feature vector related to the second audio data corresponding to the second section which is the previous section of the first section, and
Based on the degree of similarity between the first speaker feature vector and the second speaker feature vector, it is determined whether or not the keyword is included in the first audio data, and whether or not to wake up is determined. And how the voice control device operates, including. When the detected candidate keyword corresponds to a single instruction keyword,
In the audio stream data, a step of extracting a third speaker feature vector related to the third audio data corresponding to the third section, which is the next section of the first section, and
The degree of similarity between the first speaker feature vector and the second speaker feature vector is equal to or less than the set reference value, and the degree of similarity between the first speaker feature vector and the third speaker feature vector is The operation of the voice control device according to claim 6, wherein when the value is equal to or less than the set reference value, the first audio data further includes a step of determining that the single command keyword is included. Method. When the detected keyword is the candidate keyword corresponding to the wake-up keyword,
In response to the determination that the wake-up keyword is included in the first audio data, the third audio data corresponding to the third section, which is the next section of the first section, is received in the audio stream data. And the stage to do
The operation of the voice control device according to claim 6, further comprising a step of recognizing the third audio data by voice or transmitting the third audio data to the outside so that the third audio data is recognized by voice. Method. A computer program including a command word that causes a processor of a voice control device to execute the operation method according to any one of claims 6 to 8.

Images