KR100922963B1 - User speech recognition apparatus using microphone array and method for driving microphone array - Google Patents

Abstract

The present invention relates to a voice recognition device, and more particularly, to a microphone array device and a microphone array device for forming a beam pattern having a high directivity in the direction of input of a voice signal in a voice recognition device using a microphone array. A method for driving a microphone array. To this end, the present invention is equipped with an end-fired microphone array to be embedded in the voice recognition device, and then, if the user is recognized, after projecting the microphone array, the microphone array to suppress the noise of the surrounding environment and receive a more accurate voice signal To help.

Endfire, Broadside, Beam Pattern, Robot

Description

Apparatus for recognizing user's voice using microphone array and method of driving the microphone array}

1 is a view for explaining a gain (Gain) that varies depending on the direction of the input signal input to the microphone array in general,

2 is a view for explaining beamforming formed by a broadside method and an endfire method;

3 is a view illustrating a beam pattern formed by a broadside method and an endfire method;

4 illustrates a robot generally equipped with a broadside microphone array;

5 is an internal configuration diagram of a voice recognition device equipped with an endfire type microphone array according to an embodiment of the present invention;

6 is a flowchart illustrating a process for driving a microphone array in a voice recognition device equipped with an endfire type microphone array according to an embodiment of the present invention;

7 is a diagram illustrating an example in which a microphone array is mounted to a robot in an endfire form according to an embodiment of the present invention.

The present invention relates to a voice recognition device, and more particularly, to a microphone array device and a microphone array device for forming a beam pattern having a high directivity in the direction of input of a voice signal in a voice recognition device using a microphone array. A method for driving a microphone array.

In general, a microphone is a transducer that converts an acoustic signal transmitted by air vibration into an electrical signal. Recently, with the development of the technology for robot control, a microphone is used as a voice interface of a robot as a means of freely communicating a robot and a user. The robot may recognize the user's voice by converting the voice signal input through the microphone, which is the voice interface of the robot, into an electrical signal and analyzing the contents.

When the robot receives the user's specific voice signal, if the position of the microphone forms a directivity toward the direction in which the voice signal is input, the input of noise generated in the surrounding environment may be excluded. In this case, even one microphone having high directivity may form directivity toward a direction in which a specific voice signal is input. However, when a microphone array is formed by arranging a plurality of microphones rather than a single microphone, it is possible to have an advantage of freely obtaining a directional characteristic of a form suitable for a purpose of use.

On the other hand, if the software process for filtering the voice signal input through the microphone array is performed to form a beam from the microphone array in a specific direction in accordance with the software processing. In addition, a beamforming technique is used for the purpose of forming a beam using the microphone array so as to exhibit high directivity from the microphone in a desired direction.

When the high directivity is formed in the direction in which the user's voice is input through such beamforming, the voice signal input from the directions outside the beam is automatically attenuated, and the voice signal input from the direction of interest may be selectively acquired. Can be. Microphone arrays can use this beamforming technology to suppress part of echo waves reflected from objects such as furniture and walls and computer noise in the room, ambient noise such as TV sounds, and furniture. That is, the microphone array can use beamforming technology to obtain higher signal to noise ratio (SNR) for sound signals originating from the beam in the direction of interest. Therefore, beamforming plays an important role in spatial filtering, which points the "beam" to the sound source and suppresses all signals coming from different directions.

In general, the beam pattern formed through the microphone array is calculated as a gain obtained according to the direction of an input signal input to the array, and the gain thus calculated may be represented as shown in FIG. 1. In the microphone array, the gain is highest near 90 ° to the front and the gain in the other direction is significantly lower. In FIG. 1, a beam formed in a direction in which the gain of the input signal is greatest is called a mainlobe, and a beam formed in a low gain in other directions is called a sidelobe. The beamforming aims to keep the main lobe of the direction in which the audio signal is to be obtained to have the maximum gain, and to minimize the gain of the side lobe. And such beamforming characteristics show more effective performance as they appear consistently in all frequency domains.

In general, in a device having a microphone array, the microphone array uses a uniform arrangement method, which is arranged in a single space at regular intervals, or a method in which the microphone array is lined up to face a sound source. The technology related to the microphone array beam forming configured in the above form is described in Korean Patent Nos. 2001-7005783, 2003-0014006, and 2004-0013029.

The sound source is input to the microphone array arranged in a line form, and may be classified into a broadside method and an endfire method according to the arrangement position of the microphone array. Referring to FIG. 2, the difference between the broadside method and the endfire method is shown. The broadside method is a method in which the input signal position is located in the vertical direction of the direction in which the microphone array is arranged. In addition, the endfire method is a method in which the input signal position is located on an extension line in the direction in which the microphone array is arranged. In this case, if the input signal consists of only one frequency component, as shown in FIG. 1, the beam pattern can be perfectly represented in one two-dimensional plane. However, since the frequency band of the actual input signal is wide, it is necessary to express and analyze the beam pattern formed at each frequency in detail.

Meanwhile, the beam formed in the broadside method and the endfire method according to the position of the microphone array and the input signal may be illustrated in FIG. 3. Referring to FIG. 3, signals input to the microphone array are classified and expressed in detail from 1 Hz to 4000 Hz. In FIG. 3, it can be seen that the main lobe is formed in a dark direction. 3 is a result of calculating the input signal by applying the MVDR algorithm, that is, the formed beam pattern. Here, since the MVDR algorithm is used for general beam pattern recognition, a detailed description of this algorithm will be omitted in the present invention.

Referring to FIG. 3, the results of the above two schemes are compared, and (a) is a beam pattern formed when the input signal is input to the microphone array in a broadside manner. Further, (b) is a beam pattern formed when the input signal is input to the microphone array in an endside manner. Referring to (a), it can be seen that the main lobe is formed near 90 ° in which the voice signal is input. However, it can be seen that the gain characteristics are weak due to the wide width of the formed main lobe. In particular, in the low frequency band, the gain value is larger than 0.5 in all directions, so the width of the main lobe becomes very wide, and receives input signals in almost all directions to the microphone array. This greatly interferes with the normal functioning of the beamforming.

As described above, the microphone array is mounted in the conventional voice recognition apparatus using a method of arranging the microphone array in a broadside direction perpendicular to the input signal as shown in FIG. 4. As such, the beams formed by the microphone arrays arranged in the broadside direction have a weak directivity, and have a low degree of suppressing a signal coming out of the direction of the sound source. In addition, since the microphones are arranged at equal intervals, even if the angle of incidence is the same, in the low frequency band, the length of the wavelength of the input signal is much longer than the interval between the microphones, which greatly weakens the gain characteristic of the main lobe.

Accordingly, the present invention relates to a voice recognition device having a microphone array for forming a beam pattern having a high directivity in the input direction of a voice signal, and a method for driving the microphone array.

In addition, the present invention is a microphone array device and a microphone array device for suppressing the noise generated in the surrounding environment in the indoor environment in the device for recognizing the voice using the microphone array, and more accurately recognize the user's voice signal A method for driving a microphone array in.

According to an aspect of the present invention, a voice recognition device for recognizing a voice using a microphone array includes at least two microphones and an end fire in the voice recognition device to receive a voice input in an end fire direction. A microphone array mounted in a form, a microphone array embedding unit for embedding the microphone array, a microphone array driving unit for protruding or embedding the microphone array from the microphone array embedding unit according to a control signal for driving the microphone array; And a control unit for outputting a control signal for projecting the microphone array to the microphone array embedded unit when the user is recognized.

In addition, the present invention is a method for driving the microphone array in a speech recognition apparatus for recognizing speech using a microphone array, comprising at least two microphones, so that the voice input can be received in the end fire direction Providing a microphone array mounted in an endfire form in the recognition device and a microphone array embedding unit for embedding the microphone array, checking whether the user is recognized, and protruding the microphone array from the microphone array embedding unit if the user is recognized. And protruding the microphone array from the microphone array internal part, and then embedding the microphone array into the microphone array internal part if there is no voice signal input for a predetermined time. It characterized by more than true.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the same elements in the figures are represented by the same numerals wherever possible. In addition, detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention will be omitted.

The present invention configures the microphone array in the form of an end fire in the voice recognition device using the microphone array so that a sharp beam is formed in the direction of the voice signal input by the user. The microphone array is mounted in such a way that a sharp beam is formed by concentrating in the direction in which the sound source is provided to the voice recognition device, thereby providing a method for suppressing the surrounding noise signal and recognizing the user's voice signal more accurately.

Then, a voice recognition apparatus equipped with a microphone array in the form of an end fire to form a sharp beam in a direction in which a sound source is provided according to an embodiment of the present invention will be described with reference to FIG. 5.

The controller 500 controls the overall operation of the voice recognition device and performs the beamforming operation through the beamformer 508 for spatially filtering the input signal input through the microphone array 504, and then recognizes the voice and recognizes the user. Perform the same operation. In particular, when an event in which a sound source is provided according to the present invention occurs, the microphone array driver 506 is controlled to control the microphone array 504 to automatically protrude. In addition, when the sound source is not input for a predetermined time, the microphone array driver 506 is controlled to allow the projected microphone array 504 to be embedded in the microphone array internal part 502.

The memory unit 510 stores a program for voice recognition, and stores data generated in the voice recognition operation.

The microphone array 504 is embedded in the microphone array embedding 502, and the microphone array 504 in the present invention is composed of at least two microphones. In addition, the microphone array 504 is embedded in or protrudes from the microphone array internal part 502 according to the driving signal of the microphone array driver 506. In addition, the microphone array 504 according to the present invention is mounted on the voice recognition device in the form of an end fire so that the beam can be sharply formed in the direction of the input signal. In the present invention, by mounting the microphone array 504 in the form of an end fire in the speech recognition apparatus as described above, it is possible to have a more improved directivity when filtering the input signal through the microphone array 504. That is, the microphone array 504 is configured to suppress noise generated from the surroundings in the indoor environment and to recognize the voice signal of the user more clearly.

Then, the case in which the microphone array 504 is mounted on the voice recognition device in the form of an end fire will be described with reference to FIG. 7. FIG. 7 illustrates an example in which the microphone array 504 is mounted on the robot in the form of an end fire in the case of a robot among voice recognition devices. At this time, the microphone array 504 is ideally mounted at the same height as the position where the user's voice is spoken. In general, since the height of the robot is smaller than the height of the human, the microphone array 504 is designed to look upward in the direction in which the user is positioned as an example. By installing the array in the form of an end fire on the top of the robot as shown in Figure 7, it is necessary to adjust the angle so that the formed beam can be directed in the direction of the user as much as possible. In addition, if the microphone array 504 mounted on the top of the robot is continuously maintained in a state that protrudes outside, there may be a risk of damage to the external shock. In addition, when the microphone array 504 is not used, it may be a factor that impedes movement.

Therefore, the present invention includes a microphone array embedding unit 502, which is a space in which the microphone array 504 is embedded in the robot. It also includes a microphone array driver 506, which is a small electric motor for embedding and projecting the microphone array 504 in the microphone array internal portion 502. As such, when the microphone array 504 is not used in the voice recognition device including the microphone array embedding unit 502 and the microphone array driving unit 506, the microphone array 504 is used in the microphone array embedding unit 502 of the robot. ). In addition, the microphone array 504 is automatically protruded to the outside only when the input of the voice signal is required. In this case, the case in which the input of the voice signal is required through face recognition, color recognition, key recognition, or sound source recognition may be automatically recognized by the voice recognition device. If the user is recognized based on a video signal such as face recognition clothes color recognition or key recognition, the microphone is installed after the camera is installed in the voice recognition device and the user is recognized in front of the voice recognition device. The array can be operated to protrude. In addition, a touch sensor or a remote controller may be used to protrude the microphone array to recognize that the user needs to input a voice signal directly to the voice recognition device.

The beamformer 508 spatially filters the input signal input from the microphone array 504. Thereafter, the signal output through the beamformer 508 may be used in a recognition operation such as voice recognition or user recognition, or may be connected to an additional noise removing algorithm.

In addition, when the microphone array is mounted on the voice recognition device in the form of an end fire as described above, the beam pattern formed through the beamformer 508 for the input sound source may be represented as shown in FIG. Referring to FIG. 3, when the sound source is input in the endfire direction as shown in (b) and the sound source is input in the broadsite direction as shown in (a), the method of inputting the sound source in the endfire direction is main. The difference in lobe and sidelobe gains is greater. In addition, the way in which the sound source is input in the end fire direction maintains the width of the main lobe almost uniformly in all frequency bands from the low frequency to the high frequency band. That is, when the sound source is input in the end fire method it can be seen that the directivity to the voice signal is higher. Here, as the number of microphones constituting the microphone array 504 increases, the width of the main lobe becomes narrower and the gain of the side lobe becomes lower. Accordingly, the volume of the device to which the microphone array 504 is to be mounted and the number of microphones suitable for the purpose of the desired performance can be adjusted and applied.

Then, when the voice signal input through the microphone array mounted in the form of an end fire in the voice recognition device according to an embodiment of the present invention is input to the beam forming unit 508, the beam pattern formed by the beam forming unit 508 This will be described with reference to FIG. The beam pattern of FIG. 3B configures a superdirective beamforming unit 508 using a Minimum Variance Distortionless Response (MVDR) algorithm. At this time, the beam pattern to be formed is not a physically formed beam, the input signal is formed through the equation for forming the beam pattern. Therefore, the equation for constructing the beam pattern using the MVDR algorithm according to an embodiment of the present invention can be obtained through the following development process.

은 하기의 <수학식 1>과 같이 도출될 수 있다.When the voice signal input in the time domain is converted into the frequency domain and input to the beamformer 508, the output value of the beamformer

May be derived as in Equation 1 below.

는 잡음 환경의 형태에 따라 달라지며, MVDR 알고리즘을 이용하여 최적의 값을 얻을 수 있다. 그 수식은 하기의 <수학식 2>와 같다.At this time, the coefficient of the filter

Depends on the shape of the noise environment, and the optimal value can be obtained using the MVDR algorithm. The formula is shown in Equation 2 below.

은 잡음 신호의 N*N 크기의 전력 스펙트럼 밀도(PSD) 행렬이며, d는 마이크로폰이 음원을 향하도록 방향을 결정지어 주는 벡터이다. 로봇에서 사용할 선형(linear) 어레이에서는 d를 하기의 <수학식 3>과 같다.From here

Is the N * N magnitude power spectral density (PSD) matrix of the noise signal, and d is a vector that directs the microphones toward the sound source. In the linear array to be used in the robot, d is represented by Equation 3 below.

이며, c는 음속 ,

는 마이크로폰 어레이가 음원을 향하는 각도이다. 이때, 엔드파이어의 경우에는

=0이고, 브로드사이드의 경우에는

이다. Specifically, the nth corresponding value is

C is the speed of sound,

Is the angle at which the microphone array is facing the sound source. At this time, in the case of end fire

= 0 and for Broadside

to be.

를 가진다.Also, if the noise signal is stationary in the indoor space and is assumed to be homogeneous, then the noise is a power spectral density.

Has

의 복소 코히런스(coherence)는 하기의 <수학식 4>와 같이 정의할 수 있다.Two signals

The complex coherence of may be defined as in Equation 4 below.

The filter coefficients in Equation 4 may be expressed as Equation 5 below using coherence.

,

,

In general, in the indoor noise environment, the waveform is reflected and spread due to obstacles such as walls and furniture. Because of this, the power of the waveform input to the microphone at every location in the space is considered to be constant, which is called a diffuse environment. In this diffuse environment, coherence is modeled as in Equation 6 below.

는 두 마이크로폰 i와 j 사이의 간격을 말한다.From here

Is the distance between two microphones i and j.

Using the above Equations 5 and 6, the beam formed according to the frequency of the input signal input from the end fire direction can improve the directivity of filtering the voice signal as shown in FIG. 3. Able to know.

Of course, an example of using the MVDR algorithm when configuring the beam pattern according to an embodiment of the present invention has been described. However, since the MVDR algorithm is one of the algorithms constituting the beam pattern, an algorithm for configuring another beam pattern may be used.

Next, a process for driving the microphone array 504 in the voice recognition apparatus equipped with the endfire type microphone array as shown in FIG. 5 will be described with reference to FIG. 6.

First, in step 600, the control unit 500 of the voice recognition device maintains the standby state, and if the voice service is set in step 602, the process proceeds to step 604. In this case, the voice service is a service for extracting a voice signal after filtering the noise signal from the input signal input by the voice recognition device, and recognizes the extracted voice signal as a specific command to perform an operation corresponding to the command.

If the voice service is set in step 602, the controller 500 proceeds to step 604 and the controller 500 checks whether an automatic mode for automatically recognizing a user is set or a manual mode for manually setting.

If the manual mode is set, the controller 500 proceeds to step 618 to determine whether an event for user recognition has occurred. In this case, the event for user recognition refers to an event in which a user inputs a command to prepare a voice recognition device to the voice recognition device because the user will soon input a voice to the voice recognition device. For example, a user may recognize that a user needs to input a voice signal directly to the voice recognition device by using a remote controller that is simply designed using a touch sensor or an infrared module. If an event for user recognition occurs in step 618, the process proceeds to step 608.

If the automatic mode is set in step 604, the controller 500 proceeds to step 606 to recognize the sound source direction through sound source tracking. In this case, since the technology of recognizing the sound source direction through the sound source tracking of the control unit 500 of the voice recognition apparatus is a general technology, description of specific technical contents will be omitted in the detailed description of the present invention.

In operation 618 and operation 606, the control unit 500 outputs a control signal for driving the microphone array 504 to protrude to the microphone array driver 506. Then, the microphone array driver 506 receiving the control signal for driving to protrude protrudes the microphone array 504 from the microphone array internal part 502 in step 608. In operation 610, the controller 500 acquires a voice signal input through the microphone array 504 and outputs the voice signal to the beamformer 508 to control the beamforming operation in the beamformer 508.

Thereafter, in step 612, the controller 500 checks whether there is no voice signal input for a predetermined time. At this time, the controller 500 determines that there is no voice input when there is no voice input for a predetermined time stored in the memory unit 510.

If it is determined in step 612 that there is no voice signal input for a preset time, the controller 500 supplies a driving control signal for driving the microphone array 504 to be embedded in the microphone array embedding unit 520. Output at 506. Then, the microphone array driver 506 automatically embeds the microphone array 504 protruding in the step 614 into the microphone array embedding unit 502. That is, in this way, the projected microphone array 504 is embedded in the microphone array embedding unit 502 of the voice recognition device in the absence of a voice input.

Thereafter, if the process proceeds from step 614 to 616, the control unit 500 checks whether there is a request for terminating the voice service, and terminates if there is a request for terminating the voice service, otherwise proceeds to step 604 to continue the voice service. Perform the operation for.

As described above, the present invention mounts the end-fired microphone array to be embedded in the voice recognition device, and then, if the user is recognized, protrudes the microphone array, thereby suppressing the noise of the surrounding environment through the microphone array and more accurately correcting the voice signal. Allow input. The input voice signal may perform a voice interface function of various methods such as voice recognition and speaker recognition. Meanwhile, in the embodiment of the present invention, an example of a voice recognition device has been described as an example of a robot, but in the voice recognition device that recognizes voice using all the microphone arrays, the microphone array may be mounted in the form of an end fire as described above. . In addition, the microphone array may be driven in the voice recognition apparatus equipped with the endfire type microphone array as shown in FIG. 6.

As described above, the voice recognition device of the present invention includes an end-fired microphone array to form a beam pattern having high directivity in the input direction of the voice signal. As such, the beam formed through the microphone array provided in the form of an end fire may have direction in the voice direction of the user more accurately than the conventional broadside method. That is, there is an advantage that can suppress more noise coming from a place other than the direction of the user's voice.

Claims (12)

In the user speech recognition device having a microphone array device, A microphone array unit comprising at least two microphones and arranged to receive a voice input in an end fire direction; A microphone array built-in unit for embedding the microphone array unit; A microphone array driver configured to protrude or embed the microphone array unit from the microphone array embedded unit according to a control signal for driving the microphone array; Generate a control signal for protruding the microphone array unit in the direction of the user, and transmits the generated control signal to the array driver to protrude the microphone array unit in the direction of the user, and receives a voice signal from the protruding direction And a control unit for outputting a control signal for embedding the microphone array unit into the microphone array driving unit to the microphone array driving unit if there is no voice signal input for a preset time after controlling the microphone array unit. The microphone of claim 1, wherein the controller recognizes a sound source direction through sound source tracking when the mode for recognizing a user voice is set to an automatic mode, and outputs a control signal for protruding the microphone array unit in the sound source direction. And a user speech recognition apparatus output to the array driver. The apparatus of claim 1, wherein the controller outputs a control signal for protruding the microphone array unit to the microphone array driver when an event for user voice recognition occurs when a mode for recognizing a user voice is set to a manual mode. User speech recognition device, characterized in that. The apparatus of claim 3, wherein the event for recognizing the user's voice is an event for notifying that the user inputs a voice signal directly to the user's voice recognition device. delete A method for driving the microphone array device in a user speech recognition device having a microphone array device, the method comprising: Generating a driving signal composed of at least two microphones and driving a microphone array unit arranged to receive a voice input in an end fire direction; Providing the generated driving signal to the microphone array to protrude the microphone array; And after the protrusion of the microphone array unit, embedding the microphone array unit if there is no voice signal input for a predetermined time. The method of claim 6, Checking whether the mode for recognizing the user's voice is set to automatic mode or manual mode; And recognizing the user's voice through sound source tracking if the mode for recognizing the user's voice is set to the automatic mode in the checking step. The method of claim 7, wherein And if the mode for recognizing the user's voice is set to the manual mode in the checking step, recognizing the user's voice according to the occurrence of an event for recognizing the user's voice. Way. The method of claim 8, wherein the event for recognizing a voice of the user is an event indicating that a voice signal is input directly from the user. delete delete delete

Images