JP4517838B2 - Audio processing device - Google Patents

Description

  The present invention relates to a technique for recognizing human speech.

  When practicing pronunciation in language learning, for example, a learning method is used in which a model voice recorded on a recording medium such as a CD (Compact Disk) is reproduced and pronounced by imitating the model voice. When performing this learning, in order to advance learning more effectively, it is necessary to objectively evaluate the difference between the model voice and one's own voice. By listening to the model voice recorded on the CD, the imitation can be imitated. There is a problem that it is difficult to know in detail whether your pronunciation is correct just by doing.

  As a technique for specifically recognizing correctness of his / her pronunciation, there is a technique disclosed in Patent Document 1, for example. In Patent Document 1, nasal sounds, unvoiced sounds, voiced sounds, friction sounds, and the like emitted by humans are detected by a microphone provided for each sound, and the level of each sound is displayed by the number of lighted LEDs (Light Emitting Diodes). A pronunciation practice device is disclosed. According to this apparatus, the level of each sound that is normally invisible is visualized, and it can be specifically grasped whether or not the sound is correct.

As a technique for recognizing sound, there is a technique disclosed in Patent Document 2, for example. The device disclosed in Patent Document 2 includes a microphone array on the focal plane of a parabolic reflector, and detects a sound pressure distribution in front of the parabolic reflector based on an output signal from each microphone constituting the microphone array. . Then, the measured sound pressure distribution is compared with the correct sound pressure distribution stored in advance, and the abnormality of the sound is determined based on the difference.
JP-A-11-352876 JP-A-6-113387

  By the way, in the pronunciation training device disclosed in Patent Document 1, in order to correctly detect the voice uttered by the user, the housing containing each microphone is brought into close contact with the face so as to cover the nose and mouth, It is necessary to position a microphone provided for each sound in the immediate vicinity of the nose and lips. However, if you cover the nose or mouth in this way, some users may feel inconvenient or uncomfortable, and the device will touch the face, so when used by an unspecified number of people Some people feel discomfort from the hygiene side. And if a user does not position correctly from discomfort, the problem that a sound cannot be recognized correctly will arise. Further, the apparatus disclosed in Patent Document 2 has a problem that the apparatus becomes large because a parabolic antenna is used, and the problem of inconvenience of using the apparatus also occurs here.

  The present invention has been made under the above-described background, and an object of the present invention is to provide a technique that enables a user to recognize a voice without causing inconvenience or discomfort to the user.

In order to solve the above-described problems, the present invention provides a storage means for storing a sound pressure level distribution of standard language sounds and a frequency spectrum of standard language sounds, a microphone unit in which a plurality of microphones are arranged in an array, For each signal output from each microphone of the microphone unit, sound pressure level detection means for detecting the sound pressure level of the sound represented by the signal, and the frequency spectrum of the signal output from the predetermined microphone of the microphone unit are detected. Spectrum detection means, sound pressure distribution detection means for obtaining a sound pressure level distribution on a surface of the microphone unit on which the microphone is disposed, from a plurality of sound pressure levels obtained by the sound pressure level detection means, and the sound The sound pressure level distribution obtained by the pressure distribution detection means and the spectrum detection means A speech discriminating means for comparing the obtained frequency spectrum with the sound pressure level distribution of the standard language sound and the frequency spectrum of the standard language sound stored in the storage means, and discriminating the language sound input to the microphone unit; And a voice processing apparatus having output means for outputting the discrimination result of the voice discrimination means.
In the present invention, a user speaks toward a microphone unit in which microphones are arranged in an array. Sound is input to a plurality of microphones, and a sound pressure level distribution and a frequency spectrum at the time of sound generation are detected from signals output from the plurality of microphones arranged in an array. Since the voice uttered by the user is discriminated from the sound pressure level distribution and the frequency spectrum, the voice can be discriminated with high accuracy, and the user does not need to bring the microphone unit into close contact with the user.

In a preferred aspect of the present invention, the sound processing device has a light emitter associated with each of the plurality of microphones, and is based on a sound pressure level corresponding to each microphone detected by the sound pressure level detecting means. And a control means for lighting a light emitter associated with each microphone. In another preferred embodiment, the control means may vary the illuminance of the light emitter according to the sound pressure level.
In another preferable aspect, the sound processing apparatus includes a display unit that generates a sound pressure distribution image indicating the sound pressure level distribution obtained by the sound pressure distribution detection unit and displays the sound pressure distribution image. . In another preferred embodiment, the speech processing apparatus includes speech request means for requesting the user to speak a language sound by at least either an image or sound, the language sound determined by the sound determination means, and the sound Judgment means for judging whether or not the speech sound requested by the request means matches, and the output means outputs a judgment result of the judgment means.

  According to the present invention, the user can easily recognize the pronunciation without feeling uncomfortable.

[Constitution]
FIG. 1 is a diagram showing a configuration of a sound processing apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the sound processing device 1 is roughly divided into a microphone unit 100 and a control device 200.

  The microphone unit 100 includes a stand 140 and a substrate 130 attached to the stand 140 as shown in FIG. On the substrate 130, rectangular silicon microphones 110A-1, 110A-2,..., 110A-16 to silicon microphones 110P-1, 110P-2,. As shown in FIG. 2, 16 LEDs 16A-1 and 120A-2 are connected to the control unit 210 between the vertical directions of the silicon microphones. ,..., 120A-16 to LEDs 120P-1, 120P-2,. Each silicon microphone disposed on the substrate 130 converts the input sound into an electrical signal and outputs it, and each LED is turned on / off under the control of the control unit 210. Since the respective silicon microphones have the same configuration, the individual silicon microphones are hereinafter referred to as silicon microphones 110 when it is not necessary to distinguish them. For the same reason, the LED is hereinafter referred to as an LED 120 when it is not necessary to distinguish the individual LEDs. In addition, the number of silicon microphones 110 and LEDs 120 disposed is not limited to the number described above, and may be other numbers.

  The audio input unit 220 functions as an interface for receiving the electric signals output from the respective silicon microphones 110, outputs all the input electric signals to the sound pressure level detection unit 230, and outputs the electric signals output from the predetermined silicon microphones. The signal is output to the voice detection unit 240. The sound pressure level detection unit 230 calculates the sound pressure level of the sound input to each silicon microphone from each electrical signal output from the sound input unit 220 (that is, for each electrical signal output from each silicon microphone). Then, sound pressure data indicating the calculated sound pressure level is output to the CPU 211 of the control unit 210. The voice detection unit 240 samples the input electrical signal and stores it as digital data, performs frequency analysis by fast Fourier transform from this data, and obtains the spectrum of the voice represented by the electrical signal. Next, the voice detection unit 240 obtains a formant frequency from this spectrum, compares it with the formant frequency stored for each kind of voice in advance, and determines what kind of voice is pronounced by a technique such as pattern matching. . Then, the sound detection unit 240 generates sound generation data indicating the determined sound, and outputs the sound generation data to the control unit 210.

  The control unit 210 includes a CPU 211, ROM 212, RAM 213, and HDD (Hard Disk Drive) 214. The CPU 211 reads out and executes a program stored in the ROM 212 or a program stored in the HDD 214 and controls each unit of the sound processing device 1. The HDD 214 is a storage device for storing various application programs and data, and is a pronunciation practice program for realizing an application for practicing pronunciation, and data used in this application. Voice data to be represented, sound pressure distribution data indicating a sound pressure distribution when a sound such as a vowel or a consonant is sounded, threshold data for determining whether or not the LED 120 is lit, and the like are stored.

  The display unit 215 includes a display device such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display), and displays characters and images under the control of the CPU 211. The operation unit 216 includes a keyboard and a mouse (both not shown). The user can input various instructions to the control unit 210 by operating the operation unit 216.

[Operation]
Next, the operation of the embodiment will be described. When the user operates the operation unit 216 to instruct execution of the pronunciation practice program, the CPU 211 reads out and executes the pronunciation practice program from the HDD 214. When the pronunciation practice program is executed, a screen for prompting selection of pronunciation to be practiced is displayed on the display unit 215 as exemplified in FIG. 4 (FIG. 3: step SA1). Here, when an operation for displaying the next menu screen is performed on the operation unit 216 (step SA2; YES), the CPU 211 displays, for example, a menu screen for encouraging the practice of short vowels or the menu screen for encouraging the pronunciation of words. This is displayed on the part 215 (step SA3). When an operation for ending the execution of the pronunciation practice program is performed (step SA4; NO, step SA13; YES), the CPU 211 ends the execution of the program. On the other hand, when the user performs an operation for selecting the sound to be practiced (step SA4: YES), the CPU 211 exemplifies a correct mouth shape when the sound to be practiced is pronounced together with a phonetic symbol, for example, in FIG. It is displayed as described (step SA5). Then, the CPU 211 waits for data output from the sound pressure level detection unit 230 and the sound detection unit 240 (step SA6).

  When the user imitates the displayed mouth shape and sounds toward the microphone unit 100, each silicon microphone 110 disposed in the microphone unit 100 converts the voice uttered by the user into an electric signal and converts the voice input unit. To 220. When the electrical signal output from the microphone unit 100 is input, the audio input unit 220 outputs all the input electrical signals to the sound pressure level detection unit 230 and a predetermined silicon microphone (for example, the silicon microphone 110H). The electrical signal output from −8) is output to the sound detection unit 240.

  The sound pressure level detection unit 230 first generates sound pressure data in order of 110A-2,..., 110A-16 from the electrical signal corresponding to the silicon microphone 110A-1, and then generates the silicon microphone 110B-1. 110B-16, silicon microphones 110C-1 to 110C-16,..., Silicon microphones 110P-1 to 110P-16 are generated, and the sound pressure data is output to the CPU 211 in the generated order. . In addition, the voice detection unit 240 samples the input electric signal and stores it as digital data, performs frequency analysis by fast Fourier transform from this data, and obtains the spectrum of the voice represented by the electric signal. Then, the sound (language sound) generated by the user is determined, and sound generation data indicating the determined sound is output to the CPU 211.

  When the sound pressure data and the sound generation data are input, the CPU 211 stores the sound pressure data in the RAM 213 in the input order, and stores the sound generation data in the RAM 213 (step SA7). Next, the CPU 211 first compares the sound pressure level represented by the sound pressure data corresponding to the silicon microphone 110A-1 with the value represented by the threshold data. If the sound pressure level represented by the sound pressure data is equal to or greater than the value represented by the threshold data, the LED 120A-1 under the silicon microphone 110A-1 is turned on, and the sound pressure level represented by the sound pressure data is When the data is less than the value represented, the LED 120A-1 under the silicon microphone 110A-1 is turned off. Then, the CPU 211 compares the sound pressure data corresponding to each silicon microphone 110 with the threshold data in the order stored in the RAM 213, and turns on / off the LED 120 under each silicon microphone 110 (step SA8).

  Next, the CPU 211 reads, from the HDD 214, sound pressure distribution data indicating the distribution of sound pressure levels when the sound selected in step SA4 is correctly generated. Then, from the sound pressure data stored in the RAM 213 (data indicating the sound pressure level of the sound input to each silicon microphone), the sound pressure level distribution on the surface of the microphone unit 100 is obtained, and the obtained sound pressure level distribution; The coincidence with the distribution of the sound pressure level indicated by the sound pressure distribution data is observed by a method such as pattern matching (step SA9). If the sound pressure level distributions match (step SA9: YES), the CPU 211 reads out voice data representing the sound when the sound selected in step SA4 is correctly pronounced from the HDD 214, and the RAM 213. A coincidence with the audio data stored in (step SA10). Here, if the voices match (step SA10: YES), it is determined that the user has made the pronunciation correctly, and a screen showing the user's pronunciation and that the correct pronunciation has been made is shown in FIG. As shown in FIG. 4, the information is displayed on the display unit 215 (step SA11).

  On the other hand, if NO is determined in step SA9 or NO in step SA10, that is, if it is determined that the user's pronunciation is not correct, the CPU 211 controls the display unit 215 to perform the user's pronunciation and correct pronunciation. A screen indicating that there is no such message is displayed on the display unit 215 as exemplified in FIG. 5C (step SA12).

  As described above, according to the present embodiment, the microphone unit 100 can detect the sound pressure distribution according to the sound generation, and can accurately determine the sound together with the spectrum analysis. In addition, since the microphone unit 100 for detecting the sound produced by the user can be used away from the user, the user can use it without worrying about discomfort and also from the viewpoint of hygiene.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the present invention may be implemented by modifying the above-described embodiment as follows.

  The LED drive voltage is controlled according to the sound pressure level represented by the sound pressure data, and the illuminance of the LED 120 under each silicon microphone 110 is varied according to the sound pressure level detected by each silicon microphone 110. May be. Alternatively, the microphone unit 100 in which only the silicon microphone 110 is disposed may be disposed in front of the display unit 215 so that the sound pressure distribution is displayed on the display unit 215. In recent years, small-sized silicon microphones have been developed, and even if they are arranged in an array, they do not become a large-scale device such as a parabolic antenna. Can be confirmed on the display unit 215.

  The sound pressure level distribution when correctly pronounced may be displayed before the user pronounces it. In addition, when adopting LEDs that can emit multiple colors, the sound pressure level distribution when correctly pronounced and the sound pressure level distribution obtained by the user's pronunciation are lit in different colors so that the sound is pronounced correctly The difference between the sound pressure level distribution and the sound pressure level distribution caused by the user's pronunciation may be visible.

  In the above-described embodiment, the user is shown whether or not the input voice matches the predetermined voice, but the generated voice is not judged to match the predetermined voice. May be recognized and the recognized voice may be displayed. For example, the voice processing device displays “thing” on the display unit 215 when it detects that it pronounced “thing”, and displays “sing” when it detects that it pronounced “sing”. . When “sing” is displayed when the user pronounces “thing”, the user can know that his / her pronunciation is not recognized correctly and the pronunciation is incorrect.

  The frequency spectrum when the language sound is correctly pronounced may be stored in the HDD 214 and compared with the frequency spectrum obtained by the voice detection unit 240 to determine the language sound pronounced by the user.

It is a figure which shows the structure of the audio processing apparatus which concerns on embodiment of this invention. 1 is an external view of a microphone unit 100. FIG. It is the flowchart which showed the flow of the process which CPU211 performs. 6 is a diagram illustrating an example of a screen displayed on a display unit 215. FIG. 6 is a diagram illustrating an example of a screen displayed on a display unit 215. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Microphone unit, 110A-1 to 110P-16 ... Silicon microphone, 120A-1 to 120P-16 ... LED, 130 ... Substrate, 140 ... Stand, 200 ... Control device , 210... Control unit, 211... CPU, 212... ROM, 213... RAM, 214... HDD, 215. Voice input unit, 230... Sound pressure level detection unit, 240.

Claims (5)

Storage means for storing the sound pressure level distribution of the standard language sound and the frequency spectrum of the standard language sound;
A microphone unit in which a plurality of microphones are arranged in an array;
For each signal output from each microphone of the microphone unit, sound pressure level detection means for detecting the sound pressure level of the sound represented by the signal;
Spectrum detecting means for detecting a frequency spectrum of a signal output from a predetermined microphone of the microphone unit;
A sound pressure distribution detecting means for obtaining a sound pressure level distribution of a surface on which the microphone is disposed in the microphone unit from a plurality of sound pressure levels obtained by the sound pressure level detecting means;
The sound pressure level distribution obtained by the sound pressure distribution detection means and the frequency spectrum obtained by the spectrum detection means; the sound pressure level distribution of the standard language sound and the frequency spectrum of the standard language sound stored in the storage means; And voice discrimination means for discriminating the language sound input to the microphone unit;
An audio processing apparatus comprising: output means for outputting a discrimination result of the audio discrimination means. The microphone unit has a light emitter associated with each of the plurality of microphones,
The sound according to claim 1, further comprising control means for lighting a light emitter associated with each microphone based on a sound pressure level corresponding to each microphone detected by the sound pressure level detecting means. Processing equipment.   The sound processing apparatus according to claim 2, wherein the control unit varies the illuminance of the light emitter according to the sound pressure level.   2. The sound processing apparatus according to claim 1, further comprising display means for generating a sound pressure distribution image indicating a sound pressure level distribution obtained by the sound pressure distribution detecting means and displaying the sound pressure distribution image. . Voice request means for requesting the user to speak a language sound at least in either image or voice;
Determining means for determining whether or not the language sound determined by the sound determining means and the language sound requested by the sound requesting means match;
The speech processing apparatus according to claim 1, wherein the output unit outputs a determination result of the determination unit.

Images