KR101250051B1 - Speech signals analysis method and apparatus for correcting pronunciation - Google Patents

Abstract

Disclosed are a voice signal analysis method and apparatus for pronunciation correction. The present invention divides the input voice into a designated time interval unit of phoneme units or less, measures the distribution of the average value of absolute values of each time interval unit, and displays the histogram. Therefore, unlike the conventional technology of correcting a pronunciation based on a specified sentence and a specified word, there is no restriction on an input voice, and it can be easily implemented in a portable device such as a smartphone as well as a computer. The user can easily correct the pronunciation regardless of time or place.

Description

Speech signals analysis method and apparatus for correcting pronunciation

The present invention relates to a voice signal analysis method and apparatus, and more particularly, to a voice signal analysis method and apparatus for pronunciation correction.

As information exchanges increase, the communication between people becomes more important than ever before in modern society. The development of information and communication technology has diversified the means of communication, but the dialogue that conveys human voice is still the most important communication method. In addition, there are various items to be considered even when communicating using voice, and an important one of these items is pronunciation.

Pronunciation is a voice of a language, and there are differences in the characteristics of the pronunciation according to the type and individual of the language. Basically, the pronunciation characteristics of the same language should be expressed to enable accurate communication with each other even considering individual differences.

However, not everyone speaks the correct pronunciation according to language characteristics, and this problem often causes the same word to be repeated many times or incorrect communication occurs. In addition to these practical problems, an incorrect pronunciation can leave a bad impression on others, which can be a problem in personal image management.

Various methods for correcting pronunciation have been suggested to use the correct pronunciation. However, most of the pronunciation correction methods repeat a specific word or sentence that is difficult to pronounce or repeats the pronunciation of another person whose pronunciation is judged to be correct. As I said, most of the sensory methods were not quantitatively analyzed. In other words, iteratively followed the method of repeating repeatedly without knowing the pronunciation characteristics of the person evaluated as correct. This pronunciation correction method has a problem that not only the listening ability of the individual must be preceded but also difficult to apply to various pronunciations in common.

An object of the present invention is to provide a voice signal analysis method that can correct pronunciation by measuring the pronunciation quantitatively and visually displaying the pronunciation characteristics.

Another object of the present invention is to provide a voice signal analysis apparatus that can correct a user's pronunciation using the voice signal analysis method.

In the speech signal analysis method for achieving the above object, a speech signal in the form of speech is applied, and the speech signal is divided into predetermined time interval units to obtain a plurality of speech signals for each section, each of the speech signals for each of the plurality of sections. Calculating an average value of the absolute values of and calculating a plurality of average values, generating a histogram according to the distribution of the plurality of average values, and determining clarity of pronunciation according to the number of peaks shown in the histogram. It is characterized by.

Acquiring a plurality of section-specific speech signals to achieve the above object includes: sampling the applied speech signal at a predetermined frequency and converting the speech signal into a digital speech signal, and the digital speech signal having a shorter time than a phoneme pronunciation time. And dividing the signal into intervals to obtain the plurality of intervals.

(여기서, x(n)은 데시벨(dB) 단위 측정된 디지털화된 음성 신호, y(i)는 i(여기서 i는 자연수)번째 시간 구간에서 측정된 음성 신호(x(n))의 절대값의 평균값, N은 설정된 구간별 샘플링된 디지털 음성 신호의 개수)에 의해 계산되는 것을 특징으로 한다.In the calculating of a plurality of average values for achieving the above object, the average value of each absolute value of each of the plurality of sound signals for each section is expressed by the following equation.

(Where x (n) is the digitized speech signal measured in decibels (dB) and y (i) is the absolute value of the speech signal x (n) measured in the i time interval, where i is a natural number. The average value, N, is calculated according to the set number of digital voice signals sampled for each section.

Generating a histogram for achieving the object may include obtaining a maximum value and a minimum value from the plurality of average values, equally dividing the range between the maximum value and the minimum value into a specified number, and evenly dividing the same. And generating a histogram displaying the number of average values of a corresponding size among the plurality of average values in each of the regions.

The step of determining the clarity of the pronunciation to achieve the object comprises the steps of checking the number of peaks shown in the histogram, if the number of peaks is one, determining that the pronunciation is unclear, and the number of the peaks is a plurality In this case, it is characterized in that it comprises a step of determining that the pronunciation is clear.

The voice signal analyzing apparatus for achieving the another object includes a voice input unit for receiving a voice from outside and converting the voice signal into an electrical signal, and transmitting the voice signal by dividing the voice signal in units of a predetermined time interval. A control unit for acquiring a histogram according to an average value distribution of absolute values of each of the plurality of speech signals for each section, and determining clarity of pronunciation of the speech signal according to the number of peaks of the generated histogram, and the histogram And an output unit for displaying.

The controller may further include a voice signal processor configured to receive the voice signal, sample the voice signal at a predetermined frequency, and convert the voice signal into a digital voice signal.

The control unit for achieving the above object is characterized by obtaining a plurality of speech signals for each of the sections by dividing the digital speech signal by a time interval unit shorter than the phonetic pronunciation time.

The control unit for achieving the object obtains the maximum value and the minimum value from the average value of the absolute value of each of the plurality of voice signals for each section, and evenly divided the range between the maximum value and the minimum value by a specified number, and then equally A histogram is generated in each of the divided regions according to the number of average values of corresponding magnitudes among the average values of the plurality of absolute values.

The control unit for achieving the above object checks the number of peaks shown in the histogram, determines that the pronunciation is unclear if the number of peaks is one, and determines that the pronunciation is clear when the number of the peaks is plural. do.

Voice signal analysis apparatus for achieving the above object is characterized in that it further comprises an input unit for receiving a command from the user.

Voice signal analysis device for achieving the above object is characterized in that the smartphone.

Therefore, the voice signal analysis method and apparatus of the present invention may classify the input voice into a designated time interval unit, measure a distribution of an average value of absolute values of each time interval unit, and display the histogram. Therefore, unlike the conventional technology of correcting a pronunciation based on a specified sentence and a specified word, there is no restriction on the input voice, and it can be easily implemented in a portable device such as a smartphone as well as a computer. The user can easily correct the pronunciation regardless of time or place.

1 shows an example of an audio signal waveform.
2 shows an example of a voice signal analysis method according to the present invention.
3 and 4 show examples of the note intensity distribution curve.
5 shows an example of an apparatus for analyzing voice signals according to the present invention.
6 shows an example of an apparatus for analyzing voice signals according to the present invention.
7 illustrates an example of a pronunciation correction method using the smartphone of FIG. 6.

In order to fully understand the present invention, operational advantages of the present invention, and objects achieved by the practice of the present invention, reference should be made to the accompanying drawings and the accompanying drawings which illustrate preferred embodiments of the present invention.

Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. However, the present invention can be implemented in various different forms, and is not limited to the embodiments described. In order to clearly describe the present invention, parts that are not related to the description are omitted, and the same reference numerals in the drawings denote the same members.

Throughout the specification, when an element is referred to as " including " an element, it does not exclude other elements unless specifically stated to the contrary. The terms "part", "unit", "module", "block", and the like described in the specification mean units for processing at least one function or operation, And a combination of software.

1 shows an example of an audio signal waveform.

The syllables of Korean pronunciation are basically composed of a consonant consonant, a neutral vowel, and a consonant consonant. In some cases, the syllable consists of only a consonant consonant, or a consonant consonant, a vowel consonant, and a consonant consonant.

The sound signal waveform shown in FIG. 1 is a sound signal waveform for one syllable of Korean pronunciation and is a waveform of an audio signal composed of only consonants and neutral vowels. Referring to FIG. 1, it can be seen that the voice energy of the initial consonant interval (0 to 70 msec) is very small compared to the voice energy of the neutral vowel interval (70 to 300 msec). In addition, the inventors of the present invention found that the higher the accuracy of pronunciation, the remarkably different intensities of the signal waveforms of the initial consonant and the neutral vowel. In other words, if the difference between the voice energy between the consonant and the vowels is large, more accurate pronunciation is possible.

2 shows an example of a voice signal analysis method according to the present invention.

2, the voice signal analysis method of the present invention first receives a voice signal (S11). The voice signal may be authorized and stored in real time through a microphone, or the pre-stored voice signal may be applied in the form of data.

The input voice signal (voice signal) is divided into units of a predetermined section (S12). The speech signal may be divided into words and syllable units, and the time required to pronounce one syllable is typically 300msec to 500msec. In the present invention, since a single syllable analyzes the energy of a speech signal in units of phonemes or less, such as consonants and vowels, the speech signal is divided into interval units within a time when one phoneme is pronounced. In the present invention, for example, 1msec is set as the division unit of the voice signal. However, as described above, if one phoneme is a section within a time when a phoneme is pronounced, another time section may be set in division units. The time when one phoneme is pronounced by the measurement is within about 10 msec. Therefore, the time interval unit for dividing the voice signal may be set in units of 1 to 10 msec. However, when the time interval unit is set to 8msec or more, the division of phonemes may become unclear, so it is preferable that the time interval unit is set smaller than 8msec.

When the speech signal is divided in units of time sections, an average value of absolute values of the divided speech signals is calculated (S13). The average value of the absolute value of the speech signal for each section may be calculated as in Equation 1.

In Equation 1, x (n) represents the measured digitized speech signal. N is the number of sampled digital voice signals for each set interval (here 1 msec). Where n and N are natural numbers. For example, if a speech signal is sampled at a rate of 16 kHz, the speech signal is sampled 16000 times per second, so the number N of the sampled digital speech signals is 16000/10 = 1600. Y (i) represents an average value of absolute values of the speech signal x (n) measured in the i th time interval, where i is a natural number. If the voice signal is measured for 10 seconds and the set unit time interval is 1 ms, the average number y (i) of the absolute value of the voice signal for each time interval is 10000/1 = 10000, and i is 1 to 1 It may have a value of 10000. The average value y (i) of the absolute value of the speech signal is typically converted in decibels (dB) by the formula 20 * log ₁₀ (y (i)).

When the average value y (i) of absolute values is calculated for the entire input voice signal section, the maximum and minimum values of the calculated average value y (i) of the absolute value are obtained (S14). Then, the obtained maximum value and the minimum value are equally divided between the specified number (S15). In the present invention, the number divided between the maximum value and the minimum value is set as an example to 100. However, the number of divisions is not limited to 100, but may be variously set.

  If the difference between the maximum value and the minimum value is equally divided, a histogram indicating the number of average values y (i) of the corresponding absolute values for each divided area is calculated (S16). The calculated histogram is a histogram representing an average distribution of speech intensity for each section.

When the histogram is calculated, the number of peaks is determined in the calculated histogram (S17). If the number of peaks is plural, it is determined that the pronunciation is clear (S18). This is because the energy difference between the consonant and the vowel is remarkable as described above, and it can be understood that the consonant and the vowel are clearly distinguished and pronounced. However, if the number of peaks is one, the distinction between consonants and vowels is not clear, and it is determined that the pronunciation is unclear (S19). In other words, the clarity of the pronunciation can be determined according to the number of peaks on the histogram.

When the sampling frequency of the voice signal is low, the number of digital voice signals decreases, making it difficult to calculate a valid histogram. On the other hand, when the sampling frequency of the voice signal is high, the number of digital voice signals increases. However, even if the number of digital voice signals increases, the shape of the histogram does not change significantly. Since the present invention requires finding the number of peaks in the histogram, it does not require a very accurate histogram. In addition, when the number of digital voice signals increases, the amount of data to be calculated increases, so it is not desirable to increase the amount of calculation unnecessarily. Using 8kHz ~ 45kHz as the sampling frequency, it was confirmed through the measurement that the histogram can be efficiently derived.

In addition, although the above description is made to equally divide the obtained maximum value and the minimum value into a predetermined number, the predetermined range may be equally divided into the specified number regardless of the obtained maximum value and the minimum value. The difference between the extremely small and loud sounds that a person can speak is known up to 120 dB. That is, the dB SPL (sound pressure level) of speech is 0 dB to 120 dB, so 0 dB, the sound range that can be spoken by humans, regardless of the maximum and minimum values of the average value (y (i)) of the absolute value calculated. You can also divide the maximum 120 dB into 120 units in dB. That is, the average value y (i) of the absolute value of the speech signal is converted into dB unit by the formula of 20 * log10 (y (i)), so it can be naturally used in 0 ~ 120 dB SPL unit.

As the number of divided regions decreases, the sharpness of the histogram decreases accordingly. However, in the present invention as described above, when the histogram shows a rough shape, the sharpness of the histogram is not very important, so even if the human voice can divide up to 60 dB from 0 dB to 120 dB in 2 dB units, do. That is, in the present invention, the number of divided regions is preferably set to 60 to 120.

However, the above classification is based on the calculated maximum and minimum values of the average value y (i) of the absolute values, and in general, the maximum and minimum values of the speech signal may not exceed a difference of 100 dB. In the above, the average value y (i) of the calculated absolute value is equally divided between the maximum value and the minimum value of 100, and in some cases, the value may be divided into 50.

3 and 4 show examples of the note intensity distribution curve.

Fig. 3 shows a distribution curve of speech intensity average values when the pronunciation is good, and Fig. 4 shows a distribution curve of speech intensity average values when the pronunciation is incorrect. As described above, the higher the accuracy of pronunciation, the more distinctly the intensity of the signal waveforms of the initial consonant and the neutral vowel. Therefore, the distribution of the average value y (i) of the absolute value of the voice signal is represented in the form of a double peak clearly distinguished by intensity as shown in FIG. However, if the pronunciation is incorrect, the intensity of the signal waveforms of the consonant consonants and the neutral vowels cannot be clearly distinguished, and thus shows a general Gaussian distribution in the form of a single peak as shown in FIG.

Therefore, a person who wants to correct a pronunciation can input a voice, analyze the voice signal analysis method, and view histogram visualized to check his / her pronunciation state to correct the pronunciation. And the voice signal analysis method according to the present invention enables to visually confirm by analyzing the pronunciation based on the syllables and phonemes, not sentences or words, to correct the pronunciation using the voice signal analysis method of the present invention Regardless of the sentence, you can check the pronunciation state from the voice input freely. This indicates that a specific word or sentence is designated in the existing pronunciation correction method, and that the present invention can have greater generality than the pronunciation can provide correction only for the specified sentence or word.

5 shows an example of an apparatus for analyzing voice signals according to the present invention.

The voice signal analyzing apparatus 10 includes a voice input unit 110, a voice signal processing unit 120, a controller 130, an input unit 140, and an output unit 150. The voice input unit 110 may be implemented as a microphone, and converts an externally applied voice into an analog voice signal and transmits it to the voice signal processor 120. The voice signal processor 120 receives an analog voice signal and samples the sample at a specified sampling rate to convert the voice signal into a digital voice signal. The converted digital voice signal is transmitted to the controller 130, and the controller 130 calculates a histogram by analyzing the voice signal according to the voice signal analysis method. The output unit 150 may be implemented as a display device and displays a histogram applied by the controller 130. In addition, the output unit 150 may further include a voice signal output device such as a speaker to output a voice by receiving a digital voice signal. In this case, the user can listen to the voice input by the voice input unit 110. The input unit 140 receives a user's command and transmits the command to the controller 130. The input unit 140 may be implemented as a keyboard, a mouse, or the like, and in some cases, may be implemented as a touch screen coupled with the display device of the output unit 150.

Although the voice processor 120 and the controller 130 are illustrated separately, the voice processor 120 may be included in the controller 130.

The voice signal analysis device may be implemented as a mobile computer such as a personal computer, a portable embedded system, or a smart phone. In particular, since the smart phone is already equipped with all hardware components, it is only necessary to provide an application that is software. In addition, pronunciation correction can be utilized as a voice signal analysis device.

6 shows an example of an apparatus for analyzing voice signals according to the present invention.

6 shows that the voice signal analysis device is implemented as a smart phone.

As shown in FIG. 6, the smartphone 20 includes a touch screen 210, a microphone 220, and a speaker 230. The touch screen 210 is a form in which the input unit 140 and the output unit 150 of FIG. 5 are combined, and the speaker 230 is also included in the output unit 150. The microphone 220 corresponds to the voice input unit 110 of FIG. 5. The voice processor 120 and the controller 130 may be implemented as an MPU (Micro Process Unit) provided in the smart phone 20. The pronunciation correction application is installed in the smart phone 20, and the user may execute the pronunciation correction application through the touch screen 210 so that the smart phone 20 may be used as a voice signal analysis device for correcting the pronunciation.

7 illustrates an example of a pronunciation correction method using the smartphone of FIG. 6.

The smartphone user first selects and executes a pronunciation correction program from the menus displayed on the touch screen 210 of the smartphone 20 (S21). When the pronunciation correction program is executed, the user inputs a voice through the microphone 220 provided in the smart phone 20. In this case, the voice input by the user may speak a desired word without having to input a predetermined word or sentence. However, the user must enter the note as a voice signal. The present invention is based on a speech signal processing method for pronunciation correction practice, and the speech signal processing method according to the present invention uses a difference in signal waveforms between phonemes in a syllable, and thus is not a meaningless voice, but a consonant with consonants and vowels combined. Should be entered. Therefore, the voice must be input as speech.

When a voice is input from the user, the MPU of the smart phone 20 analyzes according to the voice signal analysis method described above to calculate a histogram (S23). The calculated histogram is displayed through the touch screen 210. The user determines whether his pronunciation is good based on the displayed histogram (S24). If the pronunciation state is good, the pronunciation correcting application may be terminated. If the pronunciation state is not good, the user may continue to correct the pronunciation by inputting a voice again. In addition, the user may terminate the pronunciation correction application at any time regardless of the pronunciation state.

Although only the histogram for the voice input by the user has been described on the touch screen, the histogram for the correct pronunciation is displayed together with the histogram for the user's voice as shown in FIG. 3 to visualize the inaccuracy of the user's pronunciation. In this way, you can inspire pronunciation correction.

The device according to the invention can be embodied as computer readable code on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and a carrier wave (for example, transmission via the Internet). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art.

Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (16)

A speech signal in the form of speech is applied, and the speech signal is divided into predetermined time intervals to obtain a plurality of speech signals for each section;
Calculating a plurality of average values by calculating an average value of absolute values of each of the plurality of sound signals for each section;
Generating a histogram according to the distribution of the plurality of average values; And
Clarity of pronunciation is determined according to the number of peaks shown in the histogram,
The step of generating the histogram
Obtaining a maximum value and a minimum value from the plurality of average values;
Dividing the range between the maximum value and the minimum value evenly into a specified number; And
Generating a histogram in which the number of average values of corresponding magnitudes among the plurality of average values is displayed in each of the evenly divided regions.
The method of claim 1, wherein the acquiring of the plurality of interval speech signals is performed.
The applied voice signal is sampled at a sampling frequency and converted into a digital voice signal; And
And dividing the digital voice signal into units of time intervals shorter than a phoneme pronunciation time to obtain the plurality of voice signals for each of the plurality of sections.
The method of claim 2, wherein the sampling frequency is
Speech signal analysis method characterized in that the frequency in the range of 8kHz ~ 45kHz.
The method of claim 3, wherein the time interval unit
Speech signal analysis method characterized in that the time interval of 1msec ~ 10msec range.
The method of claim 1, wherein the calculating of the plurality of average values
The average value of the absolute value of each of the plurality of audio signals for each section is
Equation

(Where x (n) is the measured digitized speech signal, y (i) is the average value of the absolute value of the speech signal x (n) measured in the i time interval, where i is a natural number, and N is set). Number of digital audio signals sampled per section)
Lt; / RTI >
The average value of the absolute values of each of the plurality of audio signals for each section is
Equation
y (i) = 20 * log ₁₀ (y (i))
Voice signal analysis method characterized in that the unit conversion in linear units to decibels (dB) by.
delete The method of claim 1, wherein the clarity of the pronunciation is determined.
Confirming the number of peaks shown in the histogram;
If the number of peaks is one, determining that the pronunciation is unclear; And
And if the number of peaks is plural, determining that the pronunciation is clear.
A voice input unit which receives voice from the outside and converts the voice signal into an electrical signal and transmits the voice signal;
After dividing the speech signal into units of a designated time interval, a speech signal for a plurality of sections is obtained, a histogram is generated according to an average value distribution of absolute values of each of the plurality of sections, and then the number of peaks of the generated histogram. A control unit for determining the clarity of the pronunciation of the voice signal according to; And
An output unit for displaying the histogram;
The control unit
The voice signal processor may be further configured to receive the voice signal, sample the voice signal at a sampling frequency, and convert the voice signal into a digital voice signal. Acquire a speech signal for each section,
Obtaining the maximum value and the minimum value from the average value of the absolute value of each of the plurality of voice signals for each section, and evenly divided the range between the maximum value and the minimum value by a specified number, and then the plurality of equally divided areas in each And a histogram is generated according to the number of average values of corresponding magnitudes among the average values of the two dogs.
delete The method of claim 8, wherein the sampling frequency is
Voice signal analysis device, characterized in that the frequency in the range of 8kHz ~ 45kHz.
delete The method of claim 8, wherein the time interval unit is
Voice signal analysis device, characterized in that the time interval of 1msec ~ 10msec range.
delete The method of claim 8, wherein the control unit
And checking the number of peaks shown in the histogram, determining that the pronunciation is unclear when the number of peaks is one, and determining that the pronunciation is clear when the number of the peaks is plural.
The apparatus of claim 8, wherein the voice signal analysis device is
Voice signal analysis device further comprises an input unit for receiving a command from the user.
The apparatus of claim 8, wherein the voice signal analysis device is
Voice signal analysis device, characterized in that the smartphone.

Images