CN104732977B - A kind of online spoken language pronunciation quality evaluating method and system - Google Patents

Abstract

The invention discloses a method and system for evaluating the pronunciation quality of online spoken language. The method comprises: receiving a test voice collected by a mobile client through a network; performing preprocessing on the received test voice; and performing preprocessing on the preprocessed test voice The extraction of speech characteristic parameter obtains the characteristic parameter of described test speech; According to the characteristic parameter of described test speech and the characteristic parameter of standard speech, described test speech is evaluated, obtains evaluation result; Described evaluation result is fed back through network to the mobile client, and display the evaluation result through the mobile client. The invention can realize online, convenient and accurate oral pronunciation quality evaluation.

Description

A method and system for evaluating online oral pronunciation quality

technical field

The invention relates to the technical field of speech recognition and evaluation, in particular to a method and system for evaluating the quality of online spoken language pronunciation.

Background technique

The application of signal processing technology in language learning is an important part of the integration of information technology and language learning. Its goal is to combine the latest voice technology with current teaching and learning methods to establish a computer-aided language learning system, and the quality of spoken pronunciation is evaluated As an important content to assist language learning, it has always attracted much attention.

However, traditional oral pronunciation quality evaluation systems are mostly limited to traditional language learning machines or personal computers, which are inconvenient to carry and limited by wired network connections, which is not conducive to learners' oral learning anytime and anywhere. The existing oral pronunciation quality scoring system suitable for mobile smart phones adopts the mode of offline operation, but the current mainstream mobile smart phones cannot support the storage of large amounts of data and complex voice calculations, which limits the Due to the complexity of the pronunciation quality evaluation algorithm, the scoring results cannot truly reflect the pronunciation quality level of oral English learners. At the same time, the existing scheme deploys the speech evaluation system in language learning machines, personal computers and mobile devices, which is not conducive to data update, storage and algorithm improvement. Furthermore, in the existing oral pronunciation quality evaluation system, the evaluation indicators comprehensively considered in the pronunciation quality evaluation are not comprehensive enough, most of them are limited to a single or a small number of evaluation indicators, and cannot provide scientific, comprehensive and accurate evaluation of the user's voice pronunciation quality , often just give a score based on pronunciation, lack of evaluation and feedback.

Contents of the invention

The purpose of the embodiments of the present invention is to provide an online oral pronunciation quality evaluation method and system, so as to realize online, convenient and accurate oral pronunciation quality evaluation.

On the one hand, the embodiment of the present invention provides a method for evaluating the quality of online oral pronunciation, including:

Receive the test voice collected by the mobile client through the network;

Preprocessing the received test voice;

Carry out the extraction of speech characteristic parameter to the test speech after pretreatment, obtain the characteristic parameter of described test speech;

Evaluate the test speech according to the characteristic parameters of the test speech and the characteristic parameters of the standard speech, and obtain an evaluation result;

The evaluation result is fed back to the mobile client through the network, and the evaluation result is displayed through the mobile client.

Preferably, the online oral pronunciation quality evaluation method also includes:

storing the evaluation results in a database, and performing statistical analysis on the evaluation results to obtain statistical results;

The statistical results are sent to the webpage management terminal, and the statistical results are displayed through the webpage management terminal.

Preferably, the online oral pronunciation quality evaluation method also includes:

Get standard voice;

Preprocessing the standard voice;

Extracting speech feature parameters from the preprocessed standard speech to obtain the feature parameters of the standard speech.

Preferably, the preprocessing includes pre-emphasis, framing, windowing and endpoint detection.

Preferably, the extraction of speech feature parameters to the preprocessed test speech to obtain the feature parameters of the test speech includes:

Carry out discrete Fourier transform to described test speech, obtain the spectral coefficient of described test speech, filter described spectral coefficient with sequence triangular filter, carry out logarithmic operation to the data after filtering, utilize discrete cosine transform, obtain described The MFCC characteristic parameter of test speech;

Extract the fundamental frequency feature, short-term energy feature, and formant feature of the test speech, and form the emotional feature parameter of the test speech with the fundamental frequency feature, the short-term energy feature and the formant feature ;

Calculating the pronunciation duration of the test voice, and obtaining the pronunciation duration characteristic parameter of the test voice;

Carry out accent unit division to described test speech, extract the initial frame position group and the end frame position group of accent, obtain the stress position feature parameter of described test speech;

Carry out speech unit division to described test speech, calculate the duration of each speech unit respectively, obtain the speech unit duration characteristic parameter of described test speech;

The pitch of each frame of data of the test voice is extracted by an autocorrelation function method in the time domain, and the pitch characteristic parameter of the test voice is obtained.

Preferably, the said test speech is evaluated according to the characteristic parameters of the test speech and the characteristic parameters of the standard speech, and the evaluation result is obtained, including:

According to the MFCC characteristic parameter of described test speech, based on the probabilistic neural network integrated speech recognition model of segmentation clustering, described test speech is carried out speech recognition, obtains speech recognition result; The MFCC feature parameter of described standard speech carries out similarity calculation, obtains MFCC correlation coefficient; According to described speech recognition result and described MFCC correlation coefficient, calculate the accuracy score of described test speech;

According to the emotion characteristic parameter of described test speech, based on SVM emotion model, carry out emotion recognition to described test speech, obtain emotion recognition result; Calculate and obtain the emotion correlation coefficient; calculate the emotion score of the test speech according to the emotion recognition result and the emotion correlation coefficient;

According to the pronunciation duration characteristic parameter of described standard speech and described test speech, obtain the speech speed ratio of described standard speech and described test speech, and according to described speech speed ratio, calculate the speech rate score of described test speech;

According to the stress position characteristic parameter of the test speech and the stress position characteristic parameter of the standard speech, compare the stress position difference between the test speech and the standard speech, and calculate the test according to the stress position difference Speech stress score;

According to the speech unit duration characteristic parameter of the test speech and the speech unit duration characteristic parameter of the standard speech, utilize the dPVI algorithm to obtain the dPVI parameter of the test speech, and calculate the test speech according to the dPVI parameter tempo score;

According to the pitch characteristic parameter of the test speech and the pitch characteristic parameter of the standard speech, use the DTW algorithm to obtain the pitch difference between the standard speech and the test speech, and calculate according to the pitch difference The intonation score of the test speech.

Preferably, the evaluation of the test speech according to the characteristic parameters of the test speech and the characteristic parameters of the standard speech to obtain an evaluation result also includes:

The accuracy score, the emotional score, the speed of speech score, the stress score, the rhythm score and the intonation score are weighted and summed to obtain a comprehensive score; and according to the accuracy score, the The emotion score, the speech rate score, the stress score, the rhythm score, the intonation score and the comprehensive score, combined with the mapping relationship between each score and grade evaluation, obtain the accuracy grade evaluation of the test voice , emotional grade evaluation, speech speed grade evaluation, accent grade evaluation, rhythm grade evaluation, intonation grade evaluation and comprehensive grade evaluation; and the accuracy grade evaluation, emotional grade evaluation, speech speed grade evaluation, stress grade evaluation , rhythm level evaluation, intonation level evaluation and comprehensive level evaluation as the evaluation results of the test speech.

Preferably, the online oral pronunciation quality evaluation method also includes:

According to the evaluation result, guide the user's oral pronunciation, and obtain pronunciation guidance;

The pronunciation guidance is fed back to the mobile client through the network, and the pronunciation guidance is displayed through the mobile client.

On the other hand, an embodiment of the present invention provides an online oral pronunciation quality evaluation system, including a mobile client and a server connected through a network;

The mobile clients include:

A voice collection unit, configured to collect a test voice, and send the test voice to the server through the network;

The server side includes:

A preprocessing unit is used to preprocess the received test voice;

A characteristic parameter extraction unit is used to extract the speech characteristic parameters of the preprocessed test speech, and obtain the characteristic parameters of the test speech;

The voice evaluation unit is used to evaluate the test voice according to the feature parameters of the test voice and the feature parameters of the standard voice to obtain an evaluation result; and feed back the evaluation result to the mobile client through the network;

The mobile client also includes:

The data display unit is used to display the evaluation result.

Preferably, the system also includes a webpage management terminal, which is connected to the server through a network; the server also includes a database and a statistical analysis unit;

The database is used to store the evaluation results;

The statistical analysis unit is used to perform statistical analysis on the evaluation results to obtain statistical results; and send the statistical results to the webpage management terminal;

The webpage management terminal is used to display the received statistical results.

Compared with the prior art, the advantages of the embodiments of the present invention are:

The embodiment of the present invention is based on the C/S (Client/Server, client/server end) architecture, constructs mobile client and server end, collects user's test voice signal by mobile client end and sends to server end, and server end is to test voice signal After the evaluation, the voice evaluation result is returned to the mobile client, and finally the evaluation result is displayed through the mobile client. Users can use the mobile Internet to easily access the server to obtain services and data. The corpus and evaluation methods can be synchronized through the server, and the server provides better performance and better effect of speech analysis algorithm processing.

Secondly, the embodiment of the present invention is also based on the B/S (Browser/Server, webpage/server) framework, builds the webpage management end and the server end, can obtain the spoken language of the mobile client user in real time from the database of the server end through the web browser The statistical results of pronunciation quality evaluation provide third parties (such as educators) with the oral pronunciation of mobile client users, which is convenient for third parties to formulate offline oral guidance and improvement strategies.

Furthermore, the embodiments of the present invention perform multi-dimensional speech evaluation on the test speech. The evaluation method of each index is reasonable and credible, and the pronunciation guidance opinions can be fed back to the user's spoken pronunciation, which helps to correct the user's pronunciation errors and improve the pronunciation quality .

Description of drawings

Fig. 1 is a flow chart of the steps of an embodiment of the online oral pronunciation quality evaluation method provided by the present invention;

Fig. 2 is a schematic diagram of the establishment process of the probabilistic neural network integrated classifier provided by the present invention;

Fig. 3 is the C/S architecture schematic diagram of an embodiment of the online oral pronunciation quality evaluation system provided by the present invention;

FIG. 4 is a schematic diagram of the B/S architecture of the online oral pronunciation quality evaluation system shown in FIG. 3 .

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. It should be noted that the numbers in front of the steps in the embodiments are only for the purpose of more clearly identifying the steps, and there is no necessary sequence limitation between the steps.

Referring to Fig. 1, it is a flow chart of the steps of an embodiment of the online oral pronunciation quality evaluation method provided by the present invention, said method comprising:

S1, receiving the test voice collected by the mobile client through the network.

In the specific implementation, the mobile client is installed on the user's mobile phone or other mobile devices in the form of an application program, and records by calling the recording program in the mobile device, collects the user's voice in the oral test, and generates a unified format An audio file, the mobile client compresses and encodes the audio file and then sends it to the server through the network. Wherein, described audio file is preferably the audio file of wav format, and described network is preferably mobile internet, and mobile client and server end adopt based on TCP/IP (Transmission Control Protocol/Internet Protocol, Transmission Control Protocol/Internet Protocol) communication protocol Socket for data transmission.

S2. Preprocessing the received test voice.

After receiving the data sent by the mobile client, the server side decompresses and decodes the received data and restores it to the original file of the test voice. At the same time, before analyzing and processing the test voice, in order to eliminate the influence of the human pronunciation organ itself and the voice equipment on the test voice, the test voice is preprocessed to provide a high-quality data source for the extraction of subsequent voice feature parameters, thereby Improve the quality of speech processing. The preprocessing described in this embodiment includes, but is not limited to, pre-emphasis, framing, windowing, and endpoint detection, as follows:

2.1) Pre-emphasis: The average power spectrum of the test voice is affected by glottal excitation and mouth and nose radiation. The high-frequency end is attenuated by 6dB/oct above 800 Hz. The higher the frequency, the smaller the corresponding component. Therefore, the test voice is Before the analysis, the high-frequency part of the test speech needs to be enhanced. This embodiment adopts a 6dB/oct high-frequency boost pre-emphasis digital filter before analyzing the test voice to enhance the high-frequency part of the test voice, so that the frequency spectrum of the test voice becomes flat and remains at In the frequency band from low frequency to high frequency. The calculation formula for pre-emphasis is as follows:

y(n)=x(n)-0.9375*x(n-1) (Formula 1)

Among them, x(n) is the original test speech.

2.2) Framing: The speech signal has time-varying characteristics, but in a short time range, its characteristics remain basically unchanged, that is, relatively stable. This characteristic of the speech signal is called "short-term", and this short-term It is 10 ~ 30ms, so the analysis and processing of the test voice is based on the "short-time nature", and the test voice is "short-time analysis" (ie frame processing). Due to the correlation between the speech signals, the present embodiment divides the test speech into frames in a way of half-frame overlapping and framing.

2.3) Windowing: In order to realize that the voice waveform near the sampling position in the test voice is emphasized and the rest of the waveform is weakened, in this embodiment, the Hamming window is selected to add a window to the test voice, and the windowing process is carried out after the frame division The Gibbs phenomenon (Gibbs phenomenon) caused by truncation can be reduced, so that the frequency spectrum of the test speech is relatively smooth. In a practicable manner, the calculation formula of windowing is as follows:

S _ω (n)=y(n)*ω(n) (Formula 2)

Among them, y(n) is the speech signal after pre-emphasis, ω(n) is the window function.

2.4) Endpoint detection: In this embodiment, a double-threshold comparison method is used for endpoint detection to detect the start point and end point of the test voice. The double-threshold comparison method is characterized by short-term energy E and short-term average zero-crossing rate Z, and combines the advantages of short-term average zero-crossing rate Z and short-term energy E to make detection more accurate, effectively reduce system processing time, and improve The real-time processing of the system can eliminate the noise interference of the silent segment, thereby improving the processing performance of the speech signal.

S3. Extracting speech feature parameters from the preprocessed test speech to obtain the feature parameters of the test speech. The feature parameter of described test voice comprises MFCC (Mel-Frequency Cepstral Coefficients, Mel cepstral coefficient) feature parameter, emotion feature parameter, pronunciation duration feature parameter, stress location feature parameter, speech unit duration feature parameter and pitch feature parameter, in server The feature parameter extraction process at the end is as follows:

3.1) Carry out discrete Fourier transform (DFT, Discrete Fourier Transform) to described test speech, obtain the spectral coefficient of described test speech, filter described spectral coefficient with sequence triangular filter, carry out logarithmic operation to the data after filtering , using discrete cosine transform to obtain the MFCC characteristic parameters of the test speech. Specific steps are as follows:

Discrete Fourier transform is performed on the preprocessed test speech to obtain the spectral coefficient X(k).

Filter the spectral coefficients X(k) with a sequence triangular filter to obtain a set of coefficients m _i . The formula for calculating the coefficient ^mi is as follows:

m _i =ln[X(k)*H _i (k)] (Formula 3)

in,

(Formula 4)

f[i] is the center frequency of the triangular filter, satisfying:

Mel(f[i+1])-Mel(f[i])=Mel(f[i])-Mel(f[i-1]) (Formula 5)

Calculate the logarithm of the output of all filters, and then use the discrete cosine transform to obtain the cepstral coefficient. The calculation formula is as follows:

(Formula 6)

Among them, P is the number of triangular filters, and C _i is the characteristic parameter of the MFCC to be sought. Preferably, the order of the MFCC characteristic parameters is set to 12.

3.2) extract the fundamental frequency feature, short-term energy feature, and formant feature of the test speech, and form the emotion of the test speech with the fundamental frequency feature, the short-term energy feature and the formant feature Characteristic Parameters.

3.2.1) Fundamental frequency characteristics: The pitch period refers to the periodicity caused by the vibration of the vocal cords when making voiced sounds, and the fundamental frequency is the reciprocal of the pitch period. Pitch frequency is one of the most important parameters of speech signals, and studies have shown that pitch frequency can reflect changes in emotion. The detection methods of fundamental frequency features include but not limited to autocorrelation function method (ACF), cepstrum analysis method, average amplitude difference function method (AMDF) and wavelet method. Preferably adopt cepstrum analysis method in the present embodiment, carry out Fourier transform to the test speech after the preprocessing, obtain the magnitude spectrum of described test speech, take logarithm to described magnitude spectrum, obtain a test speech in the frequency domain A periodic signal, calculating the frequency value of the periodic signal, can obtain the fundamental frequency value of the test voice. Inverse Fourier transform is performed on the periodic signal to obtain a peak value at the pitch period. After obtaining the fundamental frequency value, the 7th-order fundamental frequency statistical change parameters such as the maximum value, minimum value, mean value, median value and standard deviation of the fundamental frequency are obtained by calculation, which are used as the fundamental frequency characteristics of the test speech.

3.2.2) Short-term energy characteristics: the energy of the speech signal has a strong correlation with the expression of emotional speech, and a large energy indicates that the volume of the sound is relatively loud. In real life, when people are angry or angry, the volume of pronunciation is louder. When people are depressed or sad, their voices tend to be lower. Speech signal energy usually has two types: short-term energy and short-term average amplitude energy, and the short-term energy of the test speech is preferably selected as the energy parameter. The short-term energy is the weighted sum of squares of a frame of sampling point values, and the short-term energy is defined as follows:

(Formula 7)

Among them, x _n (m) is the nth frame signal of the test speech.

After the short-term energy is obtained, the seven-order short-term energy statistical change parameters such as the maximum, minimum, mean, median and standard deviation of the short-term energy are calculated as the short-term energy characteristics of the test speech.

3.2.3) Formant characteristics: Formant is an important parameter reflecting the characteristics of the vocal tract. When the sound excitation passes through the vocal tract, the formant frequency will be generated. When a person is in different emotional states, the tension of their nerves is different, resulting in deformation of the vocal tract, and a corresponding change in the formant frequency. In this embodiment, the linear prediction method is preferably used to extract the formant parameters of each frame of the speech signal, which can quickly, excellently and effectively extract the formant parameters, and obtain the first formant and the second resonance of the speech signal by the linear prediction method peak, and then the first formant and the second formant are regularized into 32 order parameters by the segmented clustering method, as the formant feature of the test speech. Combine formant features, fundamental frequency features and short-term energy features to form 46-order speech emotion feature parameters.

3.3) Calculating the pronunciation duration of the test speech, and obtaining the characteristic parameter of the pronunciation duration of the test speech.

In a specific implementation, the endpoint detection of the test voice can be performed by setting the high and low limit values of the short-term energy and the zero-crossing rate, so as to obtain the pronunciation duration of the test voice.

3.4) Divide the test speech into stress units, extract the start frame position group and the end frame position group of the stress, and obtain the stress position characteristic parameters of the test speech.

The accent unit division process is as follows:

a. Extract the energy value of the test speech. The characteristics of loud stressed syllables in the test speech will be reflected in the energy intensity in the time domain, that is, the stressed syllables are characterized by high speech energy intensity.

b. Regularize the test voice. Due to the difference in speech speed between speakers, there are certain differences in the pronunciation duration of the same sentence by different speakers, but the pronunciation of the same sentence by different people follows the law that the duration of the stress unit accounts for a certain proportion of the entire sentence. Therefore, when the test speech is scored, the pronunciation duration of the test speech can be adjusted in proportion to be the same as the pronunciation duration of the standard speech by calling the pronunciation duration characteristic parameter of the standard speech, which is beneficial to the processing of data. It also makes the evaluation of the system more objective.

c. Extracting stressed syllables of the test speech. In the actual implementation, the double-threshold comparison method can be used to detect the stress endpoint. According to the energy value of the test speech, the maximum speech signal value S _max in the test speech that is greater than the stress threshold T _u is searched one by one, and then the signal value S _max is increased to about Search for the speech signal values S _l and S _r equal to the non-accent threshold T _l , set S _l and S _r as the accent signals of the test speech, and set the signal value between S _l and S _r to 0 to avoid repetition Search between S _l and S _r . Since the stressed syllables in the test speech have long pronunciation characteristics, and the stressed syllable units searched in the first step may have a large energy value, that is, the auditory performance is loud, but the duration is very short. These units may be short units They may also be the interference of signal spikes, they do not constitute stressed syllables, so the stressed syllable units can be further screened according to the longer pronunciation of the stressed syllable unit, and the minimum duration of the stressed syllable unit is set to a roughly stressed vowel duration ( Stressed vowel durations), preferably 100ms, and compare according to the set minimum duration.

Through the above steps, complete the division of the sentence stress unit, you can know the start frame position group and the end frame position group of the accent of the sentence, and use the start frame position group and the end frame position group as the test voice Accent position feature parameter.

3.5) Divide the test speech into speech units, calculate the duration of each speech unit respectively, and obtain the speech unit duration characteristic parameter of the test speech. The duration of the speech unit refers to the duration from the beginning to the end of each speech unit.

3.6) Extracting the pitch of each frame of data of the test voice through the autocorrelation function method (ACF) in the time domain, and obtaining the pitch characteristic parameters of the test voice.

The autocorrelation function method is to use the autocorrelation function to calculate the similarity between a sound frame s(i) and itself, wherein, i=0～n-1, the calculation formula is as follows:

(Formula 8)

Among them, n refers to the length of a frame of voice data, τ is the amount of time delay, find out the value of τ that can make acf(τ) within a reasonable specific interval, and then the pitch of the sound frame can be calculated. In the specific ACF calculation process, the speech frame is shifted a little to the right each time, and the overlapping part of the shifted sound frame and the original sound frame is used as an inner product, and the n inner product values obtained after repeating n times are corresponding to a speech frame The ACF value.

S4. Evaluate the test speech according to the characteristic parameters of the test speech and the characteristic parameters of the standard speech, and obtain an evaluation result.

It should be noted that the characteristic parameters of the standard speech are obtained by extracting the speech characteristic parameters of the standard speech in advance, stored in the database, and retrieved when needed. The specific steps of extracting the characteristic parameters of the standard speech include: obtaining the standard speech; preprocessing the standard speech; extracting the speech characteristic parameters of the preprocessed standard speech, and obtaining the characteristic parameters of the standard speech. The specific steps of extracting the feature parameters of the standard speech are the same as the process of extracting the feature parameters of the test speech, and will not be repeated here.

According to the feature parameters of the test voice and the feature parameters of the standard voice, the process of evaluating the test voice is as follows:

4.1) According to the MFCC characteristic parameters of the test speech, a speech recognition model is integrated based on a segmented clustering probabilistic neural network (Probabilistic Neural Network, PNN), and the speech recognition is performed on the test speech to obtain a speech recognition result. and performing similarity calculation on the MFCC characteristic parameters of the test speech and the MFCC characteristic parameters of the standard speech to obtain the MFCC correlation coefficient. Calculate the accuracy score of the test speech according to the speech recognition result and the MFCC correlation coefficient. It should be noted that the segmented clustering probabilistic neural network integrated speech recognition model is trained in advance, stored in the database, and called when needed.

In this embodiment, the idea of Bagging (Bootstrap aggregating, self-help aggregation) is used to generate the individual probabilistic neural network model required for integration. Bagging is an integrated learning method that integrates a plurality of different individual learners into one learner. Different data subsets can be obtained by repeated sampling, so that individual learners trained on different data subsets have high generalization performance and large differences. Using the distributed computing of the existing network can further improve the time efficiency of the algorithm, and Bagging can improve the performance of the learner, which is conducive to improving the classification accuracy and generalization ability of the probabilistic neural network.

Referring to FIG. 2 , it is a schematic diagram of the establishment process of the probabilistic neural network ensemble classifier provided by the present invention. Each time, n samples are randomly selected from the training sample set A (Bagging sample A1, Bagging sample A2...Bagging sample An in the figure), trained with a probabilistic neural network classification algorithm, and a PNN classifier is obtained, which is generated using the same method Multiple PNN classifiers (ie, PNN classifier C ₁ (x), PNN classifier C ₂ (x)...PNN classifier C _n (x) in the figure), after training, a classification function sequence C ₁ (x) can be obtained , C ₂ (x)...C _n (x), that is, the PNN integrated classifier, that is, the PNN integrated speech recognition model described in this embodiment, the final classification function C (x) adopts a voting method for the classification problem, and gets the most votes The classification result of is the final category of the classification function C(x).

In the process of speech recognition, it is only necessary to input the MFCC feature parameters of the test speech into the PNN integrated speech recognition model, classify by voting, and judge whether the content is correct. Simultaneously, similarity calculation is carried out to the MFCC feature parameters of the test speech and the MFCC feature parameters of the standard speech, and finally the accuracy of the test speech is scored according to whether the content is correct and the size of the MFCC correlation coefficient.

4.2) According to the emotional feature parameters of the test speech, based on the SVM (Support Vector Machine, Support Vector Machine) emotion model, perform emotion recognition on the test speech to obtain an emotion recognition result. And performing similarity calculation on the emotional characteristic parameters of the test speech and the emotional characteristic parameters of the standard speech to obtain an emotional correlation coefficient. An emotion score of the test speech is calculated according to the emotion recognition result and the emotion correlation coefficient.

After the emotional feature parameters of the test speech are extracted, the emotional feature parameters are input to the SVM emotion model for classification, and the correlation coefficient between the emotional feature parameters of the test speech and the standard speech is calculated. Finally, the sentiment score is obtained according to whether the sentiment classification result is correct or not and the correlation coefficient of the sentiment feature parameters.

4.3) According to the pronunciation duration feature parameter of the standard speech and the test speech, obtain the speech rate ratio of the standard speech and the test speech, and calculate the speech rate of the test speech according to the speech rate ratio Score.

After extracting the pronunciation duration feature parameter of the test voice, calculate the speech rate ratio by the following formula:

(Formula 9)

Wherein, the S _{pronunciation duration} refers to the duration of the standard speech, and the T _{pronunciation duration} refers to the pronunciation duration of the test speech.

Speech speed that is too fast or too slow does not meet the requirements of linguistic expression. Therefore, the speech speed of the test voice can be scored according to the degree of too fast or too slow according to the ratio of speech speed.

4.4) According to the stress position feature parameter of the test speech and the stress position feature parameter of the standard speech, compare the stress position difference between the test speech and the standard speech, and calculate the stress position difference according to the stress position difference. Stress scores for the test utterances described above.

When extracting the stress position feature parameters, the start frame position and the end frame position group of the accent are obtained, and the stress distribution difference diff between the test speech and the marked speech is calculated by the following formula:

(Formula 10)

Wherein, Len _std refers to the effective speech frame length of the standard speech, and Len _test refers to the effective speech frame length of the test speech. left _std [i] is the start frame position group of standard voice, right _std [i] is the end frame position group of standard voice. left _test [i] is the start frame position group of the test speech, and right _test [i] is the end frame position group of the test speech.

According to the stress position difference between the test speech and the standard speech, the stress of the test speech is scored.

4.5) According to the speech unit duration characteristic parameter of the test speech and the speech unit duration characteristic parameter of the standard speech, utilize the dPVI (the Distinct Pairwise Variability Index) algorithm to obtain the dPVI parameter of the test speech, and according to the dPVI parameters to calculate the rhythm score of the test speech.

After extracting the characteristic parameters of the speech unit duration of the test speech, compare the characteristic parameters of the syllable unit duration of the test speech with the speech unit duration characteristic parameters of the standard speech, and convert the dPVI parameters used for the system scoring basis, and the calculation formula of the dPVI parameters as follows:

(Formula 11)

Wherein, d is the speech unit duration of sentence division (such as: d _k is the kth speech unit duration), m=min(S _snum , T _snum ), S _snum is the speech unit number of standard speech, and T _snum is the test speech The number of speech units, Len is the duration of the standard speech.

Calculate the rhythm score of the test speech according to the size of the dPVI parameter.

4.6) according to the pitch characteristic parameter of described test speech and the pitch characteristic parameter of described standard speech, utilize DTW (Dynamic Time Warping, dynamic time rounding) algorithm, obtain the pitch of described standard speech and described test speech difference, and according to the pitch difference, calculate the intonation score of the test voice.

After extracting the pitch feature parameters of the test speech, a median filter can also be set to smooth the pitch and exclude unstable and abnormal speech frames. Utilize the DTW algorithm to compare the pitch feature parameters of the test voice with the pitch feature parameters of the standard voice, calculate the pitch difference parameter dist between the two, and then calculate the intonation score of the test voice, the intonation score The calculation formula of is as follows:

(Formula 12)

Wherein, a=0.0005 and b=2 are calculated by comparing expert scoring data and system scoring data through simulation experiments.

4.7) Perform weighted summation of the accuracy score, the emotion score, the speech rate score, the stress score, the rhythm score and the intonation score to obtain a comprehensive score. And according to the accuracy score, the emotion score, the speech rate score, the stress score, the rhythm score, the intonation score and the comprehensive score, combined with the mapping relationship between each score and grade evaluation, obtain Accuracy grade evaluation, emotion grade evaluation, speech rate grade evaluation, accent grade evaluation, rhythm grade evaluation, intonation grade evaluation and comprehensive grade evaluation of the test speech. And the accuracy grade evaluation, emotion grade evaluation, speech speed grade evaluation, accent grade evaluation, rhythm grade evaluation, intonation grade evaluation and comprehensive grade evaluation of the test speech are taken as the evaluation results of the test speech.

In the process of weighting and summing the accuracy score, the emotion score, the speech rate score, the stress score, the rhythm score and the intonation score, the weight of each index score can be based on different Different values are used for the needs of the user, and the weight combination that meets the user's needs can be selected according to the user's own characteristics. According to the mapping relationship between each score and the grade evaluation, the grade evaluation and the comprehensive grade evaluation of each indicator are obtained. For example, if the accuracy score is within the score range of 90-100, the accuracy grade is evaluated as A grade; if the accuracy score is within the score range of 70-90, the accuracy grade evaluation is Grade B; if the accuracy score is within the range of 60 to 70, the accuracy rating is grade C; if the accuracy score is within the range of 0 to 60, the accuracy The grade evaluation is D grade. The mapping relationship between other scores and grade evaluations is similar to the above mapping relationship between accuracy scores and accuracy grade evaluations, and will not be repeated here. It should be noted that the above-mentioned mapping relationship between scores and grades is only an example of the present invention. In practical applications, different thresholds can be set according to actual needs, and different score ranges can be mapped to different grades. More classes can be divided.

S5. Feed back the evaluation result to the mobile client through the network, and display the evaluation result through the mobile client.

After the server side obtains the evaluation result, it will feed back the evaluation result to the mobile client through the mobile Internet, and the mobile client will display the evaluation result information on the screen of the mobile device, or prompt the evaluation result information through audio.

In a specific implementation, after the server side obtains the evaluation result of the test voice, it can also guide the user's oral pronunciation according to the evaluation result, and obtain pronunciation guidance opinions. According to the evaluation results, it can be matched with the pronunciation guidance in the database.

The pronunciation guidance is fed back to the mobile client through the network, and the pronunciation guidance is displayed through the mobile client. Pronunciation guidance can be used to point out the errors and deficiencies in the user's oral pronunciation, and provide suggestions for improvement. If it is detected that the user's speech speed is too fast and the rhythm is chaotic, the user can be prompted to slow down the speech speed to grasp the rhythm of the sentence, etc.

The embodiment of the present invention is based on C/S (Client/Server, client/server end) architecture, builds mobile client and server end, and mobile client collects user's test voice signal and sends to server end, and server end carries out test voice After the evaluation, the voice evaluation result is returned to the mobile client, and the evaluation result is displayed through the mobile client. Users can use the mobile Internet to easily access the server to obtain services and data. The corpus and evaluation methods can be synchronized through the server, and the server provides better performance and better effect of speech analysis algorithm processing.

Further, the online oral pronunciation quality evaluation method also includes:

S6. Store the evaluation results in a database, and perform statistical analysis on the evaluation results to obtain statistical results.

In the middle of specific implementation, when the user's test is finished, the user information of the user, the test voice and the evaluation result can be stored in the database, and the server end performs statistical analysis on the evaluation result (comprising each index score and the comprehensive score) in the database, The analysis results of the learning situation of a single user can be obtained, and the analysis results of the learning situation of the group or the statistical results of the learning situation of the whole network can also be obtained for users of a specific user group or for all users of the entire network.

S7. Send the statistical result to the webpage management terminal, and display the statistical result through the webpage management terminal. The webpage management terminal receives the statistical data of the server-side evaluation of the oral pronunciation of the mobile client users, and presents it to a third party (such as a teacher) in a visualized form.

The embodiment of the present invention is based on the B/S (Browser/Server, webpage/server) architecture, constructs the webpage management end and the server end, can obtain the oral pronunciation quality evaluation of the mobile client user in real time from the database of the server end through the web browser The statistical results provide the third party with the spoken pronunciation of mobile client users, which facilitates the third party to formulate offline oral guidance and improvement strategies.

Referring to FIG. 3 , it is a C/S architecture diagram of an embodiment of the online oral pronunciation quality evaluation system provided by the present invention. The basic principle of the online oral pronunciation quality evaluation system in the embodiment shown in Figure 1 is consistent with the online spoken pronunciation quality evaluation method in the embodiment shown in Figure 1. For the parts not described in detail in this embodiment, see the relevant description in the embodiment shown in Figure 1 .

The system includes a mobile client 100 and a server 200 connected through a network.

The mobile client 100 includes:

The voice collection unit 101 is configured to collect a test voice, and send the test voice to the server 200 through the network.

Described server end 200 comprises:

The preprocessing unit 201 is configured to preprocess the received test voice.

The characteristic parameter extraction unit 202 is configured to extract the speech characteristic parameters of the preprocessed test speech, and obtain the characteristic parameters of the test speech.

The voice evaluation unit 203 is used to evaluate the test voice according to the feature parameters of the test voice and the feature parameters of the standard voice to obtain an evaluation result; and feed back the evaluation result to the mobile client 100 through the network .

The mobile client 100 also includes:

The data display unit 102 is configured to display the evaluation result.

Referring to FIG. 4 , it is a schematic diagram of the B/S architecture of the online oral pronunciation quality evaluation system shown in FIG. 3 .

The system also includes a webpage management terminal 300, which is connected to the server terminal 200 through a network. The server 200 also includes a database 204 and a statistical analysis unit 205 .

The database 204 is used to store the evaluation results.

The statistical analysis unit 205 is configured to perform statistical analysis on the evaluation results to obtain statistical results. And send the statistical result to the webpage management terminal 300.

The webpage management terminal 300 is used to display the received statistical results.

The embodiment of the present invention is based on C(B)/S, builds mobile client 100, server end 200 and webpage management end 300, collects user's test voice signal by mobile client 100 and sends to server end 200, and server end 200 pairs test After the voice evaluation is performed, the voice evaluation result is returned to the mobile client 100 , and the evaluation result is displayed through the mobile client 100 . Users can use the mobile Internet to easily access the server 200 to obtain services and data. The corpus and evaluation methods can be synchronized through the server 200, and the server 200 provides better performance and better effect of speech analysis algorithm processing. It is also possible to obtain the oral pronunciation quality evaluation statistical results of the mobile client users in real time from the database of the server end 200 through the webpage management terminal 300, and provide the oral pronunciation situation of the mobile client users for a third party (such as a teacher), so as to facilitate the third party to formulate Offline oral instruction and improvement strategies.

The online oral pronunciation quality evaluation method and system provided by the embodiments of the present invention can be applied to oral English learning to detect the pronunciation quality of spoken English. It can also be applied to the pronunciation quality evaluation of other languages, such as Japanese and French.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be realized by means of software plus necessary general-purpose hardware. Components, etc. to achieve. The essence of the technical solution of the present invention or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk , Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (9)

1. A method for evaluating the quality of online oral English pronunciation, characterized in that, comprising: Receive the test voice collected by the mobile client through the network; the mobile client is installed on the user's mobile phone or other mobile devices in the form of an application program, collect the test voice by calling the recording program in the mobile device, and test The voice is sent to the server through the Internet; The server side preprocesses the received test voice; Carry out the extraction of speech characteristic parameter to the test speech after pretreatment, obtain the characteristic parameter of described test speech; Evaluate the test speech according to the characteristic parameters of the test speech and the characteristic parameters of the standard speech, and obtain an evaluation result; Feedback the evaluation result to the mobile client through the network, and display the evaluation result through the mobile client; Described according to the characteristic parameter of described test speech and the characteristic parameter of standard speech, described test speech is evaluated, obtains evaluation result, comprises: According to the stress position characteristic parameter of the test speech and the stress position characteristic parameter of the standard speech, compare the stress position difference between the test speech and the standard speech, and calculate the test according to the stress position difference Speech stress score; The method further includes: guiding the user's spoken pronunciation according to the evaluation result, and obtaining pronunciation guidance; feeding back the pronunciation guidance to the mobile client through the network, and using the mobile client to The above pronunciation guidance is displayed. 2. online spoken language pronunciation quality evaluation method as claimed in claim 1, is characterized in that, described method also comprises: storing the evaluation results in a database, and performing statistical analysis on the evaluation results to obtain statistical results; The statistical results are sent to the webpage management terminal, and the statistical results are displayed through the webpage management terminal. 3. The online oral English pronunciation quality evaluation method as claimed in claim 1, is characterized in that, described method also comprises: Get standard voice; Preprocessing the standard voice; Extracting speech feature parameters from the preprocessed standard speech to obtain the feature parameters of the standard speech. 4. The method for evaluating the pronunciation quality of online spoken language according to any one of claims 1 to 3, wherein the preprocessing includes pre-emphasis, framing, windowing and endpoint detection. 5. the online spoken language pronunciation quality evaluation method as described in any one of claim 1～3, it is characterized in that, described test speech after preprocessing is carried out the extraction of speech feature parameter, obtains the feature parameter of described test speech, include: Carry out discrete Fourier transform to described test speech, obtain the spectral coefficient of described test speech, filter described spectral coefficient with sequence triangular filter, carry out logarithmic operation to the data after filtering, utilize discrete cosine transform, obtain described The MFCC characteristic parameter of test speech; Extract the fundamental frequency feature, short-term energy feature, and formant feature of the test speech, and form the emotional feature parameter of the test speech with the fundamental frequency feature, the short-term energy feature and the formant feature ; Calculating the pronunciation duration of the test voice, and obtaining the pronunciation duration characteristic parameter of the test voice; Carry out accent unit division to described test speech, extract the initial frame position group and the end frame position group of accent, obtain the stress position feature parameter of described test speech; Carry out speech unit division to described test speech, calculate the duration of each speech unit respectively, obtain the speech unit duration characteristic parameter of described test speech; The pitch of each frame of data of the test voice is extracted by an autocorrelation function method in the time domain, and the pitch characteristic parameter of the test voice is obtained. 6. The method for evaluating online spoken pronunciation quality as claimed in claim 5, wherein, the described test voice is evaluated according to the feature parameter of the test voice and the feature parameter of the standard voice, and obtains an evaluation result, including : According to the MFCC characteristic parameter of described test speech, based on the probabilistic neural network integrated speech recognition model of segmentation clustering, described test speech is carried out speech recognition, obtains speech recognition result; The MFCC feature parameter of described standard speech carries out similarity calculation, obtains MFCC correlation coefficient; According to described speech recognition result and described MFCC correlation coefficient, calculate the accuracy score of described test speech; According to the emotion characteristic parameter of described test speech, based on SVM emotion model, carry out emotion recognition to described test speech, obtain emotion recognition result; Calculate and obtain the emotion correlation coefficient; calculate the emotion score of the test speech according to the emotion recognition result and the emotion correlation coefficient; According to the pronunciation duration characteristic parameter of described standard speech and described test speech, obtain the speech speed ratio of described standard speech and described test speech, and according to described speech speed ratio, calculate the speech rate score of described test speech; According to the speech unit duration characteristic parameter of the test speech and the speech unit duration characteristic parameter of the standard speech, utilize the dPVI algorithm to obtain the dPVI parameter of the test speech, and calculate the test speech according to the dPVI parameter tempo score; According to the pitch characteristic parameter of the test speech and the pitch characteristic parameter of the standard speech, use the DTW algorithm to obtain the pitch difference between the standard speech and the test speech, and calculate according to the pitch difference The intonation score of the test speech. 7. online spoken language pronunciation quality evaluation method as claimed in claim 6, it is characterized in that, described according to the characteristic parameter of described test speech and the characteristic parameter of standard speech, evaluate described test speech, obtain evaluation result, also include: The accuracy score, the emotional score, the speed of speech score, the stress score, the rhythm score and the intonation score are weighted and summed to obtain a comprehensive score; and according to the accuracy score, the The emotion score, the speech rate score, the stress score, the rhythm score, the intonation score and the comprehensive score, combined with the mapping relationship between each score and grade evaluation, obtain the accuracy grade evaluation of the test voice , emotional grade evaluation, speech speed grade evaluation, accent grade evaluation, rhythm grade evaluation, intonation grade evaluation and comprehensive grade evaluation; and the accuracy grade evaluation, emotional grade evaluation, speech speed grade evaluation, stress grade evaluation , rhythm level evaluation, intonation level evaluation and comprehensive level evaluation as the evaluation results of the test speech. 8. An online oral pronunciation quality evaluation system is characterized in that, comprising a mobile client and a server connected through a network; The mobile clients include: The voice collection unit is used to collect the test voice, and send the test voice to the server through the network; the mobile client is installed on the user's mobile phone or other mobile devices in the form of an application program, and the mobile The recording program in collects the test voice, and sends the test voice to the server end through the Internet; The server side includes: A preprocessing unit is used to preprocess the received test voice; A characteristic parameter extraction unit is used to extract the speech characteristic parameters of the preprocessed test speech, and obtain the characteristic parameters of the test speech; The voice evaluation unit is used to evaluate the test voice according to the feature parameters of the test voice and the feature parameters of the standard voice to obtain an evaluation result; and feed back the evaluation result to the mobile client through the network; The mobile client also includes: a data display unit, configured to display the evaluation result; Described according to the characteristic parameter of described test speech and the characteristic parameter of standard speech, described test speech is evaluated, obtains evaluation result, comprises: According to the stress position characteristic parameter of the test speech and the stress position characteristic parameter of the standard speech, compare the stress position difference between the test speech and the standard speech, and calculate the test according to the stress position difference Speech stress score; The server end is also used to guide the user's spoken pronunciation according to the evaluation results, and obtain pronunciation guidance opinions; feed back the pronunciation guidance opinions to the mobile client through the network, and use the mobile client to The pronunciation guidance is displayed. 9. the online spoken language pronunciation quality evaluation system as claimed in claim 8, is characterized in that, described system also comprises webpage management end, and described webpage management end is connected with described server end by network; Described server end also comprises database and statistical analysis unit; The database is used to store the evaluation results; The statistical analysis unit is used to perform statistical analysis on the evaluation results to obtain statistical results; and send the statistical results to the webpage management terminal; The webpage management terminal is used to display the received statistical results.