CN103928023A - Voice scoring method and system - Google Patents

Abstract

The invention discloses a voice scoring method, comprising the steps of: S1, recording the examinee's examination paper voice; S2, preprocessing the examinee's examination paper voice to obtain the examination paper voice corpus; S3, extracting the characteristic parameters of the examination paper voice corpus ; S4, using the speech recognition method based on HMM and ANN hybrid model to carry out feature matching between the characteristic parameters of the speech corpus of the examination paper and the standard speech template, identify the content of the speech of the examination paper, and give a preliminary score; S5, if the preliminary score If it is lower than the threshold, the preliminary score will be the final score; otherwise, score indicators such as accuracy, fluency, speech rate, rhythm, stress and intonation; S6. Comprehensively calculate various scores to obtain the final score of the test paper voice. The invention also discloses a speech scoring system. The invention adopts the speech recognition method based on the mixed model, and the recognition is more accurate, and the speech test paper stored in the form of a file after the examinee is recorded can be objectively scored through the classification of the evaluation standard.

Description

A voice scoring method and system

technical field

The invention relates to speech recognition and evaluation technology, in particular to a speech scoring method and system.

Background technique

Speech recognition technology is usually divided into two categories from the perspective of application: one is specific person speech recognition, and the other is non-specific person speech recognition. Specific person speech recognition technology is a recognition technology for a specific person. Simply put, it only recognizes the voice of one person and is not applicable to a wider group; non-specific person recognition technology is just the opposite and can meet the speech recognition requirements of different people. , suitable for a wide range of people.

The IBM Speech Research Group, currently leading the way in large vocabulary speech recognition. AT&T's Bell Research Institute also started a series of experiments on neutral speech recognition, and the results established how to make standard templates for neutral speech recognition.

Significant advances made during this period include:

(1) The maturity and continuous improvement of Hidden Markov Models (HMM) technology has become the mainstream method of speech recognition;

(2) When performing continuous speech recognition, in addition to recognizing acoustic information, make more use of various language knowledge, such as word formation, syntax, semantics, dialogue background, etc., to help further recognize and understand speech; At the same time, in the field of speech recognition research, a language model based on statistical probability has also been produced;

(3) The rise of artificial neural network application research in speech recognition. In these studies, most of the multi-layer perceptual networks based on the backpropagation algorithm (BP algorithm) are used; in addition, there are forward networks with simple network structure, easy implementation, and no feedback signal; the stability of the system and the function of associative memory Feedback network with affinity and feedback between neurons. The ability of artificial neural networks to distinguish complex classification boundaries clearly contributes greatly to pattern division.

In addition, continuous voice dictation machine technology for personal use is also becoming more and more perfect. In this regard, the most representative ones are IBM's ViaVoice and Dragon's Dragon Dictate system. These systems have the ability of speaker adaptation, and new users do not need to train all vocabulary, and the recognition rate can be continuously improved during use.

The development of China's speech recognition technology: In Beijing, there are scientific research institutions and institutions of higher learning such as the Institute of Acoustics of the Chinese Academy of Sciences, the Institute of Automation, Tsinghua University, and Northern Jiaotong University. In addition, Harbin Institute of Technology, University of Science and Technology of China, and Sichuan University have also taken action. Now, many domestic speech recognition systems have been successfully developed. The performance of these systems has its own characteristics: in terms of speech recognition of isolated words and large vocabulary, the most representative one is the THED-919 real-time system for specific person speech recognition and understanding successfully developed by the Department of Electronic Engineering of Tsinghua University and China Electronic Devices Corporation; In terms of continuous speech recognition, the Computer Center of Sichuan University has implemented a subject-limited continuous English-Chinese speech translation demonstration system for a specific person on a microcomputer; in terms of non-specific person speech recognition, it has been developed by the Department of Computer Science and Technology of Tsinghua University. The voice-activated telephone directory system was put into practical use.

In addition, HKUST Xunfei, as China's largest intelligent voice technology provider, released the world's first mobile Internet intelligent voice interaction platform "Xunfei Voice Cloud" in 2010, announcing the arrival of the era of mobile Internet voice dictation.

In the field of intelligent voice technology, iFLYTEK has a long-term research accumulation, and has international leading achievements in many technologies such as Chinese speech synthesis, speech recognition, and speech evaluation: speech synthesis and speech recognition technology is the realization of human-machine voice communication, the establishment of Two key technologies necessary for a speech system capable of listening and speaking; the problem to be solved by the automatic speech recognition technology (Auto Speech Recognize, ASR) is to enable the computer to "understand" human speech, and convert the text contained in the speech The information is "extracted"; speech evaluation technology is a research frontier in the field of intelligent speech processing, also known as Computer Assisted Language Learning (Computer Assisted Language Learning) technology, which is a method that automatically scores pronunciation, detects errors and gives corrections through machines. Guidance technology; voiceprint recognition technology, also known as speaker recognition technology (SpeakerRecognition), is a technology that extracts relevant features representing the speaker's identity through speech signals (such as fundamental frequency characteristics that reflect the opening and closing frequency of the glottis, reflecting the size and shape of the oral cavity) Spectrum characteristics of vocal tract length, etc.), and then identify the speaker's identity and other working technologies; natural language has been an indispensable element in people's life, work, and study for thousands of years, and computers are the greatest in the 20th century. One of the inventions, how to use computers to process and even understand the natural language mastered by humans, so that computers can have human listening, speaking, reading and writing abilities, has always been a research work that domestic and foreign research institutions have paid close attention to and actively carried out.

Contents of the invention

The technical problem to be solved by the present invention is to provide a voice scoring method and system, which can quickly and accurately check and score examination papers, and score examinees with objective scoring standards. The present invention combines the advantages of the existing objective evaluation model of pronunciation quality, obtains a speech recognition model and a speech training model with better performance and a more accurate speech and spoken language scoring scheme; The voice test paper is objectively scored. The invention has the advantages of more stability and higher efficiency, lays a foundation for the practical application of research results, and is beneficial to realize the goal of automatic marking of large-scale oral English tests.

In order to solve the problems of the technologies described above, the invention provides a method for scoring voice, comprising steps:

S1, recording the examinee's examination paper voice;

S2. Preprocessing the speech of the examinee's examination paper to obtain the speech corpus of the examination paper;

S3, extracting the characteristic parameters of the speech corpus of the examination paper;

S4, using the speech recognition method based on the HMM and ANN hybrid model to perform feature matching on the characteristic parameters of the speech corpus of the examination paper and the standard speech template, identify the content of the speech in the examination paper, and give a preliminary score;

S5. If the preliminary score is lower than the preset threshold, the preliminary score is the final score of the test paper voice, and the test paper voice is marked as a question paper; if the preliminary score is higher than the preset threshold, the test paper voice is Scoring for accuracy, fluency, rate of speech, rhythm, stress and intonation;

S6. Perform weighted calculation on the scores of the sub-indices to obtain a final score of the voice of the examination paper.

Further, before the step S1, it also includes a step S0, and the step S0 specifically includes the steps of:

S01, recording the standard voice of the expert;

S02. Preprocessing the standard speech to obtain a standard speech corpus;

S03. Extracting characteristic parameters of the standard speech corpus;

S04. Perform model training on the characteristic parameters of the standard speech corpus to obtain the standard speech template.

Further, the specific steps of the speech recognition method based on the HMM and ANN mixed model in the step S4 are:

S41, establishing the HMM model of the characteristic parameters of the speech corpus of the examination paper, and obtaining the cumulative probability of all states in the HMM model;

S42. Process the cumulative probabilities of all states as input features of the ANN classifier, thereby outputting a recognition result;

S43. Perform feature matching on the recognition result and the standard speech template, so as to recognize the speech content of the examination paper.

Further, the preprocessing in the step S2 specifically includes pre-emphasis, framing, windowing, noise reduction, endpoint detection and word segmentation, wherein the specific step of the noise reduction is to use blank speech segments of speech as noise The base value denoises the subsequent speech.

Further, the word segmentation specifically includes steps:

S21, extracting the MFCC parameters of each phoneme in the speech, and establishing the HMM model of the corresponding phoneme;

S22. Roughly segment the speech to obtain effective speech segments;

S23. Recognize the words of the speech segment according to the HMM model of the phoneme, so as to recognize the speech as a word set.

Further, the extraction parameter feature in the step S3 is specifically to extract the MFCC feature parameter, and the specific steps are to perform fast Fourier transform, triangular window filter, logarithm, and discrete cosine transform on the corpus obtained after preprocessing to obtain the MFCC feature parameter.

Further, the specific steps of the accuracy scoring in the step S5 are:

The speech sentence to be scored is regularized to a degree close to the standard speech sentence by the method of extracting interpolation; short-term energy is used as a feature to extract the intensity curve of the speech sentence to be scored and the standard speech sentence; by comparing the speech sentence to be scored and the standard speech sentence The degree of fit to the intensity curve of standard speech sentences is scored.

Further, the specific steps of the fluency scoring in the step S5 are:

Cut the speech to be scored into two parts before and after, and segment the words of the first half and the second half to obtain an effective speech segment; divide the lengths of the effective speech segments of the two parts before and after the total length of the speech to be scored respectively, And compare the obtained value with the corresponding threshold, if both are greater than the corresponding threshold, it is judged as fluent; otherwise, it is judged as not fluent.

The specific steps of scoring speech speed are: calculating the ratio of the pronunciation part in the speech to be scored to the entire speech duration to be scored, and performing speech speed scoring according to the ratio.

The specific steps of rhythm scoring are: using the improved dPVI parameter calculation formula to calculate the rhythm of the speech to be scored.

The specific steps of stress scoring are as follows: on the basis of the regularized intensity curve, the stress unit is divided by setting the stress threshold and the non-stress threshold as the characteristic double threshold and the duration of the stressed vowel, and using the DTW algorithm to classify the speech sentences to be scored and the standard Speech sentences are pattern matched to realize stress evaluation.

The specific steps of scoring the intonation are: extracting the formants of the speech to be scored and the standard speech, and scoring the intonation according to the degree of fitting between the change trend of the formant of the speech to be scored and the change trend of the formant of the standard speech.

The present invention also provides a voice scoring system, comprising:

The voice recording module is used to record the examinee's examination paper voice;

The preprocessing module is used to preprocess the speech of the examinee's examination paper to obtain the speech corpus of the examination paper;

A parameter feature extraction module, used to extract the feature parameters of the speech corpus of the examination paper;

Speech recognition module, for adopting the speech recognition method based on HMM and ANN hybrid model to carry out feature matching to the characteristic parameter of described examination paper speech corpus and standard speech template, recognize the content of examination paper speech, and give preliminary scoring;

The speech scoring module is used to perform accuracy scoring, fluency scoring, speech rate scoring, rhythm scoring, stress scoring and intonation scoring on the speech of the test paper whose preliminary score is higher than the set threshold.

The comprehensive scoring module is used to calculate the score of comprehensive accuracy, fluency, speech rate, rhythm, stress and intonation to obtain the final score of the speech of the test paper whose preliminary score is higher than the set threshold.

Implement the present invention, have following beneficial effect:

1. The present invention adds practical noise reduction and word segmentation methods in the preprocessing module to obtain better quality speech corpus;

2. Using the speech recognition method based on the hybrid model of HMM and ANN, the performance is better and the recognition is more accurate;

3. Through the multi-indicator analysis of speech rate, rhythm, stress and intonation, the scoring indicators are more diversified than the original reading questions, and the results are more objective;

4. Through the dual analysis of accuracy and fluency, based on the original scoring of reading questions, the objective scoring of non-reading questions such as translation questions, questions and answers, and retelling questions has been established, and a reasonable and complete system has been established. The advanced voice scoring method and system can quickly and accurately mark and score the examination papers, and score candidates with objective scoring standards;

5. The present invention has the advantages of being more stable and efficient, and has strong practicability and a wide range of applications. It can be applied to the evaluation process of oral exams, greatly and effectively shorten the evaluation time, improve the efficiency of system processing, and improve The objectivity of evaluation.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic flow chart of the speech scoring method provided by the embodiment of the present invention;

Fig. 2 is a schematic flow chart of the specific steps of step S0;

Fig. 3 is the schematic flow chart of the specific steps of preprocessing in Fig. 1;

Fig. 4 is the schematic flow chart of the specific steps of word cutting among Fig. 3;

Fig. 5 is the schematic flow chart of the specific steps of MFCC feature parameter extraction;

Fig. 6 is the schematic flow chart of the specific steps of the speech recognition method based on HMM and ANN hybrid model;

Fig. 7 is a schematic structural diagram of a speech scoring system provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The embodiment of the present invention provides a kind of speech scoring method, as shown in Figure 1, comprises steps:

S1, recording the examinee's examination paper voice;

S2. Preprocessing the speech of the examinee's examination paper to obtain the speech corpus of the examination paper;

S3, extracting the characteristic parameters of the speech corpus of the examination paper;

S4, using a speech recognition method based on a hybrid model of Hidden Markov Models (HMM) and Artificial Neural Networks (ANN) to perform feature matching on the feature parameters of the speech corpus of the test paper and the standard speech template, Recognize the content of the voice in the examination paper and give a preliminary score;

S5. If the preliminary score is lower than the preset threshold, the preliminary score is the final score of the test paper voice, and the test paper voice is marked as a question paper; if the preliminary score is higher than the preset threshold, the test paper voice is Scoring for accuracy, fluency, rate of speech, rhythm, stress and intonation;

S6. Perform weighted calculation on the scores of the sub-indices to obtain the final score of the test paper voice.

Further, the step S1 also includes a step S0, as shown in FIG. 2, the step S0 specifically includes the steps:

S01, recording the standard voice of the expert;

Among them, the standard voice is recorded by most professionals in a specific environment, and the content of the voice corresponds to the content of the oral English test;

S02. Preprocessing the standard speech to obtain a standard speech corpus;

S03. Extracting characteristic parameters of the standard speech corpus;

S04. Perform model training on the characteristic parameters of the standard speech corpus to obtain the standard speech template.

Among them, the standard speech model training refers to obtaining model parameters representing the essential characteristics of the pattern from a large number of known patterns according to certain criteria, that is, standard speech templates. The process of the model training specifically refers to in order to make the speech recognition system reach a certain optimal state, by continuously iteratively adjusting the parameters of the system template (comprising the probability of the state transition matrix and the variance of the Gaussian mixture model, mean value, Weights, etc.), the process of making the performance of the system continuously approach this optimal state. Since the standard pronunciation of professionals differs to a certain extent from the pronunciation of examinees, and the scoring object of the present invention is a natural person, the present invention will strive to expand the corpus, from specific professionals to ordinary people, and from specific environments to ordinary people. environment, and contains the voices of speakers of different genders, ages, and accents.

Each step will be described in detail next.

1. Pretreatment

As shown in Figure 3, the preprocessing in step S2 specifically includes noise reduction, pre-emphasis, framing, windowing, endpoint detection, and word segmentation. The impact of equipment on the quality of speech signals provides high-quality parameters for speech feature extraction, thereby improving the quality of speech processing.

Wherein, the specific step of the noise reduction is to use the blank speech segment of the speech as the base value of the noise to perform denoising processing on the subsequent speech, because according to the research, when the examinee is recording the recording, usually in a short period of time at the beginning There is no sound, but this short recording is not blank, but a recording segment with noise. Therefore, by extracting the audio of this recording segment as the base value of the noise, a noise removal process can be performed on the subsequent recording, and the noise interference of the silent segment is also excluded.

Among them, the word cutting refers to cutting a sentence into individual words or phrases, so that the computer can "understand" the candidate's expression content by recognizing individual words or phrases, and perform corresponding bonus points or phrases for the computer in the later stage. Analysis of deduction factors and preparation for final automatic scoring. As shown in Figure 4, the word segmentation specifically includes steps:

S21, extract the Mel frequency cepstrum coefficient (Mel Frequency Cepstrum Coefficient, MFCC) parameter of each phoneme in the speech, and set up the HMM model of corresponding phoneme;

S22. Roughly segment the speech to obtain effective speech segments;

The purpose of rough segmentation is twofold: one is to reduce the amount of computation, thereby reducing the time for word segmentation; the other is to increase the accuracy of word segmentation. Regarding the rough classification, the double-threshold method is used to intercept the obvious blank places, but the threshold value used is lower, the purpose is to obtain effective speech segments;

S23. Recognize the words of the speech segment according to the HMM model of the phoneme, so as to recognize the speech as a word set.

This word segmentation method has the advantages of high recognition rate, high accuracy rate, and small error: 1) The number of recognition templates is fixed, and for the HMM model, the accuracy rate is very high; and there is no need to set the threshold of the output probability, This will greatly improve the recognition rate. 2) After the word is segmented, the pronunciation of the word is obtained, and the pronunciation can assist in the matching of keywords, thereby reducing the error caused by matching words.

2. Extract parameter features

The extraction feature parameter in the described step S3 is specifically to extract the MFCC feature parameter, as shown in Figure 5, the specific steps are to carry out fast Fourier transform, triangular window filter, logarithm, discrete cosine transform to the corpus obtained after preprocessing Get the MFCC characteristic parameters. Among them, the MFCC characteristic parameters are used because it takes into account the auditory characteristics of the human ear, and converts the spectrum into a nonlinear spectrum based on the Mel frequency, and then converts it into the cepstrum domain. Moreover, without any assumptions, mathematical methods are used to simulate the auditory characteristics of the human ear, and a series of triangular filters arranged in the low frequency area are used to capture the spectral information of speech; in addition, the anti-noise ability and anti-noise of MFCC characteristic parameters The spectrum distortion ability is strong, which can better improve the recognition performance of the system.

3. Speech content recognition

In the step S4, a speech recognition method based on a hybrid model of HMM and ANN is adopted, wherein the HMM method requires prior statistical knowledge of speech signals, weak classification and decision-making ability, complex structure, requires a large number of training samples and requires a large number of calculations. Disadvantages: Although ANN has certain advantages in decision-making ability, its ability to describe dynamic time signals is not satisfactory, and the speech recognition algorithm based on neural network has the shortcomings of too long training and recognition time. In order to overcome their respective shortcomings, the present invention organically combines the two methods of HMM with strong time modeling ability and ANN with strong classification ability, and further improves the robustness and accuracy of speech recognition. This method not only overcomes the overlapping problem between pattern categories that HMM itself is difficult to solve, improves the ability to recognize confusing words, but also overcomes the limitation that ANN can only handle fixed-length input patterns, eliminating the need for complex regression whole operation. Specifically, as shown in Figure 6, the specific steps of the speech recognition method based on the HMM and ANN hybrid model in the step S4 are:

S41, establishing the HMM model of the characteristic parameters of the speech corpus of the examination paper, and obtaining the cumulative probability of all states in the HMM model;

S42. Process the cumulative probabilities of all states as the input features of an ANN (specifically, a self-organizing neural network) classifier, thereby outputting a recognition result;

S43. Perform feature matching on the recognition result and the standard speech template, so as to recognize the speech content of the test paper.

4. Voice evaluation

Because in daily life, some candidates are not able to take the oral test well at the stipulated time, and the resulting test paper voices will have a lot of blanks or cannot be recognized. We mark these test paper recordings as question papers. Question papers include blank recordings and various unrecognizable audio recordings, such as recordings in non-English languages, recordings with excessive noise, etc. The purpose of step S4 is not only to identify what the examinee read, but also to detect the question papers. And according to the actual situation, a lower score is given, and it is not necessary to score the accuracy, fluency, speech rate, rhythm, stress and intonation of this kind of question paper. Further speech evaluation is performed only if the initial score is above a pre-set threshold.

(1) The specific steps of the accuracy scoring in the step S5 are: using the method of interpolation to regularize the speech sentence to be scored to a degree close to the standard speech sentence; using short-term energy as a feature to extract the speech to be scored The intensity curve of the sentence and the standard speech sentence; scoring is carried out by comparing the fitting degree of the strength curve of the speech sentence to be scored and the standard speech sentence.

The intensity graph of the sentence can reflect the change of the speech signal over time. The characteristic of loud stressed syllables in the sentence will be reflected in the energy intensity in the time domain, that is, the stressed syllables have high speech energy intensity. However, since different people pronounce the same sentence at different times and at different times, and the pronunciation intensity is also different, if the intensity curves of the speech sentences to be scored and the standard speech sentences are directly used for template matching, the result will affect the objectivity of the evaluation. Therefore the present invention revises a kind of strength curve extracting method based on standard speech sentence on the basis of prior art: when the duration of the speech sentence to be scored is shorter than the standard speech sentence, an interpolation method is used to supplement the duration; When the duration of the rated speech sentence is longer than that of the standard speech sentence, the value extraction method is used to adjust its duration; finally, the intensity curve of the speech sentence to be rated is adjusted by using the strongest point of the intensity curve of the standard speech sentence.

(2) The specific steps of fluency scoring are as follows: the speech to be scored is cut into two parts before and after, and the words are cut into the first half and the second half to obtain an effective speech segment; The length of the total speech to be scored is divided, and the obtained value is compared with the corresponding threshold. If both are greater than the corresponding threshold, it is judged as fluent; otherwise, it is judged as not fluent;

For sentence-level fluency, it aims to calculate the smoothness of sentence expression and the prosody score of pronunciation by using standard speech, and the two are fused to obtain a sentence fluency diagnostic model. This method of sentence fluency scoring can also be applied to discourse fluency scoring. This method takes into account the fluency of the speaker in the process of expressing sentences, and has a higher degree of correlation than the traditional method. Therefore, it can be applied to the speech scoring system.

(3) The specific steps of scoring speech speed are: calculating the ratio of the pronunciation part in the speech to be scored to the entire speech duration to be scored, and scoring the speech speed according to the ratio.

(4) The specific steps of scoring the rhythm are as follows: Calculate the rhythm of the speech to be scored using the improved formula for calculating the Distinct Pairwise Variability Index (dPVI) parameters. According to the characteristics of the differences in the duration of speech units, dPVI compares and calculates the duration of syllable unit segments of standard speech sentences and speech sentences with scores, and uses the converted parameters for objective evaluation and feedback guidance.

$\mathrm{<span\; class="notranslate">\; PPVI</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 100</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; Len</span>}}_{\mathrm{<span\; class="notranslate">\; Std</span>}}$

Wherein d is the speech unit segment duration of sentence division (as: d _k is the kth speech unit segment duration), when m=min (i.e. the standard speech sentence unit number, the speech sentence unit number to be scored), Len _Std is the standard speech Statement duration. Since the length of speech sentences to be scored has been adjusted to be equivalent to the length of standard speech sentences before the PVI calculation, only Len _Std can be used as the calculation unit in the calculation.

(5) The specific steps of stress scoring are as follows: on the basis of the regularized intensity curve, the stress unit is divided by setting the stress threshold and the non-stress threshold as the characteristic double threshold and the duration of the stressed vowel, and using Dynamic Time Warping (Dynamic Time Warping, DTW) algorithm carries out pattern matching to described speech sentence to be scored and standard speech sentence, realizes the scoring of stress.

Stress refers to the stressed sound in words, phrases, and sentences. The basic principle of the DTW algorithm is dynamic time warping, which matches the unmatched time length between the test template and the reference template. Use the traditional Euclidean distance to calculate its similarity, set the reference template and test template as R and T, the smaller the distance D[T, R], the higher the similarity. The disadvantage of the traditional DTW algorithm is that when performing template matching, all frames have the same weight, and all templates must be matched, and the amount of calculation is relatively large, especially when the number of templates increases rapidly, the amount of calculation increases very quickly. Therefore, the present invention adopts the improved DTW algorithm to carry out the pattern matching of speech sentences to be rated and standard speech sentences, which improves the shortcomings of the traditional DTW algorithm. The weight of each frame is emphasized, which greatly reduces the amount of calculation and makes the results more accurate.

(6) The specific steps of scoring the intonation are: extracting the formants of the voice to be scored and the standard voice, and scoring the intonation according to the degree of fitting between the formant of the voice to be scored and the formant of the standard voice.

Intonation is an important indicator of people's ability to express language in oral English communication, and it is a reflection of the overall momentum of the speaker's discourse use.

In the research of speech digital signal processing, the formant of speech signal is a very important performance parameter. The formant mentioned here refers to some areas where the energy is relatively concentrated in the frequency spectrum of the sound. The formant is not only the determinant of sound quality, but also reflects the physical characteristics of the vocal tract (resonant cavity). When the sound passes through the resonant cavity, it is filtered by the cavity, which redistributes the energy of different frequencies in the frequency domain. Part of it is strengthened by the resonance of the resonant cavity, and the other part is attenuated. Those frequencies that are strengthened are analyzed in time-frequency It appears as thick black stripes on the language map. Due to the uneven distribution of energy, the strong part is like a mountain peak, so it is called a resonance peak. Formant is an important feature that reflects the resonance characteristics of the vocal tract. It represents the most direct source of pronunciation information, and people use formant information in speech perception, so formant is a very important characteristic parameter in speech signal processing. A formant is a set of resonant frequencies produced when quasi-periodic pulse excitation enters the vocal tract. Formant parameters include formant frequency and frequency bandwidth, which are important parameters to distinguish different finals. The formant information is included in the frequency envelope, so the key to extracting formant parameters is to estimate the spectrum envelope of natural speech. It is generally believed that the maximum value in the spectrum envelope is the formant.

The present invention also provides a voice scoring system, as shown in Figure 7, comprising:

Voice recording module 101, for recording examinee's examination paper voice;

The preprocessing module 102 is used to preprocess the examination paper voice of the examinee to obtain the examination paper voice corpus;

Parameter feature extraction module 103, for extracting the feature parameter of described examination paper voice corpus;

Speech recognition module 104, for adopting the speech recognition method based on HMM and ANN hybrid model to carry out feature matching to the characteristic parameter of described examination paper speech corpus and standard speech template, recognize the content of examination paper speech, and give preliminary scoring;

The speech scoring module 105 is used to perform accuracy scoring, fluency scoring, speech rate scoring, rhythm scoring, stress scoring and intonation scoring on the speech of the test paper whose preliminary score is higher than the set threshold.

The comprehensive scoring module 106 is used for comprehensive score calculation of accuracy, fluency, speech rate, rhythm, stress and intonation to obtain the final score of the test paper voice whose preliminary score is higher than the set threshold.

Wherein, the speech scoring system and the speech scoring method correspond to each other, so the specific processing steps of each module can refer to the steps of the speech scoring method, and will not be described again.

Implement the present invention, have following beneficial effect:

(1) the present invention has added practical denoising and word cutting method in the preprocessing module, obtains the better speech corpus of quality;

(2) Using the speech recognition method based on the HMM and ANN hybrid model, the performance is better and the recognition is more accurate;

(3) Through the multi-indicator analysis of speech rate, rhythm, stress and intonation, the scoring indicators are more diversified than the original reading questions, and the results are more objective;

(4) Through the dual analysis of accuracy and fluency, on the basis of the original scoring of reading questions, the objective scoring of non-reading questions such as translation questions, question-and-answer questions, and retelling questions has been realized, and a reasonable score has been established. The perfect voice scoring method and system can quickly and accurately mark and score the examination papers, and score candidates with objective scoring standards;

(5) The present invention has the advantages of being more stable and efficient, has strong practicability, and has a wide range of applications. The objectivity of evaluation and reform has been improved.

The above disclosure is only a preferred embodiment of the present invention, which certainly cannot limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims (10)

1. a speech assessment method, is characterized in that, comprises step:

S1, record examinee's examination paper voice;

S2, described examinee's examination paper voice are carried out to pre-service, obtain examination paper voice language material;

S3, extract the characteristic parameter of described examination paper voice language material;

The characteristic parameter of described examination paper voice language material and received pronunciation template are carried out characteristic matching by S4, the audio recognition method of employing based on HMM and ANN mixture model, identifies the content of described examination paper voice, and give raw score;

If S5 raw score is lower than presetting threshold value, described raw score is the final scoring of these examination paper voice, and these examination paper voice of mark are problem volume; If raw score is higher than presetting threshold value, described examination paper voice are carried out to point index scoring of accuracy, fluency, word speed, rhythm, stress and intonation;

S6, to described point of index, scoring is weighted the final scoring that obtains described examination paper voice.

2. speech assessment method as claimed in claim 1, is characterized in that, before described step S1, also comprises step S0, and described step S0 specifically comprises step:

S01, record expert's received pronunciation;

S02, described received pronunciation is carried out to pre-service, obtain received pronunciation language material;

S03, extract the characteristic parameter of described received pronunciation language material;

S04, the characteristic parameter of described received pronunciation language material is carried out to model training, obtain described received pronunciation template.

3. speech assessment method as claimed in claim 1, is characterized in that, the concrete steps of the audio recognition method based on HMM and ANN mixture model in described step S4 are:

S41, set up the HMM model of the characteristic parameter of described examination paper voice language material, obtain all state cumulative probabilities in HMM model;

S42, described all state cumulative probabilities are processed as the input feature vector of ANN sorter, thus output recognition result;

S43, described recognition result and described received pronunciation template are carried out to characteristic matching, thereby identify the content of described examination paper voice.

4. speech assessment method as claimed in claim 1, it is characterized in that, pre-service in described step S2 specifically comprises noise reduction, pre-emphasis, point frame, windowing, end-point detection and cuts word, wherein, the concrete steps of described noise reduction are to adopt the blank voice segments of voice, as the base value of noise, subsequent voice is carried out to denoising.

5. speech assessment method as claimed in claim 4, is characterized in that, described in cut word and specifically comprise step:

S21, extract the MFCC parameter of each phoneme in voice, and set up the HMM model of corresponding phoneme;

S22, voice are carried out to rough lumber divide, obtain effective voice segments;

S23, go out the word of described voice segments according to the HMM Model Identification of described phoneme, thereby be set of letters by speech recognition.

6. speech assessment method as claimed in claim 1, it is characterized in that, extracting parameter feature in described step S3 is specially extracts MFCC characteristic parameter, and concrete steps are that the language material obtaining after pre-service is carried out to Fast Fourier Transform (FFT), quarter window filtering, asks logarithm, discrete cosine transform to obtain MFCC characteristic parameter.

7. speech assessment method as claimed in claim 1, is characterized in that, the accuracy scoring concrete steps in described step S5 are:

The method of the employing value of pulling and pushing by regular speech sentences to be marked to the degree close with received pronunciation statement; Adopt short-time energy extract as feature described in the intensity curve of speech sentences to be marked and received pronunciation statement; The fitting degree of the intensity curve by speech sentences relatively to be marked and received pronunciation statement is marked.

8. speech assessment method as claimed in claim 1, is characterized in that, the fluency scoring concrete steps in described step S5 are:

Two parts before and after voice to be marked are cut into, thereby and word cut in first part and latter part obtain efficient voice section; The length of two-part front and back efficient voice section is made to division operation with the length of voice always to be marked respectively, and by the value obtaining and corresponding threshold, if be greater than corresponding threshold value, it is fluent to be judged to be; Otherwise it is unfluent to be judged to be.

9. speech assessment method as claimed in claim 1, is characterized in that, in described step S5

Word speed scoring concrete steps are: calculate in voice to be marked to pronounce and partly account for the ratio of voice duration whole to be marked, according to described ratio, word speed is marked;

Rhythm scoring concrete steps are: adopt improved dPVI parameter calculation formula to calculate the rhythm of voice to be marked;

Stress scoring concrete steps are: on the intensity curve basis after regular, by being set, stress threshold value and non-stress threshold value divide stress unit as double threshold and the stressed vowel duration of feature, and adopt DTW algorithm to carry out pattern match to speech sentences described to be marked and received pronunciation statement, realize the scoring of stress;

Intonation scoring concrete steps are: extract the resonance peak of voice to be marked and received pronunciation, and according to the fitting degree of the variation tendency of the variation tendency of speech resonant peak described to be marked and received pronunciation resonance peak, intonation is marked.

10. a speech assessment system, is characterized in that, comprising:

Voice recording module, for recording examinee's examination paper voice;

Pretreatment module, carries out pre-service for the examination paper voice to described examinee, obtains examination paper voice language material;

Characteristic parameter extraction module, for extracting the characteristic parameter of described examination paper voice language material;

Sound identification module, carry out characteristic matching for adopting audio recognition method based on HMM and ANN mixture model characteristic parameter and the received pronunciation template to described examination paper voice language material, identify the content of examination paper voice, and give raw score and mark whether as problem volume;

Speech assessment module, for carrying out accuracy scoring, fluency scoring, word speed scoring, rhythm scoring, stress scoring and intonation scoring for raw score higher than the non-problem examination paper voice that preset threshold value.

Comprehensive grading module, obtains the final scoring of raw score higher than the examination paper voice of setting threshold for the score calculation of overall accuracy, fluency, word speed, rhythm, stress and intonation.