CN108074585A - A kind of voice method for detecting abnormality based on sound source characteristics - Google Patents

Abstract

The invention discloses a speech abnormality detection method based on sound source characteristics, comprising the following steps: collecting speech data in real time through a sensor; preprocessing the obtained speech segment data; using iterative self-adaptive inverse filtering for the speech segment speech data Obtain the glottal wave signal; extract characteristic parameters from the glottal wave signal: normalize the amplitude quotient and glottis closure time ratio data; input the extracted characteristic data into the ideal SVM model for classification; obtain classification labels to judge speech Speaker status, output the speaker status label, and send it to the execution module for feedback. The feature of the present invention is that, for the variant speech under mental stress, it gets rid of the recognition method based on the traditional linear speech generation model and extracts the acoustic characteristic parameters lacking physical meaning, establishes a sound source estimation model, and uses the inverse filtering technology of speech generation , analyze and extract the characteristic parameters based on the vibration of human vocal cords to detect abnormal speech.

Description

A Speech Anomaly Detection Method Based on Sound Source Features

technical field

The invention relates to a voice abnormality detection method based on sound source characteristics, and belongs to the technical field of intelligent voice.

Background technique

Stress is the body's natural response to physical, mental, or emotional stimuli. When we are exposed to these stimuli, the brain releases alcohols and peptides into the body, causing a stress response. This continuous anxiety about work will be reflected on the vocal organs, which will cause changes in a series of parameters such as vocal frequency and pronunciation speed. These changes are of great significance in many fields of speech signal processing, such as variant speech recognition, emotion recognition, etc.

An important manifestation of stress is the voice of the speaker when speaking, which has become a very important factor affecting the production of voice. When the surrounding environment or the speaker's own conditions change abnormally, or because the user is mostly focused on a certain job, speech recognition is only a secondary job that assists other jobs. In this process, due to the existence of work pressure, speaking When a person is under mental stress, the interlocutor's pronunciation will be greatly affected, resulting in an abnormal state and voice variation, and the abnormal state is often reflected in the speaker's voice, forming a speech signal under abnormal stress.

However, the variant speech under mental stress, especially the variant speech under multi-task brain load pressure, has a relatively low degree of auditory discrimination, and the general acoustic features cannot classify it correctly, lacking stability and robustness. In addition, during the generation process of variant speech, its sound source features are significantly different from normal speech. Therefore, in the detection process, we improve the reliability of variant speech classification through sound source features. By improving the identification efficiency of variant speech, it lays the foundation for the strong robustness of the speech recognition system.

Contents of the invention

The problem to be solved by the present invention is to detect the pressure state from the perspective of the sound source of speech generation, and propose a stress detection method based on speech generation modeling. The feature of the present invention is to get rid of the recognition method based on the traditional linear speech generation model and the acoustic characteristic parameters lacking physical meaning, establish a sound source estimation model, and use the inverse filtering technology of speech generation to analyze and extract the characteristics based on the vibration of human vocal cords parameters to detect abnormal speech.

Technical scheme of the present invention is as follows:

A method for abnormal speech detection based on sound source features, comprising the steps of:

(1) Real-time collection of voice data through sensors;

(2), judge the voice segment and the noise segment of voice data by endpoint detection, to decide whether to carry out the next step voice signal processing work;

(3), windowing the speech data of the speech segment obtained, and carrying out high-frequency pre-emphasis processing to each frame;

(4), for the voice data of the voice segment, use iterative adaptive inverse filtering to obtain the glottal wave signal;

(5), extracting the characteristic parameter normalized amplitude quotient of glottal wave and glottis closure time ratio data;

(6), inputting the extracted data into the trained SVM model for classification;

(7) Obtain classification labels for judging the status of the speaker, output the status label of the speaker, and send it to the execution module for feedback.

The windowing in the above step (3) uses a Hamming window to window a frame of speech.

In the above step (3), the high-frequency pre-emphasis processing uses a first-order finite excitation response high-pass filter to enhance its high-frequency part.

The steps of obtaining the glottal wave signal in the above-mentioned steps (4) are as follows:

(a), using iterative adaptive inverse filtering to establish a vocal tract model;

Iterative adaptive inverse filtering eliminates the influence of glottal excitation in the original speech signal spectrum;

(b), then eliminate the influence of the formant by the method of inverse filtering;

Acoustic models are accurately built by linear predictive coding and discrete ensemble models, and finally inverse filtering is used to obtain glottal wave signals.

The extraction method of the normalized amplitude quotient of the characteristic parameter of glottal wave in the above-mentioned steps (5) is as follows:

In the formula, NAQ is the normalized amplitude quotient; T is the pitch period; AQ is the amplitude quotient, which is the ratio of the maximum amplitude of the glottal wave to the maximum negative peak of its corresponding first-order derivative;

Where f _ac is the maximum peak value of the glottal pulse; dpeak is the maximum negative peak value of the first derivative of the glottal pulse.

The extraction method of the glottis closure time ratio data in the above step (5) is as follows:

In the formula, CPR is the ratio of glottis closure time; CP is the glottis closure phase; O is the total opening time of the glottis.

The beneficial effect that the present invention reaches:

Based on the study of the change characteristics of the vocalization physiological system under the influence of pressure, the present invention studies the internal relationship between the physiological characteristics and the sound source parameters, and finds out the important factors that can reflect the pressure state in the glottal wave characteristics, so that the obtained The glottal wave parameters not only have theoretical guidance, but also have clear physical meaning; find out the glottal wave parameters that can describe the characteristics of the pressure-related sound source in the vocal system, and establish the internal relationship between the sound source characteristics and physiological characteristics, so as to The feature is used to identify the correlation with the pressure variation factor, which marks the vibration mode of the vocal cords and has physical meaning. It is finally used to detect the abnormal state of speech and improve the accuracy and reliability of the speech recognition system.

The present invention can be applied to the in-vehicle environment, by detecting the voice data of the driver and passengers to judge their stress state, feeding back the state information to the execution module through the transmission device, and then the execution module automatically takes effective measures such as reminding the driver to pay attention to safety , Use the Internet of Vehicles to notify nearby vehicles to pay attention to avoiding, etc., so as to achieve the purpose of protecting life and property safety.

Description of drawings

Fig. 1 is a basic flow chart of the present invention;

Figure 2 is a basic flowchart of obtaining the SVM classification model.

Fig. 3 is the iterative adaptive inverse filtering (IAIF) technical structural diagram that the present invention establishes;

Fig. 4 is the ROC curve of five kinds of parameters in embodiment 1;

Fig. 5 is each parameter value of the ROC curve of five kinds of characteristics in embodiment 1, and wherein AUC is the area under the curve, SE is the standard deviation; CL is the confidence interval;

Fig. 6 is the average recognition rate of the classifier obtained after 50 rounds of experiments in Example 1.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solution of the present invention more clearly, but not to limit the protection scope of the present invention.

As shown in Figure 1, a method for abnormal speech detection based on sound source features includes the following steps:

(1) Real-time collection of voice data through sensors;

(2), judge the voice segment and the noise segment of voice data by endpoint detection, to decide whether to carry out the next step voice signal processing work;

The invention uses a speech endpoint detection method based on energy and short-term zero-crossing rate to effectively distinguish speech segments. The above methods are all existing mature detection methods, and will not be elaborated here.

(3), windowing the speech data of the speech segment obtained, and carrying out high-frequency pre-emphasis processing to each frame;

Pre-emphasis: The average power spectrum of the voice signal is affected by glottal excitation and mouth and nose radiation. The high-frequency end is attenuated by 6dB/oct (octave) above 800Hz. The higher the frequency, the smaller the corresponding component. Before the speech signal is analyzed, its high-frequency part is enhanced by a first-order finite excitation response high-pass filter.

Framing: Due to the short-term stationarity of the speech signal, we can process the signal in frames. From a macro point of view, it must be short enough to ensure that the signal within the frame is stable, that is, the length of a frame should be less than the length of a phoneme. At a normal speech rate, the duration of a phoneme is about 50-200 milliseconds, so the frame length is generally less than 50 milliseconds. From a microscopic point of view, it must include enough vibration periods, because the Fourier transform is to analyze the frequency, and the frequency can only be analyzed if it is repeated enough times. The fundamental frequency of speech is around 100 Hz for male voices and around 200 Hz for female voices, and the converted cycles are 10 milliseconds and 5 milliseconds. Since a frame contains multiple cycles, it is generally at least 20 milliseconds.

Windowing: The Hamming window is used to window a frame of speech. It not only has better frequency resolution, but also reduces spectrum leakage, thereby reducing the influence of the Gibbs effect.

(4), as shown in Figure 3, for the speech data of speech segment, use iterative adaptive inverse filter to obtain glottal wave signal;

(a), using iterative adaptive inverse filtering (IAIF) to establish a vocal tract model;

Iterative adaptive inverse filtering eliminates the influence of glottal excitation in the original speech signal spectrum;

(b), then eliminate the influence of the formant by the method of inverse filtering (IF);

Accurately build the acoustic model through linear predictive coding (LPC) and discrete ensemble model (DAP), and finally use inverse filtering (IF) to obtain the glottal wave signal, as shown in Figure 2.

(5), extracting the characteristic parameter normalized amplitude quotient of glottal wave and glottis closure time ratio data;

Irregularity of the vibration of the vocal cords due to contraction of the vocal cord muscles during work stress. As a result, the airflow state in the glottis changes, and the speech signal changes. These changes in the characteristics of the vocal cords will be reflected in the characteristics of the glottal waves, thus making the glottal waves reflect the working pressure to some extent. We use normalized amplitude quotient (NAQ) and glottal closure time ratio (CPR) to characterize the essential characteristics of glottal waves. The proposed features have clear physical meanings and reflect the different vibration modes of the vocal cords during speech production.

The first characteristic parameter of the glottal wave: the normalized amplitude quotient, which mainly reflects the closing mode of the vocal cords, and its extraction method is as follows:

In the formula, NAQ is the normalized amplitude quotient; T is the pitch period; AQ is the amplitude quotient, which is the ratio of the maximum amplitude of the glottal wave to the maximum negative peak of its corresponding first-order derivative;

Where f _ac is the maximum peak value of the glottal pulse; dpeak is the maximum negative peak value of the first derivative of the glottal pulse. Since the moment when the glottis opens or closes does not need to be measured, AQ is relatively easy to obtain, but the value of AQ depends on the measurement of the fundamental frequency (F0) of the signal. Therefore, in formula (1), by normalizing the pitch period, the Get NAQ, eliminating dependence on fundamental frequency measurements.

The second characteristic parameter of glottal wave: glottis closure time ratio (CPR). The CPR parameters reflect the ratio of the glottis closing phase to the total glottis opening time, and the glottal wave mainly shows the skewness of the glottis signal. .

The extraction method of glottis closure time ratio data is as follows:

In the formula, CPR is the ratio of glottis closure time; CP is the glottis closure phase; O is the total opening time of the glottis.

(6), inputting the extracted data into the trained SVM model for classification;

Support vector machine (SVM) has always played an important role in the field of pattern recognition [8]. The so-called support vector refers to those training sample points on the edge of the interval area. SVM uses linear and non-linear hyperplanes for classification. SVM is based on the VC dimension theory of statistical learning theory and the principle of structural risk minimization, according to the complexity of the model (that is, the learning accuracy of a specific training sample) and the learning ability (that is, to identify any The ability of the sample) to seek the best compromise in order to obtain the best generalization ability. Support vector machine is essentially a second-class nonlinear classifier, which is very suitable for the unique recognition characteristics of mutated speech: (1) Since the speaker is not in a state of stress all the time during the vocalization process, stress manifests itself in continuous speech Short transient, so only a small number of samples can be defined as variant speech under pressure, so variant speech recognition is generally a small-sample problem. (2) A typical second-class recognition problem for variant speech recognition of emotional stress. We established a classification recognition model based on SVM. In the case of speaker correlation, since the number of samples of each tested speaker is relatively small, it is a typical small sample problem, so in this case, the SVM model has achieved better recognition effect.

The invention uses the SVM classification model to identify and classify the variant speech and the speech in the normal state, and realize the evaluation of the sensitivity of the sound source parameters to the variant state in the proposed method, thereby verifying the effectiveness of the proposed method.

(7) Obtain classification labels for judging the status of the speaker, output the status label of the speaker, and send it to the execution module for feedback.

Example 1

We used a database collected by Fujitsu Corporation containing speech samples of 11 speakers (4 male and 7 female). In order to simulate the specific situation of psychological stress generation, three different tasks were set up for the speaker, carried out during the telephone conversation with the operator, to simulate the situation of stress in the telephone.

The three tasks involved were (A) highly focused (the speaker was asked to complete tasks that included solving logic puzzles and finding the difference between two pictures); (B) time pressure (the speaker was asked to answer questions under time pressure); ( C) Risky Behavior (taking risky tasks to assess the speaker's desire for monetary gain). For each speaker, there are four dialogues with different tasks. In two dialogues, the speakers were asked to complete tasks within a limited time, while in the other dialogues, there were no tasks and chatting was easy.

The parts taken from the speech are the vowels /a/, /i/, /u/, /e/, /o/. These experiments were performed for each speaker, and all results were determined by speaker. The experiment is carried out in the loudspeaker system with 11 subjects selected, the number of samples depends on the speaker, and the total number of speech samples is 700.

In the present invention, the verification data used are all from telephone communication data, wherein 100 subjects (50 males and 50 females) participated in the experiment. In the experiment, the operator chatted with each subject over the phone, with an average of four conversations per person, and each chat lasted 10 minutes, and recorded the most authentic voice communication data. Among the four groups of conversations, two groups were casual chatting in a relaxed state, and in the other two groups of conversations, the subjects were subjected to different types of pressure, including: (1) multi-tasking; (2) tight time; ( 3) Risky speculation, see Table 1 for details. The real voice data of the subjects speaking under stress was recorded to verify the effectiveness of the stress detection method.

Table 1

In order to verify the effectiveness of the proposed method, the present invention uses the receiver operating characteristic curve (ROC) to evaluate the recognition performance of different parameters, as shown in Figure 4 and Figure 5, the ROC curve is based on a series of different two classification methods (cut-off value or decision threshold), a curve drawn with the true positive rate (sensitivity) as the ordinate and the false positive rate (1-specificity) as the abscissa. The closer the ROC curve is to the upper left corner, the larger the area under the curve (AUC), which means the better the recognition performance of the method and the higher the accuracy.

True Positive (TPR):

False Positive (FPR):

TP: true positive; TN: true negative; FP: false positive; FN: false negative

The present invention compares the parameters of the proposed sound source model with the traditional parameters, and compares the average recognition rate of pressure detection, indicating that the method based on voice generation modeling has obvious advantages in the pressure detection method, so as to distinguish the normal state and exception status purposes. Three traditional speech parameters including fundamental frequency, Mel-frequency cepstral coefficient, and parabolic spectral parameters (F0, MFCC, PSP) were used as the experimental control group.

In the classification stage, NAQ and CPR are used as two-dimensional vectors to establish the SVM model, 125 sets of samples are selected as training samples, and 125 sets of samples are used as test groups. The experiment selects the voices of 7 different speakers (4 males and 3 females) from the database. The sample is designed to eliminate the variation of speech parameters due to individual specificity. At the same time, F0, MFCC, and PSP are trained in the form of one-dimensional samples as the experimental control group. As shown in Figure 6, after 50 rounds of experiments, the average recognition rate of the parameters in the SV classification model is calculated. It can be seen that, compared with traditional parameters, NAQ and CPR sound source characteristics reflect good abnormal speech recognition performance in abnormal conditions.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, some improvements and modifications can also be made. It should also be regarded as the protection scope of the present invention.

Claims (6)

1. a kind of voice method for detecting abnormality based on sound source characteristics, it is characterised in that include the following steps：

(1), sensor real-time collecting voice data is passed through；

(2), the voice segments and noise segment of voice data are judged by end-point detection, to decide whether to carry out next step voice signal Handle work；

(3), the voice data framing adding window to obtained voice segments, and high frequency preemphasis processing is carried out to each frame；

(4), for the voice data of voice segments, glottis ripple signal is obtained using iteration self-adapting liftering；

(5), extract the characteristic parameter normalized amplitude business of glottis ripple and glottis closing time compares data；

(6), by the data extracted input, trained SVM models are classified；

(7), tag along sort is obtained, for judging speaker's situation, exports speaker's situation label, execution module is transferred to carry out anti- Feedback.

2. a kind of voice method for detecting abnormality based on sound source characteristics according to claim 1, it is characterised in that：The step Suddenly adding window uses Hamming window to a frame voice adding window in (3).

3. a kind of voice method for detecting abnormality based on sound source characteristics according to claim 1, it is characterised in that：The step Suddenly (3) medium-high frequency preemphasis processing promotes its high frequency section by the limited exciter response high-pass filter of a single order.

4. a kind of voice method for detecting abnormality based on sound source characteristics according to claim 1, it is characterised in that：The step Suddenly the step of glottis ripple signal is obtained in (4) is as follows：

(a), channel model is established using iteration self-adapting liftering；

Iteration self-adapting liftering eliminates the influence that glottal excitation is brought in primary speech signal frequency spectrum；

(b) and then by the method for liftering the influence of formant is eliminated；

Acoustic model is established by linear predictive coding and discrete complete or collected works' model exactly, finally obtains glottis using liftering Ripple signal.

5. a kind of voice method for detecting abnormality based on sound source characteristics according to claim 1, it is characterised in that：The step Suddenly the extracting method of the characteristic parameter normalized amplitude business of glottis ripple is as follows in (5)：

<mrow> <mi>N</mi> <mi>A</mi> <mi>Q</mi> <mo>=</mo> <mfrac> <mrow> <mi>A</mi> <mi>Q</mi> </mrow> <mi>T</mi> </mfrac> <mo>=</mo> <mfrac> <msub> <mi>f</mi> <mrow> <mi>a</mi> <mi>c</mi> </mrow> </msub> <mrow> <mi>d</mi> <mi>p</mi> <mi>e</mi> <mi>a</mi> <mi>k</mi> <mo>&amp;CenterDot;</mo> <mi>T</mi> </mrow> </mfrac> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow>

NAQ is normalized amplitude business in formula；T is pitch period；AQ is amplitude business, is glottis ripple peak swing corresponding with its one The ratio between maximum negative peak of order derivative；

<mrow> <mi>A</mi> <mi>Q</mi> <mo>=</mo> <mfrac> <msub> <mi>f</mi> <mrow> <mi>a</mi> <mi>c</mi> </mrow> </msub> <mrow> <mi>d</mi> <mi>p</mi> <mi>e</mi> <mi>a</mi> <mi>k</mi> </mrow> </mfrac> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>2</mn> <mo>)</mo> </mrow> </mrow>

F in formula_acFor the maximum crest value of glottal；Dpeak is the maximum negative peak that glottal corresponds to first derivative.

6. a kind of voice method for detecting abnormality based on sound source characteristics according to claim 1, it is characterised in that：The step Suddenly glottis closing time is as follows than the extracting method of data in (5)：

<mrow> <mi>C</mi> <mi>P</mi> <mi>R</mi> <mo>=</mo> <mfrac> <mrow> <mi>C</mi> <mi>P</mi> </mrow> <mi>O</mi> </mfrac> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>3</mn> <mo>)</mo> </mrow> </mrow>

CPR compares data for glottis closing time in formula；CP is the Closure states of glottis stage；O is glottis total opening time.