CN107103913B - Speech recognition method based on power spectrum Gabor characteristic sequence recursion model - Google Patents

Abstract

The invention discloses a speech recognition method based on a power spectrum Gabor characteristic sequence recursion model, which comprises the following basic steps: 1. preprocessing a voice input signal; 2. respectively extracting power spectrum features and dynamic spectrum Delta features; 3. filtering the frequency spectrum characteristic by using a space-time Gabor filter, and obtaining a voice characteristic sequence by PCA (principal component analysis) dimension reduction processing; 4. constructing a recursion graph according to the voice feature sequence; 5. and performing distance detection on the voice recursive model to complete voice recognition. The invention preprocesses the voice signal, obtains the voice characteristic sequence through characteristic extraction, and then converts the voice characteristic sequence into the recursion model for similarity detection, thereby effectively solving the problems that the recognition rate of the existing automatic voice recognition system is not ideal and the performance is easy to deteriorate under the complex conditions of unsteady noise, low signal-to-noise ratio and the like, and improving the robustness of the voice recognition algorithm.

Description

A Speech Recognition Method Based on Power Spectrum Gabor Feature Sequence Recursive Model

technical field

The invention belongs to the technical field of speech recognition, and relates to a speech recognition method under complex background, in particular to a speech recognition method based on a recursive model of power spectrum Gabor feature sequence.

Background technique

As the most natural and convenient way of communication, speech has always been one of the most important researches in the field of human-computer communication and interaction. Automatic Speech Recognition (ASR) is a particularly critical technology for realizing human-computer interaction. After years of research, ASR has entered our lives, and voice transcription, automatic translation, mobile phone assistants, etc. are typical representatives. But most of these systems depend on the acoustic environment in which they are located, and the robustness is not strong.

The existing speech recognition consists of two stages: one is the study of speech feature extraction, and the other is the study of classifier design. Traditional feature extraction methods, such as Mel cepstral coefficients (MFCC) and perceptual linear prediction (PLP), are difficult to extract effective features in complex environments; and traditional classification algorithms are difficult to achieve ideal results in the recognition process, such as dynamic Time Warped (DTW) Distance, Support Vector Machine (SVM), etc. Because a key problem that needs to be solved in the matching of speech recognition is that the speaker cannot pronounce the same word twice, and the feature dimensions after the feature extraction of the two pronunciations are also different, so it is difficult to achieve ideal results using traditional recognition methods. Neural networks are also plagued by problems such as requiring large samples of labeled data for training, and easy overfitting in the case of small samples, making it difficult to provide effective solutions.

SUMMARY OF THE INVENTION

In view of the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a speech recognition method based on the recursive model of the power spectrum Gabor feature sequence. The speech feature sequence is converted into a recursive graph for similarity detection, which effectively solves the problem that the recognition rate of the current automatic speech recognition system is not ideal and the performance is easy to deteriorate under complex conditions such as non-steady noise and low signal-to-noise ratio, thereby improving speech recognition. Robustness of the algorithm.

In order to realize the above-mentioned tasks, the present invention adopts the following technical solutions:

A speech recognition method based on a power spectrum Gabor feature sequence recursive model, comprising the following steps:

Step 1: Preprocessing of the speech signal

Perform endpoint detection on the acquired voice signal, separate and remove the noise information in the voice signal, obtain the effective part in the voice signal, and obtain its power spectrum;

Step 2: Extract the Gabor feature sequence of the power spectrum

Step 2.1, according to the power spectrum, extract the power normalized spectrum feature of the effective part;

Step 2.2, forming a sequence C according to the frame index sequence of the power-normalized spectral features, and then performing first-order difference and second-order difference processing, respectively, to obtain Delta spectral features and Double Delta spectral features;

Step 2.3, the power normalized spectral features, Delta spectral features and Double Delta spectral features are formed into a power normalized spectral feature set, and then the time-modulated filter is represented as a row vector through the space-time Gabor filtering, which is combined with the power normalized spectral feature set. Each channel of the normalized spectral feature set is independently convolved; similarly, the frequency-domain modulation filter, represented as a column vector, is independently convolved with each frame of the power-normalized spectral feature set;

Step 2.4, for the power normalized spectral feature set, perform histogram equalization, and then project the high-dimensional feature onto the low-dimensional space through PCA to obtain the power spectral Gabor feature set, and combine the power spectral Gabor feature set into Speech feature sequence X;

Step 3: Build a recursive graph of the speech feature sequence

Calculate the recursive graph r of the speech feature sequence X according to the following formula:

r=θ(ε-||x(i _k )-x(im )||), _{i k ,} im =1… _n

In the above formula, n is the number of states experienced by the speech feature sequence, k, _m represent the frame index, x( _ik ) and x(im ) are the observed speech feature sequences at the positions of the _{i k} and im sequences. value, ‖.‖ represents the Euclidean norm, ε represents the critical distance, ε<1; and θ represents the Heaviside function, which is defined as follows:

Step 4: Voice Signal Similarity Detection

Calculate the distance between the recursive graph r and the recursive graph r _i of each type of signal in the template library, then the voice signal in the template library corresponding to the smallest distance value is the result of the identification;

The template library refers to that, before speech recognition, various standard speech signals are collected, and processed according to the methods of steps 1 to 3 to obtain the recursive graph r _i of the standard speech signal, and the recursive graphs of these speech signals are obtained. Stored in the template library.

Further, the specific process of step 1 includes:

Step 1.1, for the acquired voice signal, use short-term energy or zero-crossing rate index to perform endpoint detection, and separate the effective part and noise information in the voice signal;

Step 1.2, complete the pre-emphasis process of the effective part through a high-pass filter;

Step 1.3, after the effective part of the pre-emphasis, the windowing and framing operation is performed;

Step 1.4, perform fast Fourier transform to obtain the energy distribution on the spectrum; take the modulo square of the spectrum to obtain the power spectrum.

Further, in step 2.3, the formula used in the convolution is:

In the above formula, k and n are the spectral and time indices of the power-normalized spectral features, respectively, i and j represent the relative center offset of the spectrum and time, respectively, PNSpec(k, n) represents the input speech signal feature set, filterfunction (i,j) represents the spatiotemporal Gabor filter function.

Further, the formula used to calculate the distance in step 4 is:

In the above formula, C(r _i |r _j ) is the size of the value after first training the compressed image r _j and then compressing the image r _i according to the MPEG-1 compression algorithm, so as to obtain the redundant redundancy in the r _i image and the r _j image removed. The smallest approximation between the two after the remaining information.

Compared with the prior art, the present invention has the following technical characteristics:

1. In terms of feature extraction, compared with the traditional static spectral features such as MFCC and PLP, this algorithm introduces the dynamic spectral Delta feature on the basis of the PNS spectrum, and then fuses the static spectral feature with the dynamic spectral feature. More robust speech features are obtained, and the distortion problem of speech signals in complex situations such as strong noise interference is overcome.

2. In terms of speech recognition, this algorithm focuses on the speech feature sequence, and proposes a scheme of recursive graph compression of the speech feature sequence, and then using the MPEG-1 compression algorithm to calculate the CK-1 distance between different recursive graphs, so that the The robust matching problem of speech feature sequences of different lengths in complex backgrounds is effectively solved.

Description of drawings

Figure 1 is the original speech signal (abscissa: time; ordinate: amplitude);

Fig. 2 is the effective part (abscissa: time; ordinate: amplitude) of speech signal after windowing and framing;

Fig. 3 is the PNSpec power spectrogram of speech signal (abscissa: frame number; ordinate: amplitude);

Fig. 4 is a Delta dynamic spectrum comparison diagram;

Fig. 5 is a dynamic spectrum recognition effect diagram;

Fig. 6 is the overall block diagram of speech feature extraction;

Fig. 7 is the recursive graph of the speech feature sequence; wherein (a), (b) and (c) are the recursive graphs of three different speech feature sequences, and (b) and (d) are the same speech signals in different environments the recursive graph of the generated feature sequence;

Fig. 8 is the speech similarity detection schematic diagram based on CK distance;

Fig. 9 is the schematic diagram of speech signal similarity detection in step 4;

Figure 10 is an overall flow chart of the method of the present invention.

Detailed ways

Following the above technical solutions, as shown in Figures 1 to 10, the present invention discloses a speech recognition method based on the recursive model of the power spectrum Gabor feature sequence. The detailed steps are described as follows:

Step 1: Preprocessing of the speech signal

Step 1.1, for the acquired voice signal, use indicators such as short-term energy or zero-crossing rate to perform endpoint detection, separate and remove noise information in the voice signal, and obtain an effective part in the voice signal;

Step 1.2, for the effective part of the speech signal obtained in step 1.1, pass it through a high-pass filter to complete the pre-emphasis process. Pre-emphasis can enhance the high-frequency part, make the spectrum of the signal flat, and keep it in the whole range from low frequency to high frequency. frequency band, and the spectrum can be obtained with the same signal-to-noise ratio. Its formula is as follows:

H(z)=1-μz ^-1

Among them, z represents the effective voice signal after endpoint detection, H(z) represents the voice signal after pre-emphasis, μ is a constant, generally 0.97.

Step 1.3, for the effective part of the voice signal after the pre-emphasis in step 1.2, the windowing and framing operation is performed to ensure the stability of the voice signal. The voice signal has short-term stability (it can be considered that the voice signal is approximately unchanged within 10-30ms), so that the voice signal can be divided into some short segments for processing, which is framing, and the framing of the voice signal is a movable A weighted method of finite-length windows is implemented.

In step 1.4, after the windowing of the speech signal in step 1.3, each frame must be subjected to fast Fourier transform to obtain the energy distribution in the frequency spectrum, and then the frequency spectrum of the speech signal is modulo squared to obtain the power spectrum of the speech signal.

Step 2: Extract the Gabor feature sequence of the power spectrum

Step 2.1, according to the power spectrum obtained in step 1.4, extract the power normalized spectrum (PNSpec) feature of the effective part; wherein a nonlinear compression method with a power-law value of 1/15 is used, and a Gammatone filter bank is used instead. Triangular filter bank to obtain PNSpec spectral characteristics, as shown in Figure 3. This makes PNSpec more robust to the acoustic variability of speech signals in environments such as steady-state noise.

In step 2.2, the power- _normalized spectral features are formed into a sequence C=( _{c 1} , c ₂ , . Then, after first-order difference and second-order difference processing, respectively, Delta spectral characteristics and Double Delta spectral characteristics are obtained; the calculation formula of difference parameters is as follows:

D[n]=C[n+m]-C[n-m]

In the above formula, D[n] represents the nth first-order difference, C[n] represents the nth PNSpec coefficient, n represents the index of the analysis frame, and the m value is about 2 or 3 in practice. Similarly, Double Delta is defined as the first-order difference of the delta feature.

Step 2.3, the power normalized spectral feature (13 dimensions), the Delta spectral feature and the DoubleDelta spectral feature are formed into a power normalized spectral feature set (39 dimensions), and then the time-modulated filter is represented by the time-space Gabor filter. is a row vector, convolved with each channel of the power-normalized spectral feature set independently; similarly, the frequency-domain modulated filter is represented as a column vector, convolved with each frame of the feature set independently. PNSpec is convolved with a row or column vector, and the formula is defined as follows:

Among them, k and n are the spectral and time indices of PNSpec, respectively, i and j represent the relative center offset of the spectrum and time, PNSpec(k, n) represents the input speech signal feature set, and filterfunction(i, j) represents the space-time Gabor filter function.

Step 2.4, for the power normalized spectral feature set, perform histogram equalization, and then project the high-dimensional features onto the low-dimensional space through PCA to obtain a robust power spectral Gabor (PNSG) feature set, and use The power spectrum Gabor feature set is composed of a speech feature sequence X={x(i ₁ ),x(i ₂ ),..., _x (in )}, where x(in ) represents the speech feature of the _nth frame, and X represents the entire The feature set of the speech signal; this sequence contains the valid information of the speech signal, and the subsequent processing is also performed for this sequence.

In Figure 4, different feature spectra of a sequence are shown, and in Figure 5, the classification effects of these feature maps are compared, and the effectiveness of adding dynamic features to the algorithm is verified from the side.

Step 3: Build a recursive graph of the speech feature sequence

Calculate the recursive graph r of the speech feature sequence X according to the following formula:

r=θ(ε-||x(i _k )-x(im )||), i _{k, i m =} 1…n

In the above formula, n is the number of states experienced by the speech feature sequence, k, _m represent the frame index, x( _ik ) and x(im ) are the observed speech feature sequences at the positions of the _{i k} and im sequences. value, ‖.‖ represents the Euclidean norm, ε represents the critical distance, ε<1; and θ represents the Heaviside function, which is defined as follows:

In the above formula, z corresponds to (ε-||x( _{i k} )-x(im )||) in the recursive graph calculation formula.

This step uses the recursive graph principle to convert the speech feature sequence into a recursive graph. During the calculation process, if the values of an n-dimensional feature sequence at the spatial positions of the i and j sequences are very close (threshold ε), then The place where the recursive matrix coordinate is ( _{i k} , im ) is marked with a value of 1, otherwise, it is marked as 0 at the corresponding position, and the recursive graph is drawn by black and white dots to reflect the time series.

Figure 7 shows the recursive graphs of three different speech feature sequences. From the figure, we can intuitively observe the internal structures of the three speech feature sequences, and the three recursive graphs are symmetrically distributed on both sides of the diagonal diagonal. Figure 7(b) and Figure 7(d) are recursive graphs of feature sequences generated by the same speech signal in different environments. It can be seen from the figures that the two have similar internal spatial structures.

Step 4: Voice Signal Similarity Detection

Calculate the distance between the recursive graph r and the recursive graph r _i of each type of signal in the template library. In this scheme, the compression coefficient _dmpeg between the speech feature sequences is used to represent the distance between the models:

As shown in the above formula, ri is the recursive graph of the speech feature sequence of each type of signal in the template library, and r _j is the recursive graph of the speech feature sequence to be tested, that is, r _obtained in step 3; C(r _i |r _j ) is the size of the value after first training the compressed image r _j and then compressing the image r _i according to the MPEG-1 compression algorithm, so as to obtain the minimum approximation between the two after removing the redundant information shared by the r _i image and the r _j image.

When r _i and r _j are two very similar images, calculate the MPEG-1 inter-frame compression coefficient, if a smaller value d _mpeg is generated, it indicates that r _i , r _j have significant similarity, r _i and r The similarity between _j recursive graphs is the part where they share a common structure, as shown in Figure 9 below:

By performing similarity detection and calculation with the recursive model of the speech signal in the template library, the different distances between the recursive model of the current speech to be tested and the recursive graph of each type of speech signal in the template library can be obtained, and these distance values are sorted. The speech signal in the template library corresponding to the smallest distance value is taken as the recognition result.

The template library mentioned in this step means that before speech recognition, various standard speech signals are collected and processed according to the methods in steps 1 to 3 to obtain the recursive model r _i of the standard speech signals. The recursive model is stored in a template library; in subsequent speech similarity detection, the recursive model of the speech signal to be tested is compared with the recursive model of each standard speech signal in the template library. The higher the similarity between the two is, the speech to be tested is considered to be the speech with the highest similarity, and this speech is output as the recognition result.

Figure 8 shows the discriminant analysis process of the algorithm in this paper. First, the obtained speech data is preprocessed, the corresponding feature sequence samples are extracted, and the corresponding recursive graph ④ is obtained. After comparing with the speech sequence recursion graph in the training set, find the training set sample ② with the smallest distance from ④, so as to obtain the final recognition result.

In order to verify the validity of this method, the present invention selects the public corpus to carry out experimental verification:

In the experiment, 6 kinds of non-stationary noises in NOISEX-92 corpus and TIMIT dataset are selected to be mixed to construct the experimental data of this paper. The noise types include factory noise (Factory), noisy noise (Babble), tank interior noise (Tank), car noise (Volvo), F16 cockpit noise (F16) and machine gun noise (Machinegun).

In order to create a training set, in the signal-to-noise ratio range of -15dB to 3dB (with a step size of 3dB), we select 400 TIMIT dataset sentences and the first half of the 6 kinds of non-stationary noise to mix in turn to obtain different signal-to-noise ratios. mixed speech signal.

For the test set, we choose 100 sentences and mix the second half of 6 kinds of non-stationary noise. Using different parts of non-stationary noise ensures that the noise segment used in the test set is different from the noise segment in the training set, which can better highlight the robustness of the system.

The average recognition accuracy rate of the algorithm proposed in the present invention is as high as 90.36%, which is 25.49%, 20.36% and 3.9% higher than MFCC, PNCC and SGBFB respectively. High recognition accuracy and good recognition effect.

Claims (2)

1. A speech recognition method based on a power spectrum Gabor characteristic sequence recursive model is characterized by comprising the following steps:

step one, preprocessing a voice signal

Carrying out end point detection on the obtained voice signal, separating and removing noise information in the voice signal, obtaining an effective part in the voice signal, and solving a power spectrum of the effective part;

step two, extracting a power spectrum Gabor characteristic sequence

Step 2.1, extracting the power normalization frequency spectrum characteristic of the effective part according to the power spectrum;

step 2.2, forming a sequence C by the power normalization spectrum characteristics according to a frame index sequence, and respectively carrying out first-order difference and second-order difference processing to respectively obtain Delta spectrum characteristics and DoubleDelta spectrum characteristics;

step 2.3, the power normalized spectrum feature, the Delta spectrum feature and the DoubleDelta spectrum feature form a power normalized spectrum feature set, and then a time modulation filter is represented as a row vector through space-time Gabor filtering and is independently convolved with each channel of the power normalized spectrum feature set; similarly, the frequency domain modulation filter is represented as a column vector, convolved independently with each frame of the set of power normalized spectral features;

in step 2.3, the formula adopted by the convolution is as follows:

in the above formula, k and n are respectively the frequency spectrum and time index of the power normalized frequency spectrum feature, i and j respectively represent the deviation of the frequency spectrum and time from the center, PNSpec (k, n) represents the feature set of the input voice signal, and filter function (i, j) represents the space-time Gabor filter function;

step 2.4, executing histogram equalization aiming at the power normalization spectrum feature set, then projecting high-dimensional features onto a low-dimensional space through PCA to obtain a power spectrum Gabor feature set, and forming the power spectrum Gabor feature set into a voice feature sequence X;

step three, constructing a recursion graph of the voice characteristic sequence

Calculating a recursion graph r of the speech feature sequence X according to the following formula:

r＝θ(ε-||x(i_k)-x(i_m)||)，i_k，i_m＝1...n

in the above formula, n is the number of states experienced by the speech feature sequence, k and m represent frame indexes, and x (i)_k) And x (i)_m) Is at i_kAnd i_mThe value of the voice characteristic sequence observed at the sequence position, | | | | represents Euclidean norm, epsilon represents critical distance, and epsilon is less than 1; and θ represents the Heaviside function, which is defined as follows:

step four, voice signal similarity detection

Calculating the recursion graph r and the recursion graph r of each type of signal in the template library_iThe voice signal in the template library corresponding to the minimum distance value is the recognized result;

the template library is that before speech recognition, various standard speech signals are collected and processed according to the method of the first step to the third step to obtain a recursion graph r of the standard speech signals_iStoring a recursive graph of the speech signals in a template library;

the formula adopted for calculating the distance in the fourth step is as follows:

in the above formula, r_iIs a recursive graph of the speech feature sequence of each class of signal in the template library, r_jIs a recursive graph of the speech feature sequence to be measured, C (r)_i|r_j) Firstly training a compressed image r according to an MPEG-1 compression algorithm_jRecompressed image r_iThen the magnitude of the value, thereby obtaining r_iRemoving sum r from image_jMinimum approximation between redundant information common to images。

2. The method for speech recognition based on the recursive model of power spectrum Gabor feature sequence according to claim 1, wherein the specific process of the first step comprises:

step 1.1, carrying out end point detection on the obtained voice signal by using short-time energy or zero-crossing rate indexes, and separating an effective part from noise information in the voice signal;

step 1.2, completing the pre-emphasis process of the effective part through a high-pass filter;

step 1.3, performing windowing and framing operation on the pre-emphasized effective part;

step 1.4, performing fast Fourier transform to obtain energy distribution on a frequency spectrum; and performing modulus squaring on the frequency spectrum to obtain a power spectrum.

Images