CN108198547B - Voice endpoint detection method, apparatus, computer equipment and storage medium - Google Patents

Abstract

The application relates to a voice endpoint detection method, a voice endpoint detection device, computer equipment and a storage medium. The method comprises the following steps: acquiring a voice signal with noise, and extracting acoustic features and spectrum features corresponding to the voice signal with noise; converting the acoustic features and the spectral features to obtain corresponding acoustic feature vectors and spectral feature vectors; acquiring a classifier, and inputting the acoustic feature vector and the spectral feature vector into the classifier to obtain an acoustic feature vector added with a voice tag and a spectral feature vector added with the voice tag; analyzing the acoustic feature vector added with the voice tag and the frequency spectrum feature vector added with the voice tag to obtain a corresponding voice signal; and determining a starting point and an ending point corresponding to the voice signal according to the time sequence of the voice signal. The method can effectively improve the accuracy of voice endpoint detection.

Description

Voice endpoint detection method, apparatus, computer equipment and storage medium

technical field

The present application relates to the technical field of signal processing, and in particular, to a language endpoint detection method, apparatus, computer equipment and storage medium.

Background technique

With the continuous development of speech technology, speech endpoint detection technology occupies a very important position in speech recognition technology. Voice endpoint detection is to detect the starting point and ending point of the voice part from a continuous noise voice, so that the voice can be recognized effectively.

There are two traditional voice endpoint detection methods. One is to extract the characteristics of each segment of the signal according to the different characteristics of the voice and noise signals in the time domain and frequency domain, and compare the characteristics of each segment of the signal with the set threshold. Perform voice endpoint detection. However, this method is only suitable for detection under stationary noise conditions, and the noise robustness is poor, and it is difficult to distinguish between pure speech and noise, resulting in low accuracy of speech endpoint detection. . The other is a neural network-based approach, which uses a trained model to perform endpoint detection on speech signals. However, the input vectors of most models only contain the features of noisy speech, which makes the noise robustness poor, resulting in lower accuracy of speech endpoint detection. Therefore, how to effectively improve the accuracy of voice endpoint detection has become a technical problem that needs to be solved at present.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a voice endpoint detection method, apparatus, computer equipment and storage medium that can effectively improve the accuracy of voice endpoint detection in view of the above technical problems.

A voice endpoint detection method, comprising:

Acquiring a noisy speech signal, and extracting the acoustic features and spectral features corresponding to the noisy speech signal;

Converting the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector;

Obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier, obtaining the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag;

Analyzing the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain a corresponding voice signal;

The starting point and the ending point corresponding to the voice signal are determined according to the time sequence of the voice signal.

In one of the embodiments, before the extracting the acoustic feature and the spectral feature corresponding to the noisy speech signal, the method further includes:

converting the noisy speech signal into a noisy speech spectrum;

Time domain analysis and/or frequency domain analysis and/or transform domain analysis are performed on the noisy speech spectrum to obtain acoustic features corresponding to the noisy speech signal.

In one of the embodiments, before the extracting the acoustic feature and the spectral feature corresponding to the noisy speech signal, the method further includes:

Converting the noisy speech signal into a noisy speech spectrum, and calculating a noisy speech amplitude spectrum according to the noisy speech spectrum;

Perform dynamic noise estimation on the noisy speech spectrum according to the noisy speech amplitude spectrum to obtain a noise amplitude spectrum;

Estimate the speech amplitude spectrum of the pure speech signal according to the noisy speech amplitude spectrum and the noise amplitude spectrum;

The spectral feature corresponding to the noisy speech signal is generated by using the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum.

In one of the embodiments, the converting the acoustic feature and the spectral feature includes:

Extracting a preset number of frames before and after the current frame in the acoustic feature and the spectral feature;

Calculate the mean value vector and/or the variance vector corresponding to the current frame by using a preset number of frames before and after the current frame;

Perform logarithmic domain transformation on the acoustic feature and the spectral feature after calculating the mean vector and/or the variance vector corresponding to the current frame, to obtain the transformed acoustic feature vector and spectral feature vector.

In one of the embodiments, before the step of obtaining the classifier, the step further includes:

Obtaining the noisy speech data to which the speech category label is added, and obtaining an initial classifier by training the noisy speech data;

acquiring a first verification set, where the first verification set includes a plurality of first voice data;

Inputting a plurality of first voice data into a classifier to obtain class probabilities corresponding to the plurality of first voice data;

Screening the class probabilities corresponding to the plurality of first voice data, adding a class label to the selected first speech data, and obtaining a verification set with the class label added;

Use the verification set with the added category label and the training set for training to obtain a verification classifier;

acquiring a second verification set, the second verification set includes a plurality of second voice data;

Inputting a plurality of second voice data into the verification classifier to obtain class probabilities corresponding to the plurality of second voice data;

When the class probabilities corresponding to the plurality of second speech data reach the preset probability value, the required classifier is obtained.

In one embodiment, the step of classifying the acoustic feature vector and the spectral feature vector by using the classifier includes:

Using the acoustic feature vector and the spectral feature vector as the input of the classifier, obtain the decision value corresponding to the acoustic feature vector and the spectral feature vector;

When the decision value is the first threshold, adding a voice tag to the acoustic feature vector or the spectral feature vector;

When the decision value is the second threshold, a non-speech tag is added to the acoustic feature vector or the spectral feature vector.

A voice endpoint detection device, comprising:

an extraction module for acquiring a noisy speech signal, and extracting the acoustic features and spectral features corresponding to the noisy speech signal;

a conversion module, configured to convert the acoustic features and spectral features to obtain corresponding acoustic feature vectors and spectral feature vectors;

A classification module, for obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier to obtain the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag;

The parsing module is used to analyze the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain a corresponding voice signal; determine the corresponding starting point and termination of the voice signal according to the time sequence of the voice signal point.

In one embodiment, the conversion module is further configured to extract a preset number of frames before and after the current frame in the acoustic feature and the spectral feature; calculate the mean value corresponding to the current frame by using the preset number of frames before and after the current frame vector and/or variance vector; perform logarithmic domain transformation on the acoustic feature and spectral feature after calculating the mean vector and/or variance vector corresponding to the current frame, to obtain the converted acoustic feature vector and spectral feature vector.

A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Acquiring a noisy speech signal, and extracting the acoustic features and spectral features corresponding to the noisy speech signal;

Converting the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector;

Obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier, obtaining the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag;

Analyzing the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain a corresponding voice signal;

The starting point and the ending point corresponding to the voice signal are determined according to the time sequence of the voice signal.

A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the following steps are implemented:

Acquiring a noisy speech signal, and extracting the acoustic features and spectral features corresponding to the noisy speech signal;

Converting the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector;

Obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier, obtaining the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag;

Analyzing the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain a corresponding voice signal;

The starting point and the ending point corresponding to the voice signal are determined according to the time sequence of the voice signal.

The above-mentioned voice endpoint detection method, device, computer equipment and storage medium obtains a noisy voice signal, and extracts the corresponding acoustic feature and spectral feature of the noisy voice signal; by converting the acoustic feature and the spectral feature, the corresponding acoustic feature vector and Spectral eigenvectors. Obtain a classifier, and by inputting the acoustic feature vector and the spectral feature vector into the classifier, obtain the acoustic feature vector with the added voice tag and the spectral feature vector with the added voice tag, so that the acoustic feature vector and the spectral feature vector can be effectively classified. , so that speech and non-speech can be effectively recognized. The acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag are analyzed to obtain the corresponding voice signal; the time sequence of the voice signal determines the corresponding start point and end point of the voice signal, so that the noisy voice signal can be accurately identified It can effectively improve the accuracy of voice endpoint detection.

Description of drawings

1 is a flowchart of a method for detecting a voice endpoint in one embodiment;

Fig. 2 is the internal structure diagram of the voice endpoint detection device in one embodiment;

FIG. 3 is a diagram of the internal structure of a computer device in one embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present application clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, but not to limit the application. It will be understood that the terms "first", "second", etc., as used herein, may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element.

In one embodiment, as shown in FIG. 1 , a method for detecting a voice endpoint is provided, which is described by taking the method applied to a terminal as an example, including the following steps:

Step 102: Acquire the noisy speech signal, and extract the acoustic features and spectral features corresponding to the noisy speech signal.

Generally speaking, the actually collected speech signal usually contains a certain intensity of noise. When the intensity of these noises is high, it will have a significant impact on the effect of speech applications, such as low speech recognition efficiency and reduced endpoint detection accuracy.

The terminal can acquire the voice input by the user through the voice input device. The terminal may be a terminal such as a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., and the terminal may further include a voice input device, for example, a device with a voice input function such as a microphone. The voice input by the user obtained by the terminal is usually a noisy voice signal containing noise, and the noisy voice signal may be a call voice, recorded audio, voice commands, etc. input by the user. After acquiring the noisy speech signal, the terminal extracts the acoustic features and spectral features corresponding to the noisy speech signal. The acoustic features may include feature information such as unvoiced, voiced, vowels, and consonants of the noisy speech signal. The spectral features may include the vibration frequency and vibration amplitude of the noisy speech signal, and characteristic information such as the loudness and timbre of the noisy speech signal.

Specifically, after acquiring the noisy speech signal, the terminal performs windowing and framing on the noisy speech signal. For example, a Hanning window can be used to divide the noisy speech signal into multiple frames with a frame length of 10-30ms (milliseconds), and the frame shift can be 10ms, so that the noisy speech signal can be divided into multi-frame noisy speech signals. After the terminal performs windowing and framing on the noisy speech signal, the terminal performs fast Fourier transform on the noisy speech signal after windowing and framing, thereby obtaining the frequency spectrum of the noisy speech signal. The terminal can extract the acoustic features and spectral features corresponding to the noisy speech signal according to the frequency spectrum of the noisy speech.

Step 104: Convert the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector.

After the terminal extracts the acoustic features and spectral features corresponding to the noisy speech signal, it converts the extracted acoustic features and spectral features corresponding to the noisy speech signal, converts the acoustic features into corresponding acoustic feature vectors, and converts the spectral features into The corresponding spectral eigenvectors.

Step 106: Obtain a classifier, input the acoustic feature vector and the spectral feature vector into the classifier, and obtain the acoustic feature vector with the added voice tag and the spectral feature vector with the added voice tag.

The terminal obtains the classifier. The classifier is a classifier trained before voice endpoint detection. The classifier can add voice labels and non-voice labels to the acoustic feature vector and the spectral feature vector to convert the input acoustic feature vector and spectral feature vector. It is divided into acoustic feature vector and spectral feature vector of speech class and acoustic feature vector and spectral feature vector of non-speech class. The terminal inputs the acoustic feature vector and spectral feature vector corresponding to the noisy speech to the classifier, and uses the classifier to classify the input acoustic feature vector and spectral feature vector. When the input acoustic feature vector or spectral feature vector is a speech category, add a voice label to the acoustic feature vector or spectral feature vector; when the input acoustic feature vector or spectral feature vector is a non-speech category, add the acoustic feature vector or spectral feature vector The vectors add non-speech tags, so that speech and non-speech can be accurately identified. After the terminal uses the classifier to compare the acoustic feature vector and the spectral feature vector, the acoustic feature vector with the added voice tag and the spectral feature vector with the added voice tag can be obtained.

Further, the terminal uses the acoustic feature vector and the spectral feature vector as the input of the classifier, and can also obtain decision values corresponding to the acoustic feature vector and the spectral feature vector. The terminal can add a voice tag or a non-voice tag to the acoustic feature vector and the spectral feature vector according to the obtained decision value. Thus, accurate classification of acoustic feature vectors and spectral feature vectors is achieved.

Step 108 , analyze the acoustic feature vector added with the voice tag and the spectral feature vector added with the voice tag to obtain the voice signal after adding the voice tag.

Step 110: Determine the corresponding start point and end point of the speech signal according to the speech label and the time sequence of the speech signal.

After the terminal classifies the acoustic feature vector and the spectral feature vector, it needs to parse the acoustic feature vector to which the voice label is added and the spectral feature vector to which the voice label is added. Specifically, the terminal parses the acoustic feature vector to which the voice tag is added and the spectral feature vector to which the voice tag is added, to obtain the spectrum corresponding to the acoustic feature and the spectral feature to which the voice tag is added. According to the time sequence of the noisy speech signal, the terminal converts the frequency spectrum corresponding to the acoustic characteristic and the frequency spectrum characteristic to which the speech tag is added, into a corresponding speech signal, so that the corresponding speech signal can be obtained by analysis.

The noisy speech signal has a time sequence, and the time sequence of the speech signal after adding the speech tag still corresponds to the time sequence of the noisy speech signal. The terminal parses the acoustic feature vector to which the voice tag is added and the spectral feature vector to which the voice tag is added into the corresponding voice signal to which the voice tag is added. start and end points.

For example, after the terminal classifies the input acoustic feature vector and spectral feature vector through the classifier, the obtained decision value may be a value between 0 and 1. When the obtained decision value is 1, the terminal adds a voice tag to the acoustic feature vector or the spectral feature vector. When the obtained decision value is 0, the terminal adds a non-speech tag to the acoustic feature vector or the spectral feature vector. Thereby, the acoustic feature vector and the spectral feature vector can be accurately classified. After the terminal analyzes the acoustic feature vector to which the voice tag is added and the spectral feature vector to which the voice tag is added, the voice signal to which the voice tag is added can be obtained. According to the time sequence of the voice signal after adding the voice tag, when the voice frame added with the voice tag appears for the first time, it is the starting point of the noisy voice signal, and when the voice frame corresponding to the voice tag appears for the last time, it is the noisy voice signal. the termination point. Further, the starting point of the speech signal may be determined according to the jump of the decision value 0 to 1, and the end point of the speech signal may be determined according to the jump of the decision value 1 to 0. In this way, the corresponding starting point and ending point of the noisy speech signal can be accurately determined.

In this embodiment, after acquiring the noisy speech signal, the terminal extracts the acoustic feature and spectral feature corresponding to the noisy speech signal, and obtains the corresponding acoustic feature vector and spectral feature vector by converting the acoustic feature and the spectral feature. By inputting the acoustic feature vector and the spectral feature vector into the classifier, the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag are obtained, so that the acoustic feature vector and the spectral feature vector can be effectively classified, so that the Voice and non-voice are recognized. The terminal obtains the corresponding voice signal by analyzing the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag. The terminal determines the corresponding starting point and ending point of the voice signal according to the time sequence of the voice signal, thereby accurately identifying the starting point and ending point of the noisy voice signal, thereby effectively improving the accuracy of voice endpoint detection.

In one embodiment, before extracting the acoustic features and spectral features corresponding to the noisy speech signal, the method further includes: converting the noisy speech signal into a noisy speech spectrum; performing time domain analysis and/or frequency domain analysis on the noisy speech spectrum Analysis and/or transform domain analysis to obtain acoustic features corresponding to the noisy speech signal.

In phonetics, speech features can be divided into acoustic features such as vowels, consonants, unvoiced, voiced, and silence. After acquiring the noisy speech signal, the terminal performs windowing and framing on the noisy speech signal. For example, a Hanning window can be used to divide the noisy speech signal into multiple frames with a frame length of 10-30ms (milliseconds), and the frame shift can be 10ms. Thereby, the noisy speech signal can be divided into multi-frame noisy speech signals. After the terminal performs windowing and framing on the noisy speech signal, the terminal performs fast Fourier transform on the noisy speech signal after windowing and framing, thereby obtaining the frequency spectrum of the noisy speech signal.

Further, the terminal may perform time domain analysis and/or frequency domain analysis and/or transform domain analysis on the noisy speech spectrum, so as to obtain acoustic features corresponding to the noisy speech signal.

For example, the terminal may extract the acoustic features corresponding to the noisy speech signal by adopting the MFCC (Mel-Frequency Cepstrum Coefficients, Mel frequency cepstrum coefficients) manner. The terminal converts the noisy speech signal into a frequency spectrum of the noisy speech signal after windowing and framing the noisy speech signal. The terminal transforms the spectrum of the noisy speech signal into a noisy speech cepstrum, the terminal performs cepstrum analysis according to the noisy speech cepstrum, and performs discrete cosine transform on the noisy speech cepstrum to obtain the acoustic characteristics of each frame, so as to obtain Effective acoustic features of noisy speech.

In one embodiment, before extracting the acoustic features and spectral features corresponding to the noisy speech signal, the method further includes: converting the noisy speech signal into a noisy speech spectrum, and calculating the noisy speech amplitude spectrum according to the noisy speech spectrum; The noisy speech amplitude spectrum performs dynamic noise estimation on the noisy speech spectrum to obtain the noise amplitude spectrum; estimates the speech amplitude spectrum of the pure speech signal according to the noisy speech amplitude spectrum and the noise amplitude spectrum; uses the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum The magnitude spectrum generates spectral features corresponding to noisy speech signals.

After acquiring the noisy speech signal, the terminal performs windowing and framing on the noisy speech signal. For example, a Hanning window can be used to divide the noisy speech signal into multiple frames with a frame length of 10-30ms (milliseconds), and the frame shift can be 10ms. Thereby, the noisy speech signal can be divided into multi-frame noisy speech signals. After the terminal performs windowing and framing on the noisy speech signal, the terminal performs fast Fourier transform on the noisy speech signal after windowing and framing, thereby obtaining the frequency spectrum of the noisy speech signal. The spectrum of the noisy speech signal may be the energy amplitude spectrum of the noisy speech after fast Fourier transform.

Further, the terminal can calculate the noisy speech amplitude spectrum and the noisy speech phase spectrum by using the noisy speech spectrum. The terminal performs dynamic noise estimation on the noisy speech spectrum according to the noisy speech amplitude spectrum and the noisy speech phase spectrum. Specifically, the terminal can use the improved minimum controlled recursive averaging algorithm to perform dynamic noise estimation on the noisy speech spectrum, so as to obtain the noise amplitude spectrum. The terminal estimates the speech amplitude spectrum of the speech signal according to the noisy speech amplitude spectrum, the noisy speech phase spectrum and the noise amplitude spectrum. For example, the terminal may estimate the speech amplitude spectrum of the speech signal by using the logarithmic amplitude spectrum minimum mean square error estimation method.

The terminal uses the estimated noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum of the pure speech signal to generate the spectral features corresponding to the noisy speech signal, so that the terminal can effectively extract the spectral characteristics corresponding to the noisy speech signal.

In one embodiment, converting the acoustic feature and the spectral feature includes: extracting a preset number of frames before and after the current frame in the acoustic feature and the spectrum feature; calculating the mean vector sum corresponding to the current frame by using the preset number of frames before and after the current frame /or variance vector; perform logarithmic domain transformation on the acoustic feature and spectral feature after calculating the mean vector and/or variance vector corresponding to the current frame, to obtain the converted acoustic feature vector and spectral feature vector.

After acquiring the noisy speech signal, the terminal performs windowing and framing on the noisy speech signal. Thereby, the noisy speech signal can be divided into multi-frame noisy speech signals. After the terminal performs windowing and framing on the noisy speech signal, the terminal performs fast Fourier transform on the noisy speech signal after windowing and framing, thereby obtaining the frequency spectrum of the noisy speech signal. The terminal can then extract the acoustic features and spectral features corresponding to the noisy speech signal according to the frequency spectrum of the noisy speech.

After extracting the acoustic features and spectral features corresponding to the noisy speech signal, the terminal converts the acoustic features and spectral features into acoustic feature vectors and spectral feature vectors. The terminal extracts a preset number of frames before and after the current frame in the acoustic feature vector and the spectral feature vector. The terminal calculates the mean vector or variance vector corresponding to the current frame by using a preset number of frames before and after the current frame, so that the acoustic feature and the spectral feature can be smoothed to obtain the smoothed acoustic feature vector and the spectral feature vector.

For example, the terminal may acquire five frames ahead and five subsequent frames of the current frame of acoustic features or spectral features, for a total of 11 frames of noisy speech spectrum. By calculating the average of these 11 frames, the average vector of the current frame can be obtained. Specifically, the terminal may acquire a filter bank, wherein the shape of the filter is a triangle, and the triangle window represents the filter window. Each filter has the characteristics of a triangular filter, which can be of equal bandwidth in the noisy speech spectrum. The terminal can use the filter bank to calculate the mean vector of the current frame, thereby smoothing the noisy speech spectrum to obtain the smoothed acoustic feature vector and spectral feature vector.

After smoothing the noisy speech spectrum, the terminal calculates the logarithmic domain for the smoothed acoustic eigenvectors and the smoothed spectral eigenvectors to obtain the converted acoustic eigenvectors and spectral eigenvectors. Specifically, the terminal can calculate the logarithmic energy of the acoustic feature and the spectral feature output by each filter, thereby obtaining the logarithmic domain of the acoustic feature vector and the logarithmic domain of the spectral feature vector, thereby effectively obtaining the converted Acoustic eigenvectors and spectral eigenvectors.

In one embodiment, before the step of obtaining the classifier, the step further includes: obtaining noisy speech data to which the speech category label is added, and training the noisy speech data to obtain an initial classifier; obtaining a first verification set, which is in the first verification set including a plurality of first voice data; inputting the plurality of first voice data into the classifier to obtain class probabilities corresponding to the plurality of first voice data; screening the class probabilities corresponding to the plurality of first voice data, adding a category label to the first voice data to obtain a verification set with the category label added; using the verification set and training set to which the category label is added for training to obtain a verification classifier; obtaining a second verification set, the second verification set includes a plurality of second voices inputting multiple second voice data into the verification classifier to obtain the class probabilities corresponding to the multiple second voice data; when the class probabilities corresponding to the multiple second voice data reach the preset probability value, obtain the required classification device.

Before obtaining the classifier, it is necessary to train the classifier with a large amount of noisy speech data. These large amounts of noisy speech data can be the noisy speech data obtained by the terminal from the database, or the noisy speech data obtained by the terminal from the Internet. . When training the classifier, firstly, the noisy speech data is manually labeled, and the classifier is obtained by training the artificially labeled noisy speech data.

Specifically, after extracting the acoustic features and spectral features corresponding to the noisy speech data, the terminal converts the acoustic features and spectral features into corresponding acoustic feature vectors and spectral feature vectors. The staff can annotate the acoustic feature vector and the spectral feature vector according to the category comparison table, and add a voice tag or a non-voice tag to each frame of the noisy voice signal. The terminal obtains the noisy speech data after the staff has marked the noisy speech data according to the category comparison table.

The terminal combines the labeled acoustic feature vector and spectral feature vector to input to the input layer of LSTM (Bidirectional Long Short-term Memory, bidirectional long short-term memory neural network). The nonlinear hidden layer in the LSTM neural network can be input from the input layer. New features are learned in the vector, and the category of the input vector is calculated through the activation function. Specifically, there are three gates in each LSTM unit, namely forget gate, candidate gate and output gate. The specific calculation formula can be:

表示遗忘门权重矩阵，

是遗忘门输入层与隐层之间的权重矩阵，b_f表示遗忘门的偏置，遗忘门是通过将前一隐层的输出h_t-1与当前的输入x_t进行了线性组合，然后利用激活函数将其输出值压缩到0到1之间。当输出值越靠近1，则表明记忆体保留的信息越多；反之，越靠近0，则表明记忆体保留的信息越少。where σ represents the activation function,

represents the forget gate weight matrix,

is the weight matrix between the input layer and the hidden layer of the forget gate, b _f represents the bias of the forget gate, and the forget gate is a linear combination of the output h _t-1 of the previous hidden layer and the current input x _t , and then Use an activation function to compress its output values between 0 and 1. When the output value is closer to 1, it indicates that the memory retains more information; on the contrary, the closer it is to 0, it indicates that the memory retains less information.

The candidate gate calculates the current input cell state, and the specific formula can be:

Among them, C _i represents the current input unit state, and the output value can be adjusted to between -1 and 1 through the tanh activation function.

The output gate can control the amount of memory information used for the next layer network update, the formula can be expressed as:

Among them, O _t represents the amount of memory information used for the next layer network update.

The final output can be calculated by the LSTM unit, and the formula can be expressed as:

h _t =O _t ×tanh(C _t )

The final acoustic eigenvector or spectral eigenvector is obtained by forward and reverse calculations, and the formula can be expressed as:

为正向的输出向量，

为反向的输出向量，h_i为最后的标注了类别标签的多个声学特征向量或频谱特征向量。in

is the forward output vector,

is the reversed output vector, and _hi is the final multiple acoustic feature vectors or spectral feature vectors labeled with class labels.

Further, the output layer in the LSTM can calculate the value of the output unit C _i according to the preset decision function. Wherein, the value of the output unit C _i can be a value between 0 and 1, where 1 represents a speech class, and 0 represents a non-speech class.

The terminal calculates the probability that each acoustic feature and spectral feature belong to the voice category and non-speech category in the category comparison table by using the multiple acoustic feature vectors and spectral feature vectors marked with the voice category label, and extracts the acoustic feature vector and the spectral feature vector in the category. The category with the largest probability value in the table is compared, and the voice category label corresponding to the category with the largest probability value is added to the acoustic feature vector or the spectral feature vector.

The terminal uses the noisy speech data to which the speech category label is added for training to obtain an initial classifier; the terminal obtains a first verification set, and the first verification set includes a plurality of first speech data. The terminal inputs the plurality of first voice data into the classifier, and after obtaining the class probabilities corresponding to the plurality of first voice data, the terminal screens the class probabilities corresponding to the plurality of first voice data. The staff uses the terminal to add a voice category tag to the selected first voice data, the terminal obtains the voice category tagged first voice data, and uses the voice category tagged first voice data to generate a voice category tagged verification set. The terminal uses the verification set with the added category voice labels and the noisy voice data to retrain to obtain a verification classifier. The terminal acquires a second verification set, and the second verification set includes a plurality of second voice data; and inputs the plurality of second voice data into the verification classifier to obtain class probabilities corresponding to the plurality of second voice data. The terminal filters out the second language data whose category probability is within the preset range, and then labels the filtered second speech data, and retrains the labelled second speech data and the labelled noisy speech data to obtain new data. classifier. The training continues until the probability values of the preset number of acoustic feature vectors or spectral feature vectors in all validation sets are between the preset probability range values, and then the training is stopped, and the desired classifier can be obtained. Thereby, a classifier with a high accuracy rate can be obtained, so that the acoustic feature vector and the spectral feature vector can be accurately classified, and then the speech and non-speech can be accurately identified.

In one embodiment, the step of using the classifier to classify the acoustic feature vector and the spectral feature vector includes: using the acoustic feature vector and the spectral feature vector as the input of the classifier to obtain the decision value corresponding to the acoustic feature vector and the spectral feature vector; When the decision value is the first threshold, a voice tag is added to the acoustic feature vector or the spectral feature vector; when the decision value is the second threshold, a non-voice tag is added to the acoustic feature vector or the spectral feature vector.

After acquiring the noisy speech signal, the terminal extracts the acoustic features and spectral features corresponding to the noisy speech signal. The terminal converts the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector. After acquiring the classifier, the terminal inputs the acoustic feature vector and the spectral feature vector to the classifier. After the classifier classifies the input acoustic eigenvectors and spectral eigenvectors, decision values corresponding to the acoustic eigenvectors and the spectral eigenvectors can be obtained. When the obtained decision value is the preset first threshold, the terminal adds a voice tag to the acoustic feature vector or the spectral feature vector. Wherein, the first threshold may be a range value. When the obtained decision value is a preset second threshold, the terminal adds a non-speech tag to the acoustic feature vector or the spectral feature vector. By using the classifier to accurately classify the acoustic feature vector and the spectral feature vector, the speech signal and the non-speech signal in the noisy speech signal can be accurately identified.

For example, the resulting decision value can be a value between 0 and 1. The preset first threshold may be 1, and the preset second threshold may be 0. When the obtained decision value is 1, the terminal adds a voice tag to the acoustic feature vector or the spectral feature vector. When the obtained decision value is 0, the terminal adds a non-speech tag to the acoustic feature vector or the spectral feature vector. Thereby, the acoustic feature vector and the spectral feature vector can be accurately classified.

In one embodiment, as shown in FIG. 2, a voice endpoint detection apparatus is provided, including an extraction module 202, a conversion module 204, a classification module 206 and a parsing module 208, wherein:

The extraction module 202 is used for acquiring the noisy speech signal, and extracting the acoustic features and spectral features corresponding to the noisy speech signal.

The conversion module 204 is configured to convert the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector.

The classification module 206 is configured to obtain a classifier, input the acoustic feature vector and the spectral feature vector to the classifier, and obtain the acoustic feature vector with added voice tag and the spectral feature vector with added voice tag.

The parsing module 208 is configured to parse the acoustic feature vector added with the voice tag and the spectral feature vector added with the voice tag to obtain a corresponding voice signal; determine the corresponding start point and end point of the voice signal according to the time sequence of the voice signal.

In one embodiment, the extraction module 202 is further configured to convert the noisy speech signal into a noisy speech spectrum; perform time domain analysis and/or frequency domain analysis and/or transform domain analysis on the noisy speech spectrum, Acoustic features corresponding to the noisy speech signal are obtained.

In one embodiment, the extraction module 202 is further configured to convert the noisy speech signal into a noisy speech spectrum, calculate the noisy speech amplitude spectrum according to the noisy speech spectrum; perform dynamic noise on the noisy speech spectrum according to the noisy speech amplitude spectrum Estimate and obtain the noise amplitude spectrum; estimate the speech amplitude spectrum of the pure speech signal according to the noisy speech amplitude spectrum and the noise amplitude spectrum; use the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum to generate the corresponding spectral features of the noisy speech signal.

In one embodiment, the conversion module 204 is further configured to extract a preset number of frames before and after the current frame in the acoustic feature and the spectral feature; calculate the mean vector sum corresponding to the current frame by using the preset number of frames before and after the current frame /or variance vector; perform logarithmic domain transformation on the acoustic feature and spectral feature after calculating the mean vector and/or variance vector corresponding to the current frame, to obtain the converted acoustic feature vector and spectral feature vector.

In one embodiment, the device further includes a training module for acquiring noisy speech data to which a speech category label is added, and obtaining an initial classifier by training the noisy speech data; acquiring a first verification set, the first verification set including a plurality of first voice data; inputting the plurality of first voice data into the initial classifier to obtain the class probabilities corresponding to the plurality of first voice data; screening the class probabilities corresponding to the plurality of first voice data, and selecting Add category labels to the first voice data of the 1st voice data to obtain a verification set with category tags; use the verification set with category tags and the noisy voice data with voice category tags for training to obtain a verification classifier; obtain the second verification set, the second The verification set includes multiple second voice data; the multiple second voice data is input into the verification classifier to obtain the class probability corresponding to the multiple second voice data; when the class probability corresponding to the multiple second voice data reaches the preset probability value, the desired classifier is obtained.

In one embodiment, the classification module 206 is further configured to use the acoustic feature vector and the spectral feature vector as the input of the classifier to obtain decision values corresponding to the acoustic feature vector and the spectral feature vector; when the decision value is the first threshold A voice tag is added to the feature vector or the spectral feature vector; when the decision value is the second threshold, a non-speech tag is added to the acoustic feature vector or the spectral feature vector.

In one embodiment, a computer device is provided, and the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 3 . For example, the computer device may be a terminal, and the terminal may be, but not limited to, various devices having functions of inputting speech, such as smart phones, tablet computers, notebook computers, personal computers, and portable wearable devices. The computer equipment includes a processor, memory, a network interface, and a voice input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by a processor, implements a voice endpoint detection method. The voice input device of the computer equipment may include a microphone, and may also include an external earphone or the like.

Those skilled in the art can understand that the structure shown in FIG. 3 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. More or fewer components are shown in the figures, either in combination or with different arrangements of components.

In one embodiment, a computer device is provided, including a memory and a processor, where a computer program is stored in the memory, and when the processor executes the computer program, the processor implements the following steps: acquiring a noisy speech signal, and extracting a corresponding noise signal. Acoustic feature and spectral feature; convert the acoustic feature and spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector; obtain a classifier, input the acoustic feature vector and spectral feature vector into the classifier, and obtain the acoustic feature with the added voice tag vector and the spectral feature vector of the added voice tag; analyze the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain the corresponding voice signal; determine the corresponding start point and end point of the voice signal according to the time sequence of the voice signal .

In one embodiment, the processor further implements the following steps when executing the computer program: converting the noisy speech signal into a noisy speech spectrum; performing time domain analysis and/or frequency domain analysis and/or on the noisy speech spectrum Or transform domain analysis to obtain the acoustic features corresponding to the noisy speech signal.

In one embodiment, the processor further implements the following steps when executing the computer program: converting the noisy speech signal into a noisy speech spectrum, calculating the noisy speech amplitude spectrum according to the noisy speech spectrum; Perform dynamic noise estimation on the speech spectrum to obtain the noise amplitude spectrum; estimate the speech amplitude spectrum of the pure speech signal according to the noisy speech amplitude spectrum and the noise amplitude spectrum; use the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum to generate the noisy speech signal corresponding spectral features.

In one embodiment, the processor further implements the following steps when executing the computer program: extracting a preset number of frames before and after the current frame in the acoustic feature and the spectral feature; calculating the current frame by using the preset number of frames before and after the current frame Corresponding mean vector and/or variance vector; perform logarithmic domain transformation on the acoustic feature and spectral feature after calculating the mean vector and/or variance vector corresponding to the current frame, to obtain the transformed acoustic feature vector and spectral feature vector.

In one embodiment, the processor further implements the following steps when executing the computer program: acquiring noisy speech data with added speech category labels, and obtaining an initial classifier by training the noisy speech data; acquiring a first validation set, first The verification set includes a plurality of first voice data; the plurality of first voice data are input into the classifier to obtain the class probabilities corresponding to the plurality of first voice data; the class probabilities corresponding to the plurality of first voice data are screened, and the The first voice data outputted is added with category labels to obtain a verification set with added category labels; training is performed by using the verification set and training set with added category labels to obtain a verification classifier; a second verification set is obtained, and the second verification set includes a plurality of Second voice data; input multiple second voice data into the verification classifier to obtain the class probability corresponding to the multiple second voice data; when the class probability corresponding to the multiple second voice data reaches the preset probability value, obtain the required classifier.

In one embodiment, the processor also implements the following steps when executing the computer program: using the acoustic feature vector and the spectral feature vector as the input of the classifier to obtain decision values corresponding to the acoustic feature vector and the spectral feature vector; when the decision value is the first When the threshold is set, a voice tag is added to the acoustic feature vector or the spectral feature vector; when the decision value is the second threshold, a non-voice tag is added to the acoustic feature vector or the spectral feature vector.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: acquiring a noisy speech signal, extracting acoustic features corresponding to the noisy speech signal and Spectral feature; convert the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector; obtain the classifier, input the acoustic feature vector and the spectral feature vector to the classifier, and obtain the acoustic feature vector and add the voice tag. The spectral feature vector of the voice tag; analyze the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain the corresponding voice signal; determine the corresponding start point and end point of the voice signal according to the time sequence of the voice signal.

In one embodiment, the computer program, when executed by the processor, further implements the following steps: converting the noisy speech signal into a noisy speech spectrum; performing time domain analysis and/or frequency domain analysis and/or frequency domain analysis on the noisy speech spectrum and/or transform domain analysis to obtain acoustic features corresponding to the noisy speech signal.

In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: converting a noisy speech signal into a noisy speech spectrum, calculating a noisy speech amplitude spectrum according to the noisy speech spectrum; Perform dynamic noise estimation on the noisy speech spectrum to obtain the noise amplitude spectrum; estimate the speech amplitude spectrum of the pure speech signal according to the noisy speech amplitude spectrum and the noise amplitude spectrum; use the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum to generate the noisy speech The corresponding spectral characteristics of the signal.

In one embodiment, the computer program further implements the following steps when executed by the processor: extracting a preset number of frames before and after the current frame in the acoustic feature and the spectral feature; calculating the current frame by using a preset number of frames before and after the current frame mean vector and/or variance vector corresponding to the frame; perform logarithmic domain transformation on the acoustic feature and spectral feature after calculating the mean vector and/or variance vector corresponding to the current frame to obtain the transformed acoustic feature vector and spectral feature vector.

In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: acquiring the noisy speech data to which the speech category label is added, and obtaining an initial classifier by training the noisy speech data; acquiring a first verification set; A verification set includes a plurality of first voice data; inputting the plurality of first voice data into a classifier to obtain class probabilities corresponding to the plurality of first voice data; screening the class probabilities corresponding to the plurality of first voice data, The selected first voice data is added with a category label to obtain a verification set with the added category label; the verification set and the training set with the added category label are used for training to obtain a verification classifier; a second verification set is obtained, and the second verification set includes multiple second voice data; inputting a plurality of second voice data into the verification classifier to obtain the class probabilities corresponding to the plurality of second voice data; when the class probabilities corresponding to the plurality of second voice data reach the preset probability value, obtain the required classifier.

In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: the acoustic feature vector and the spectral feature vector are used as the input of the classifier, and the decision value corresponding to the acoustic feature vector and the spectral feature vector is obtained; when the decision value is the first When a threshold is set, a voice tag is added to the acoustic feature vector or the spectral feature vector; when the decision value is the second threshold, a non-voice tag is added to the acoustic feature vector or the spectral feature vector.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (10)

1. A voice endpoint detection method, comprising: acquiring a noisy speech signal, and extracting the acoustic features corresponding to the noisy speech signal; extracting the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum of the noisy speech signal; generating a spectral feature corresponding to the noisy speech signal according to the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum; Converting the acoustic feature and the spectral feature to obtain the corresponding acoustic feature vector and spectral feature vector; Obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier, obtaining the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag; Analyzing the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain a corresponding voice signal; The starting point and the ending point corresponding to the voice signal are determined according to the time sequence of the voice signal. 2. The method according to claim 1, characterized in that, before extracting the corresponding acoustic features and spectral features of the noisy speech signal, further comprising: converting the noisy speech signal into a noisy speech spectrum; Time domain analysis and/or frequency domain analysis and/or transform domain analysis are performed on the noisy speech spectrum to obtain acoustic features corresponding to the noisy speech signal. 3. The method according to claim 1, wherein the extracting the noisy speech amplitude spectrum, the noise amplitude spectrum and the speech amplitude spectrum of the noisy speech signal comprises: Converting the noisy speech signal into a noisy speech spectrum, and calculating a noisy speech amplitude spectrum according to the noisy speech spectrum; Perform dynamic noise estimation on the noisy speech spectrum according to the noisy speech amplitude spectrum to obtain a noise amplitude spectrum; The speech amplitude spectrum of the clean speech signal is estimated from the noisy speech amplitude spectrum and the noise amplitude spectrum. 4. The method according to claim 1, wherein the converting the acoustic feature and the spectral feature comprises: Extracting a preset number of frames before and after the current frame in the acoustic feature and the spectral feature; Calculate the mean value vector and/or the variance vector corresponding to the current frame by using a preset number of frames before and after the current frame; Perform logarithmic domain transformation on the acoustic feature and the spectral feature after calculating the mean vector and/or the variance vector corresponding to the current frame, to obtain the transformed acoustic feature vector and spectral feature vector. 5. The method according to claim 1, wherein before the step of obtaining the classifier, it further comprises: Obtaining the noisy speech data to which the speech category label is added, and obtaining an initial classifier by training the noisy speech data; acquiring a first verification set, where the first verification set includes a plurality of first voice data; inputting a plurality of first voice data into the initial classifier to obtain class probabilities corresponding to the plurality of first voice data; Screening the class probabilities corresponding to the plurality of first voice data, adding a class label to the selected first speech data, and obtaining a verification set with the class label added; Use the verification set with the added category label and the noisy speech data with the added voice category label for training to obtain a verification classifier; acquiring a second verification set, the second verification set includes a plurality of second voice data; Inputting a plurality of second voice data into the verification classifier to obtain class probabilities corresponding to the plurality of second voice data; When the class probabilities corresponding to the plurality of second speech data reach the preset probability value, the required classifier is obtained. 6. The method according to any one of claims 1 to 5, wherein the step of using the classifier to classify the acoustic feature vector and the spectral feature vector comprises: Using the acoustic feature vector and the spectral feature vector as the input of the classifier, obtain the decision value corresponding to the acoustic feature vector and the spectral feature vector; When the decision value is the first threshold, adding a voice tag to the acoustic feature vector or the spectral feature vector; When the decision value is the second threshold, a non-speech tag is added to the acoustic feature vector or the spectral feature vector. 7. A voice endpoint detection device, comprising: an extraction module for acquiring a noisy speech signal, and extracting the acoustic features corresponding to the noisy speech signal; extracting the noisy speech amplitude spectrum, noise amplitude spectrum and speech amplitude spectrum of the noisy speech signal; The speech amplitude spectrum, the noise amplitude spectrum, and the speech amplitude spectrum generate spectral features corresponding to the noisy speech signal; a conversion module, configured to convert the acoustic features and spectral features to obtain corresponding acoustic feature vectors and spectral feature vectors; A classification module, for obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier to obtain the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag; The parsing module is used to analyze the acoustic feature vector of the added voice tag and the spectral feature vector of the added voice tag to obtain a corresponding voice signal; determine the corresponding starting point and termination of the voice signal according to the time sequence of the voice signal point. 8. The device according to claim 7, wherein the conversion module is also used to extract a preset number of frames before and after the current frame in the acoustic feature and the spectral feature; Set the number of frames to calculate the mean vector and/or variance vector of the current frame; perform logarithmic domain transformation on the acoustic features and spectral features after calculating the mean vector and/or variance vector corresponding to the current frame, to obtain the converted acoustic feature vector and spectrum Feature vector. 9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when the processor executes the computer program . 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images