CN111564163B - RNN-based multiple fake operation voice detection method - Google Patents

Abstract

The invention discloses a speech detection method for multiple forgery operations based on RNN, which includes the following steps: 1) Obtain original speech samples, perform M kinds of forgery processing on the original speech samples, and obtain the speech after M forgery operations and 1 Unprocessed original speech, perform feature extraction on the above speech, obtain the LFCC matrix of the training speech sample, and send it to the RNN classifier network for training to obtain a multi-class training model; 2) Obtain a test speech, and Perform feature extraction on the test voice, obtain the LFCC matrix of the test voice data, and send it to the RNN classifier trained in step 1) for classification. Each test voice gets an output probability, and all output probabilities are combined as the final prediction result: If If the prediction result is the original speech, the test speech is recognized as the original speech; if the prediction result is the speech that has undergone a certain forgery operation, the test speech is recognized as the forged speech that has undergone the corresponding forgery operation.

Description

A speech detection method for multiple forgery operations based on RNN

Technical field

The present invention relates to a speech detection method, in particular to a speech detection method for multiple forgery operations based on RNN.

Background technique

With the continuous enhancement of the functions of voice editing software, non-professionals can easily modify the voice content. If some criminals maliciously forge and modify the voice, and even use the modified voice for news reporting, judicial evidence collection, scientific research and other fields, this will pose a huge threat and even have an immeasurable impact on social stability. . Digital voice forensics method is the detection of forgery operations, which plays a vital role in identifying the originality and authenticity of audio materials. It is a key research topic in the current field of multimedia forensics.

Most of the existing digital voice forensic detection technologies detect a single forgery operation, that is, the forensic personnel assume that the voice to be detected will undergo a specific forgery operation. Mengyu Qiao et al. proposed a detection algorithm based on the statistical characteristics of quantized MDCT coefficients and their derivatives to detect up-converted and down-converted MP3 audio files, generate a reference audio signal by recompressing and calibrating the audio, and then use support vectors Machine classification, experimental results show that this method effectively detects MP3 double compression and can detect audio processing history for digital forensics. For example, Wang Lihua and others proposed a history detection of pitch-shifted speech processing based on convolutional neural network. They applied four different pitch-shifting software to three speech libraries to perform pitch-shifting, and used CNN to perform intra-bank and library-based integration of the pitch-shifting factors of speech. The detection rate reaches more than 90% between time and pitch modification methods.

Existing digital voice forensic detection technology can detect a single forgery operation, and the detection rate can reach a very high level. However, in practical applications, evidence collectors often cannot predict the specific operation of forgery, and using a certain operation classifier for detection may result in misjudgment.

At present, most existing digital forensics work that is applicable to various forgery operations focuses on the field of digital images, and there is still little research on digital voice forensics. In the field of digital speech, Luo Weiqi's team designed a convolutional neural network model that can be used to detect audio processing operations with default settings in two different audio editing software, and provides better results. This method can be significantly better than existing methods. Some forensic methods are based on manual features. However, although this experiment conducted pioneering research on the detection of various forgery operations in speech, there are still some problems that cannot be ignored, such as the complexity of the calculation is too high and the application scenarios for the forgery operations are too ideal.

Contents of the invention

The technical problem to be solved by the present invention is to provide an RNN-based voice detection method for multiple forgery operations in view of the deficiencies in the above-mentioned existing technologies, which can improve the detection accuracy.

The technical solution adopted by the present invention to solve the above technical problems is: a multiple forgery operation speech detection method based on RNN, which is characterized by: including the following steps:

1) Training network: Obtain original voice samples, perform M forgery processes on the original voice samples, and obtain M forged voices and 1 unprocessed original voice. Features are extracted from the original speech to obtain the LFCC matrix of the training speech sample, which is sent to the RNN classifier network for training to obtain a multi-classification training model;

2) Speech recognition: Obtain a test speech, perform feature extraction on the test speech, obtain the LFCC matrix of the test speech data, and send it to the RNN classifier trained in step 1) for classification. Each test speech gets an output probability. , combine all output probabilities as the final prediction result: if the prediction result is the original speech, the test speech is recognized as the original speech; if the prediction result is the speech that has undergone a certain forgery operation, the test speech is recognized as the corresponding forgery operation. Fake voice.

Preferably, in steps 1) and 2), the steps to obtain the LFCC matrix are:

1) FFT: First, preprocess the speech and calculate the spectrum energy E(i,k) of each speech frame after FFT:

Among them, i is the number of speech frames, k is the frequency component, x _i (m) is the speech signal data of the i-th frame, and N is the number of Fourier transforms;

Then calculate the energy of the spectrum energy E(i,k) of each frame after passing through the triangular filter bank:

Among them, H _i (k) represents the frequency response of the triangular filter, f (l) is the filter function of the l-th triangular filter, S (i, l) is the spectral line energy after passing through the triangular filter group, and l represents The number of the triangular filter, L is the total number of triangular filters;

2) DCT: Use DCT to calculate the output data lfcc(i,n) of each triangular filter bank:

Among them, n represents the spectral line after DCT of the i-th frame;

3) Obtain the LFCC statistical moment: take the 12th order LFCC coefficient of lfcc(i,n), calculate the mean and correlation coefficient, and obtain the LFCC matrix extracted from a segment of speech:

Among them, x _s,1 ...x _1,n are the calculated n LFCCs of the s-th frame speech data.

Preferably, the RNN classifier includes an LSTM network, a Dropout layer, a fully connected layer and a Softmax layer connected in sequence, and the Dropout layer is connected to the last LSTM network.

Preferably, the LSTM network has two, and the parameters are set to (64,128) and (128,64) respectively.

Preferably, the LSTM network uses tanh activation function.

Preferably, the Dropout function value of the Dropout layer is 0.5.

Preferably, the original voice is in WAV format.

Compared with the existing technology, the advantage of the present invention is that: it adopts the speech cepstrum feature, classifies the output result probability through the circular neural network, improves the accuracy of speech detection, is more suitable for digital speech carriers, and can identify different forgery traces; The computational complexity of shared parameters in the RNN network is greatly reduced compared with existing deep learning-based methods.

Description of drawings

Figure 1 is a process diagram of the extraction process of LFCC statistical moments of the speech detection method according to the embodiment of the present invention;

Figure 2 is a schematic diagram of the overall framework of the speech detection method according to the embodiment of the present invention;

Figure 3 is a network structure diagram of the speech detection method according to the embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the drawings, in which the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " "Back", "Left", "Right", "Vertical", "Horizontal", "Top", "Bottom", "Inside", "Outside", "Clockwise", "Counterclockwise", "Axis", The orientation or positional relationship indicated by "radial direction", "circumferential direction", etc. are based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply the device or element to which they are referred. It must have a specific orientation, be constructed and operate in a specific orientation. Since the disclosed embodiments of the present invention can be arranged in different directions, these terms indicating directions are only for illustration and should not be regarded as limiting, such as "up", "up", "Down" is not necessarily limited to a direction opposite or consistent with the direction of gravity. In addition, features defined as “first” and “second” may explicitly or implicitly include one or more of these features.

A speech detection method for multiple forgery operations based on RNN (Recurrent Neural Network) is implemented by constructing a recurrent neural network framework based on cepstrum features. Referring to Figure 2, the framework consists of two parts: first, the cepstral features of the speech samples are extracted, and then sent to the designed network framework for classification to achieve the task of identifying various forgery operations.

Specifically, in the present invention, speech feature extraction is implemented in the following manner. The cepstral feature used in the present invention is Linear Frequency Cepstral Coefficients (LFCC). Speech cepstrum feature is one of the most commonly used feature parameters in speech technology. It characterizes human auditory characteristics and is widely used in speaker recognition.

LFCC is an even distribution of bandpass filters from low frequencies to high frequencies. The extraction process of LFCC statistical moments of the present invention can be seen in Figure 1:

1) FFT: First, preprocess the speech and calculate the spectral energy E(i,k) of each speech frame after fast Fourier Transform (FFT):

Among them, i is the number of speech frames, k is the frequency component, x _i (m) is the speech signal data of the i-th frame, and N is the number of Fourier transforms.

Calculate the energy of the spectrum energy E(i,k) of each frame after passing through the triangular filter bank:

Among them, H _i (k) represents the frequency response of the triangular filter, f (l) is the filter function of the l-th triangular filter, S (i, l) is the spectral line energy after passing through the triangular filter group, and l represents The number of the triangular filter, L is the total number of triangular filters.

2) DCT: Then, use Discrete Cosine Transform (DCT) to calculate the output data lfcc(i,n) of each triangular filter bank:

Among them, n represents the spectral line after DCT of the i-th frame.

3) Obtain the LFCC statistical moment: take the 12th order LFCC coefficient of lfcc(i,n), calculate the mean and correlation coefficient, the above steps can be realized through the existing matlab function, assuming that a certain segment of preprocessed speech has a total of s frames , then the LFCC matrix extracted from this speech segment is:

Among them, x _s,1 ...x _1,n are the calculated n LFCCs of the s-th frame speech data.

See Figure 3. The network framework uses an RNN classifier. The choice of the number of network layers of the RNN classifier is crucial to the optimization algorithm. A deeper network can learn more knowledge, but at the same time training also takes a long time and may Will overfit. Therefore, in the present invention, the network structure of the RNN classifier is proposed as shown in Figure 3. The network structure contains two LSTM networks, with parameters set to (64,128) and (128,64) respectively. The tanh activation function is used to improve model performance. It also includes the Dropout layer, the fully connected layer (dense) and the Softmax layer that are connected in sequence. The Dropout layer is connected to the last LSTM network. Set the value of the Dropout function to 0.5, which helps to reduce overfitting. After dimensionality reduction through the fully connected layer, use the Softmax layer (Softmax classifier) to output the probability. The overall iterative training of the network framework is set to 50 laps. Certain adjustments can be made during specific training.

Referring again to Figure 2, the speech detection method includes the following steps:

1) First, the network framework needs to be trained. Suppose there are M types of forgery operations, M types of forgery processing are performed on the original speech, and M+1 types of speech samples are obtained, including M speech after the forgery operation and 1 unprocessed original speech. In the present invention, there are certain constraints on the input of original speech, and a certain amount of WAV format audio sample library needs to be provided as training data for the network framework. Perform feature extraction on the above M+1 speech samples to obtain the LFCC matrix of the training speech samples, and send them to the designed RNN classifier network for training to obtain a multi-classification training model; multiple original speech sounds can be stored in the database Samples, feature extraction is performed on each original speech sample and sent to the RNN classifier for training;

2) Then, the detection and recognition results are obtained through the trained network framework: when a test speech is obtained, feature extraction is performed on it, the LFCC matrix of the test speech data is obtained, and the result is sent to the already trained RNN classifier. Classification. Each test speech will get an output probability, and all output probabilities are combined as the final prediction result. If the prediction result is the original speech, the test speech is recognized as the original speech; if the prediction result is the speech that has undergone a certain forgery operation, the test speech is recognized as the forged speech. The forensic examiner can judge whether a certain piece of speech has been forged based on the experimental results.

Claims (6)

1. A speech detection method for multiple forgery operations based on RNN, which is characterized by: including the following steps: 1) Training network: Obtain original voice samples, perform M forgery processes on the original voice samples, and obtain M forged voices and 1 unprocessed original voice. Features are extracted from the original speech to obtain the LFCC matrix of the training speech sample, which is sent to the RNN classifier network for training to obtain a multi-classification training model; 2) Speech recognition: Obtain a test speech, perform feature extraction on the test speech, obtain the LFCC matrix of the test speech data, and send it to the RNN classifier trained in step 1) for classification. Each test speech gets an output probability. , combine all output probabilities as the final prediction result: if the prediction result is the original speech, the test speech is recognized as the original speech; if the prediction result is the speech that has undergone a certain forgery operation, the test speech is recognized as the corresponding forgery operation. fake speech; In steps 1) and 2), the steps to obtain the LFCC matrix are: 1) FFT: First, preprocess the speech and calculate the spectrum energy E(i,k) of each speech frame after FFT: Among them, i is the number of speech frames, k is the frequency component, x _i (m) is the speech signal data of the i-th frame, and N is the number of Fourier transforms; Then calculate the energy of the spectrum energy E(i,k) of each frame after passing through the triangular filter bank: Among them, H _i (k) represents the frequency response of the triangular filter, f (l) is the filter function of the l-th triangular filter, S (i, l) is the spectral line energy after passing through the triangular filter group, and l represents The number of the triangular filter, L is the total number of triangular filters; 2) DCT: Use DCT to calculate the output data lfcc(i,n) of each triangular filter bank: Among them, n represents the spectral line after DCT of the i-th frame; 3) Obtain the LFCC statistical moment: take the 12th order LFCC coefficient of lfcc(i,n), calculate the mean and correlation coefficient, and obtain the LFCC matrix extracted from a segment of speech: Among them, x _s,1 ...x _1,n are the calculated n LFCCs of the s-th frame speech data. 2. The multiple forgery operation speech detection method based on RNN according to claim 1, characterized in that: the RNN classifier includes an LSTM network, a Dropout layer, a fully connected layer and a Softmax layer connected in sequence, and the Dropout layer Connect with the last LSTM network. 3. The multiple forgery operation speech detection method based on RNN according to claim 2, characterized in that: the LSTM network has two, and the parameters are set to (64, 128) and (128, 64) respectively. 4. The multiple forgery operation speech detection method based on RNN according to claim 2, characterized in that: the LSTM network uses a tanh activation function. 5. The multiple forgery operation speech detection method based on RNN according to claim 2, characterized in that: the Dropout function value of the Dropout layer is 0.5. 6. The multiple forgery operation speech detection method based on RNN according to claim 1, characterized in that: the original speech is in WAV format.