CN114495950A - Voice deception detection method based on deep residual shrinkage network - Google Patents

Abstract

The invention discloses a voice deception detection method based on a deep residual shrinkage network, which comprises the steps of preprocessing voice to be detected, and transforming preprocessed voice feature data to obtain corresponding constant Q cepstrum coefficient features, Mel frequency cepstrum coefficient features and spectrogram features; then, a depth residual shrinkage network is adopted to respectively process the constant Q cepstrum coefficient characteristic, the Mel frequency cepstrum coefficient characteristic and the spectrogram characteristic to obtain three corresponding depth characteristics; inputting the three depth features into a deep neural network classifier respectively, and calculating to obtain detection scores corresponding to the three depth features; and finally, fusing the detection scores corresponding to the three depth features, and judging whether the voice to be detected is real voice. The method improves the distinguishing characteristic learning ability in the complex acoustic environment, improves the system generalization and has wider application scene.

Description

A voice deception detection method based on deep residual shrinkage network

technical field

The present application belongs to the technical field of speech detection and deep learning, and in particular, relates to a speech deception detection method based on a deep residual shrinkage network.

Background technique

In recent years, biometric-based identity authentication technology plays an increasingly important role in data security and pass-through authentication. Due to the development of acquisition and sensing devices, automatic speaker verification technology has received extensive attention and has been applied to smart device login, access control, online banking, etc. However, various speech forgery techniques threaten the security performance of automatic speaker verification systems. Currently, four types of forged speech deception attacks have been identified: speech synthesis, speech conversion, speech imitation, and replay, which can generate fake speech similar to legitimate user speech. voice. Logical access attacks based on speech synthesis and speech conversion are perceptually indistinguishable from real speech, making it more challenging to distinguish fake speech from real user speech. More and more studies have confirmed that the automatic speaker verification system has serious vulnerabilities in the face of various malicious spoofing attacks on the database.

In order to deal with the threat of spoofing attacks, researchers have been working on finding effective anti-spoofing methods. At present, the speech spoofing detection system mainly consists of two parts: front-end feature extraction and back-end classifier. Different from the acoustic features used in general speaker verification and speech processing, speech deception detection requires the development of acoustic features that are more suitable for speech deception detection. After the acoustic features are extracted, a classifier with excellent performance is used to complete the distinction between true and false speech. Among the traditional machine learning methods, Gaussian Mixture Model (GMM) is the most classic classification model, which has the advantage of short training time, but limited detection accuracy; with the rise of deep learning, various types of Deep neural networks have also been applied to speech deception detection. Convolutional Neural Networks (CNN) are widely used in extracting audio features with good representation learning ability. Recurrent Neural Network (RNN) has memory due to recurrent unit and threshold structure, so it has certain advantages in the processing of time series problems.

Although the training performance of existing methods has improved, in practical applications they encounter unknown types of attacks, which often have different statistical distributions from known attacks, resulting in a large performance gap between training and application, indicating that The generalization ability of deception detection systems to unknown attacks still needs to be improved. In addition, due to noise, reverberation and channel interference in the real environment, the performance of various spoofing detection systems in the face of complex acoustic environments is greatly reduced.

SUMMARY OF THE INVENTION

The purpose of this application is to provide a speech deception detection method based on a deep residual shrinkage network, so as to detect speech deception in a complex acoustic environment.

In order to achieve the above purpose, the technical solutions of the present application are as follows:

A voice deception detection method based on a deep residual shrinkage network, comprising:

The speech to be detected is preprocessed, and the preprocessed speech feature data is transformed to obtain the corresponding constant Q cepstral coefficient feature, Mel frequency cepstral coefficient feature and spectrogram feature;

Using the deep residual shrinkage network, the constant Q cepstral coefficient feature, the Mel frequency cepstral coefficient feature and the spectrogram feature are processed respectively, and the corresponding three depth features are obtained;

Inputting the three kinds of depth features into the deep neural network classifier respectively, and calculating the detection scores corresponding to the three kinds of depth features;

The detection scores corresponding to the three depth features are fused to determine whether the speech to be detected is real speech.

Further, the deep residual shrinkage network includes a residual shrinkage building unit, and the residual shrinkage building unit includes a convolution module, an adaptive threshold learning module and a soft threshold module, and the output of the convolution module is subjected to an adaptive threshold. The learning module learns to obtain a threshold, and the soft threshold module processes the outputs of the convolution module and the adaptive threshold learning module to highlight highly discriminative sound information.

Further, the deep residual shrinkage network for processing constant-Q cepstral coefficient features stacks 6 residual shrinkage building units, and the deep residual shrinkage network for processing Mel-frequency cepstral coefficient features stacks 9 residuals Shrinkage building units, a deep residual shrinkage network for processing spectrogram features stacks 6 residual shrinkage building units.

Further, the deep neural network classifier includes a Dropout layer, a first fully hidden connection layer, a Leak-Relu activation function layer, a second hidden fully connected layer and a LogSoftmax layer.

Further, the probability of randomly dropping weights of the Dropout layer is 50%.

Further, the detection scores corresponding to the three depth features are fused, and the formula is as follows:

Among them, Score _fuse is the joint detection score after fusion, w _i is the fusion weight, and si is the detection score corresponding to the _i -th depth feature.

Further, converting the preprocessed speech feature data to obtain corresponding constant Q cepstral coefficient features, Mel frequency cepstral coefficient features and spectrogram features, including:

Perform constant Q transformation on the preprocessed speech feature data, then calculate the power spectrum and take the logarithm, then perform uniform resampling, and finally obtain constant Q cepstral features through discrete cosine transformation;

Perform short-time Fourier transform (STFT) on the preprocessed speech feature data, then map the spectrum to the Mel spectrum through filtering, and finally obtain the Mel frequency cepstral coefficient feature through discrete cosine transform;

Short-time Fourier transform is performed on the preprocessed speech feature data, and the size of each component is calculated and finally converted to logarithmic scale to obtain spectrogram features.

A speech deception detection method based on a deep residual shrinkage network proposed in this application, a deep residual shrinkage network is constructed, and the adaptive threshold learning module based on the deep attention mechanism and the residual shrinkage construction unit of the soft threshold module are used to make Each speech signal determines an independent threshold according to its own acoustic environment, and forces unimportant features to zero to eliminate noise-related information, learn more discriminative advanced features, and improve the ability to learn discriminative features in complex acoustic environments. . Aiming at the problem of poor generalization performance of detection methods, three different acoustic feature extraction algorithms, CQCC, MFCC and Spectrogram, are used to represent speech features more comprehensively. Feature joint detection to improve system generalization and wider application scenarios.

Description of drawings

Fig. 1 is the flow chart of the voice deception detection method based on the deep residual shrinkage network of the present application;

Fig. 2 is the overall framework structure diagram of the application network;

3 is a schematic diagram of feature extraction of the application;

4 is a schematic structural diagram of the residual shrinkage construction unit of the present application;

FIG. 5 is a structural diagram of a deep residual shrinkage network of the present application;

FIG. 6 is a schematic structural diagram of the DNN classifier of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

In one embodiment, as shown in FIG. 1, a method for detecting speech deception based on a deep residual shrinkage network is provided, including:

Step S1: Preprocess the speech to be detected, and transform the preprocessed speech feature data to obtain the corresponding constant Q cepstral coefficient feature, Mel frequency cepstral coefficient feature and spectrogram feature.

As shown in Figure 2, this step realizes feature extraction. First, the speech to be detected is divided into frames, and the pad filling operation is performed on the speech data with less than 64,000 sample points. Finally, data normalization is performed to complete the data preprocessing. Voice data is different from video data, and there is no concept of frame, but for transmission and storage, the audio data collected by this application is segment by segment. In order for the program to perform batch processing, it will be segmented according to the specified length (time period or number of samples) and structured into a programmed data structure, which is framing. The speech signal is not stable on the macro level, but stable on the micro level, and has short-term stability (the speech signal can be considered to be approximately unchanged within 10-30ms). This can divide the speech signal into some short segments. For processing, each short segment is called a frame (CHUNK).

Then referring to Fig. 3, transform the preprocessed speech feature data, wherein:

Constant Q cepstral coefficient feature (CQCC): perform constant Q transformation on the preprocessed speech feature data, then calculate the power spectrum and take the logarithm, then perform uniform resampling, and finally obtain the constant Q cepstral feature through discrete cosine transformation .

For example, a constant Q transform is performed on the preprocessed speech feature data with a sampling frequency of 16 kHz, the power spectrum is calculated and the logarithm is taken, and then the uniform resampling is performed with a uniform resampling period of 16, and finally the constant is obtained by discrete cosine transform. Q cepstral coefficient feature.

Mel frequency cepstral coefficient feature (MFCC): short-time Fourier transform STFT is performed on the preprocessed speech feature data, and then the spectrum is mapped to the Mel spectrum by filtering, and finally the Mel frequency cepstral coefficient is obtained by discrete cosine transform. feature.

For example, a short-time Fourier transform (STFT) is performed on the preprocessed speech feature data at a sampling frequency of 16 kHz, and then the spectrum is mapped to the Mel spectrum through a filter bank, and finally the Mel frequency cepstral coefficient MFCC is obtained through discrete cosine transform. feature. This application selects the first 24 coefficients and concatenates the MFCC with its first-order MFCC and second-order derivative MFCC to produce the MFCC feature representation, which is a two-dimensional matrix.

Spectrogram feature: perform short-time Fourier transform on the preprocessed speech feature data, calculate the size of each component, and finally convert it to logarithmic scale to obtain spectrogram features.

For example, the short-time Fourier transform is computed over a Hamming window (window size=2048, overlap 25%) on the preprocessed speech feature data. The magnitude of each component is then calculated and converted to a logarithmic scale, and the output matrix captures the time-frequency characteristics of the input audio waveform, resulting in the spectrogram features.

Step S2: Using the trained deep residual shrinkage network, the constant Q cepstral coefficient feature, the Mel-frequency cepstral coefficient feature and the spectrogram feature are processed respectively to obtain three corresponding depth features.

In this embodiment, a deep residual shrinkage network is designed and constructed, and the constant Q cepstral coefficient feature, the Mel frequency cepstral coefficient feature and the spectrogram feature obtained by the transformation in step S1 are processed to obtain the depth feature of the detected speech.

In a specific embodiment, the residual shrinkage construction unit adopted by the deep residual shrinkage network of the present application includes a convolution module, an adaptive threshold learning module and a soft threshold module.

Figure 4 shows the Residual Shrinkage Building Unit (RSBU) in the improved Deep Residual Shrinkage Networks (DRSN). The residual shrinkage construction unit in this embodiment includes a convolution module, an adaptive threshold learning module, and a soft threshold module. The output of the convolution module is learned by the adaptive threshold learning module to obtain a threshold. The output of the adaptive threshold learning module is processed to highlight highly discriminative sound information.

The adaptive threshold learning module includes absolute value function (Absolute), global average pooling (GAP), two fully connected layers (FC), BN layer, ReLu function and an activation function. The output size of the global average pooling is 1×1, and the next two fully connected layers have 32 neural units. The BN connected between them uses BatchNorm1d, and the parameter is set to 32. After the fully connected layer, the scaling parameters are obtained, and finally the sigmoid function is used to make the scaling parameters within the range of (0,1), which can be expressed as:

where _zc is the feature of the neuron in the cth channel, and _αc is the corresponding scaling factor.

The scaling factor α _c is multiplied by the result of global average pooling, and finally a positive threshold τ _c is obtained.

The soft threshold module in this embodiment is inserted into the Residual Shrinkage Building Unit (RSBU) as a nonlinear transformation layer, compares the speech feature data with the threshold, and then performs processing on the feature data, zeroing the features close to zero, and retaining usefulness The positive and negative features of , can flexibly reduce noise according to the current audio conditions and highlight highly discriminative sound information.

The soft threshold function can be expressed as:

Where x is the input feature, y is the output feature, τ is the threshold, and there are different thresholds τ _c corresponding to different channels.

The deep residual shrinking network in this embodiment is shown in FIG. 5 , which respectively processes the constant Q cepstral coefficient CQCC feature, the Mel-frequency cepstral coefficient MFCC feature, and the spectrogram feature. Among them, the deep residual shrinkage network for processing constant Q cepstral coefficient features stacks 6 residual shrinkage building units, and the deep residual shrinkage network for processing Mel-frequency cepstral coefficient features stacks 9 residual shrinkage builds unit, a deep residual shrinkage network for processing spectrogram features stacks 6 residual shrinkage building units.

The stacking of multiple RSBUs in this embodiment can enhance the ability of various nonlinear transformations to learn and discriminate features, and the soft threshold is used as a shrinking function, which is more conducive to eliminating information related to noise. Taking the constant Q cepstral coefficient CQCC feature, the Mel-frequency cepstral coefficient MFCC feature, and the spectrogram feature as inputs, respectively, three kinds of depth features are obtained.

Step S3: Input the three kinds of depth features into the deep neural network classifier respectively, and calculate the detection scores corresponding to the three kinds of depth features.

In this step, the deep neural network classifier is used to generate the detection score of the speech to be detected, referring to FIG. 6 .

Specifically, the three deep features are respectively passed to the deep neural network classifier (DNN classifier), including: Dropout layer, first fully hidden connection layer, Leak-Relu activation function layer, second hidden fully connected layer and LogSoftmax layer .

The deep residual shrinkage network adjusts the number of elements in the last dimension, and reshapes the shape of the deep feature. The settings of the first fully connected layer of the DNN classifier for different features are different. The parameters of the first hidden fully connected layer of the CQCC-DSRN model, MFCC-DRSN model and Spectrogram-model are (32,128), (480,128), ( 160, 128). The second hidden fully-connected layer has a leaky-Relu activation function with α of 0.01. The parameters of this hidden fully-connected layer are set to (128, 2), and there are logits units that generate classification in this layer. The LogSoftmax layer is then used to perform a softmax operation on all elements of each row and take the log value to convert the logits into detection scores.

For the convenience of expression, this application names the network model jointly composed of the deep residual shrinkage network DRSN and the DDN classifier as the MFCC-DRSN model, the CQCC-DRSN model and the Spectrogram-DRSN model for three different features.

Step S4, fuse the detection scores corresponding to the three depth features to determine whether the speech to be detected is real speech.

In this step, the detection unit is combined to perform weighted fusion of the detection scores generated by the MFCC-DRSN model, the CQCC-DRSN model and the Spectrogram-DRSN model to obtain the final speech deception detection result.

Considering that in general deception detection tasks, the number of real voices is much less than the number of fake voices, all models are trained using a minimized weighted cross-entropy loss function, where the ratio of weights assigned to real and fake voices is 9:1, In order to alleviate the imbalance of training data distribution. The parameters of the training model with the best performance are applied to the evaluation data set, and the single-class feature-DRSN model detection score is obtained after model detection.

A logistic regression model is established with three single-class features-DRSN model detection scores as independent variables and detection results as dependent variables, expressed as:

Wherein, w ₁ , w ₂ , and w ₃ represent weight parameters, and s ₁ , s ₂ , and s ₃ represent detection scores.

After model processing, the regression constant is obtained and normalized, and finally the weight of the model is obtained. Through weighted fusion of score files to achieve joint detection, the detection scores are fused in the logistic function of polynomial regression, which is expressed as:

Among them, Score _fuse is the joint detection score after fusion, w _i is the fusion weight, and si is the detection score corresponding to the _i -th depth feature.

The joint detection score is analyzed to obtain a decision threshold. If the joint detection score is less than the threshold, it is judged as real speech, otherwise it is judged as deceitful speech.

The speech deception detection method based on the deep residual shrinkage network of the present application has the following advantages:

(1) Construct a deep residual shrinkage network DRSN, and design a residual shrinkage construction unit RSBU including an adaptive threshold learning module based on a deep attention mechanism and a soft threshold module, so that each speech signal can determine an independent threshold according to its own acoustic environment, Unimportant features are forced to zero to eliminate noise-related information and learn more discriminative high-level features, thereby improving the ability of discriminative feature learning in complex acoustic environments.

(2) Three different acoustic feature extraction algorithms, CQCC, MFCC, and Spectrogram, are used to represent speech features more comprehensively, and the features are used as network inputs respectively, and weights are generated for each model according to its output performance and multi-feature joint detection is performed to improve the System generalizability.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (7)

1. A voice spoofing detection method based on a deep residual shrinkage network is characterized in that the voice spoofing detection method based on the deep residual shrinkage network comprises the following steps:

preprocessing the voice to be detected, and transforming the voice characteristic data after preprocessing to obtain corresponding constant Q cepstrum coefficient characteristics, Mel frequency cepstrum coefficient characteristics and spectrogram characteristics;

respectively processing the constant Q cepstrum coefficient characteristic, the Mel frequency cepstrum coefficient characteristic and the spectrogram characteristic by adopting a depth residual shrinkage network to obtain three corresponding depth characteristics;

inputting the three depth features into a deep neural network classifier respectively, and calculating to obtain detection scores corresponding to the three depth features;

and fusing the detection scores corresponding to the three depth features, and judging whether the voice to be detected is real voice.

2. The method according to claim 1, wherein the deep residual shrinking network comprises a residual shrinking construction unit, the residual shrinking construction unit comprises a convolution module, an adaptive threshold learning module and a soft threshold module, the output of the convolution module is learned by the adaptive threshold learning module to obtain a threshold, and the soft threshold module processes the outputs of the convolution module and the adaptive threshold learning module to highlight high-discriminant sound information.

3. The method of claim 2, wherein the depth residual shrinkage network for processing constant Q cepstral coefficient features is stacked with 6 residual shrinkage building units, the depth residual shrinkage network for processing mel-frequency cepstral coefficient features is stacked with 9 residual shrinkage building units, and the depth residual shrinkage network for processing spectrogram feature is stacked with 6 residual shrinkage building units.

4. The deep residual shrinkage network-based spoofing detection method of claim 1, wherein the deep neural network classifier comprises a Dropout layer, a first fully hidden connected layer, a Leak-Relu activation function layer, a second hidden fully connected layer, and a LogSoftmax layer.

5. The method of claim 4, wherein the random discard weight probability of the Dropout layer is 50%.

6. The method according to claim 1, wherein the detection scores corresponding to the three depth features are fused, and the formula is as follows:

wherein Score is_fuseFor fused joint detection score, w_iTo fuse the weights, s_iAnd the detection score corresponding to the ith depth feature.

7. The method according to claim 1, wherein the transforming the preprocessed voice feature data to obtain corresponding constant Q cepstrum coefficient features, mel-frequency cepstrum coefficient features, and spectrogram features comprises:

constant Q transformation is carried out on the preprocessed voice characteristic data, then a power spectrum is calculated and logarithm is taken, then uniform resampling is carried out, and finally constant Q cepstrum system characteristics are obtained through discrete cosine transformation;

performing short-time Fourier transform (STFT) on the preprocessed voice feature data, mapping the frequency spectrum to a Mel frequency spectrum through filtering, and finally obtaining Mel frequency cepstrum coefficient features through discrete cosine transform;

and performing short-time Fourier transform on the preprocessed voice feature data, calculating the size of each component, and finally converting the component into logarithmic scales to obtain the spectrogram features.

Images