CN114550751A - Voice speed-doubling attack detection method based on rhythm characteristics and random forest classifier - Google Patents

Abstract

The invention discloses a voice double-speed attack detection method based on prosodic features and random forest classifiers, and belongs to the technical fields of voice recognition and security. Obtain an audio data set, including normal audio and double-speed adversarial audio; extract jitter features, vibrato features and harmonic-to-noise ratio features of all audio in the audio data set to form feature vectors; use the feature vectors of normal audio and double-speed adversarial audio to train random forest classification The trained random forest classifier is used for speech double-speed attack detection. The invention can efficiently detect the voice double-speed spoofing attack through the existing microphone and voice hardware of the voice recognition system, has the characteristics of low cost and high attack detection accuracy, and can be used for the security protection of the voice recognition system on smart devices such as mobile phones. , has a wide range of needs and application prospects.

Description

Speech double-speed attack detection method based on prosodic features and random forest classifier

technical field

The invention belongs to the field of speech recognition technology and security technology, and in particular relates to a speech double-speed attack detection method based on prosodic features and random forest classifiers.

Background technique

Automatic Speech Recognition (ASR) systems can recognize speech and output speech recognition text. Existing popular ASR systems include open source systems (such as Kaldi and DeepSpeech) and commercial systems (such as Google Cloud Speech-to-Text, Baidu ASR, and iFlytek). For the input audio, the ASR system first performs signal processing to reduce noise and remove irrelevant frequency components; then divide the processed audio signal into short segments to extract features such as Mel Frequency Cepstral Coefficients (MFCC); finally, use the extracted features , the most likely word sequence is inferred by a pre-trained speech recognition model.

Time-scale Modification (TSM) refers to an operation of making an audio clip play faster or slower. Common audio players or audio editing software will use the double-speed operation to change the playback speed of the audio without changing the audio pitch. The double-speed operation mainly includes three steps: 1) signal decomposition, 2) frame relocation and change, and 3) signal reconstruction. It first decomposes the input audio into analysis frames with shorter length and overlapping each other. The length of the analysis frame is usually millisecond level, and it preserves the tonal content of the frame in the original audio. According to the processing algorithm of frame transformation, there are various implementation schemes for double-speed operation, such as FFmpeg, SoundTouch, Waveform Similarity Overlap-Add (WSOLA) and Phase Vocoder (PV-TSM). In the existing implementations, FFmpeg is the most commonly used double-speed operation in commercial audio players and audio editing software.

However, studies have shown that existing speech recognition systems are vulnerable to speech double-speed attacks. Voice double-speed attack refers to simply speeding up or slowing down the original audio to make ASR identify errors. It only needs to modify the speed of one segment (for example, 20ms) to achieve an untargeted attack, and the targeted attack only needs to go through several rounds of optimization. Achieving malicious goals with good audibility (such as opening a door). With the development of voice technology and electronic equipment, the threshold of voice double-speed attack is getting lower and lower, the effect is getting better and better, and the harm is getting bigger and bigger. Therefore, in this case, it is urgent to propose an efficient and low-cost voice double-speed attack detection method.

At present, there are many related researches to protect speech attacks by detecting the noise and distortion introduced in the process of generating speech adversarial samples. However, this type of detection method is difficult to detect speech double-speed attacks without adding noise.

SUMMARY OF THE INVENTION

In order to solve the technical problems existing in the above-mentioned background art, the present invention provides a voice double-speed attack detection method based on prosodic features and random forest classifiers. The double-speed confrontation audio is obtained through the voice double-speed attack based on the particle swarm algorithm, and the extraction can be effectively real. The three types of features that reflect the difference between normal audio and double-speed adversarial audio prosody, based on random forest classifier for attack detection, can accurately and effectively detect speech spoofing attacks represented by double-speed attacks against speech recognition systems.

In order to achieve the above object, the technical scheme adopted in the present invention is:

A speech double-speed attack detection method based on prosodic features and random forest classifier, including:

Obtain audio datasets, including normal audio and double-speed adversarial audio;

Extract the jitter features, vibrato features and harmonic-to-noise ratio features of all audios in the audio data set to form feature vectors;

The random forest classifier is trained using the feature vectors of normal audio and double-speed adversarial audio, and the trained random forest classifier is used for speech double-speed attack detection.

Further, the double-speed confrontation audio is obtained by performing a double-speed operation on the normal audio, and no additional noise is added.

Further, described shaking feature includes jitt feature, jitta feature, rap feature, ppq5 feature, and the calculation formula is:

Among them, T _i represents the duration of the ith jitter in the audio, N represents the total number of jitters in the audio, and jitta, jitt, jitt _rap , and jitt _ppq5 are the jitt feature, jitta feature, rap feature, and ppq5 feature, respectively.

Further, the vibrato feature includes Shim feature, ShdB feature, apq5 feature, apq11 feature, and the calculation formula is:

Among them, A _i represents the duration of the ith vibrato in the audio, M represents the total number of vibrato in the audio, and shim, ShdB, apq5, and apq11 represent the Shim feature, ShdB feature, apq5 feature, and apq11 feature, respectively.

Further, the calculation formula of the harmonic noise ratio characteristic is:

Among them, sig _per is the ratio of the audio periodic signal, sig _noise is the ratio of the noise in the audio signal, and hnr represents the harmonic-to-noise ratio feature. By designing different analysis window lengths, two harmonic-to-noise ratio features hnr05 and hnr15 are obtained, so The analysis window length of hnr15 described is 3 times that of hnr05.

Further, when using the trained random forest classifier to detect speech double-speed attack, extract the jitter feature, vibrato feature and harmonic noise ratio feature of the audio to be detected, form a feature vector, and use the feature vector as the trained random forest classifier. input to get the detection result.

The beneficial effects of the present invention are:

For the double-speed attack audio without adding noise, which is difficult to be identified by traditional methods, the present invention proposes to use 10-dimensional features such as jitter, vibrato, harmonic noise ratio, etc., to attack the difference between the double-speed attack audio and normal audio in terms of prosody features. The detection provides effective feature data and is combined with the random forest algorithm for replay attack detection. In the voice spoofing attack, even if the attacker produces a sound that is very similar to the real user's voice, the sound will produce differences in jitter, vibrato, and harmonic noise ratio when it passes through double-speed. Therefore, this method can be used to detect voice double-speed spoofing attacks. .

The invention can efficiently detect the voice double-speed spoofing attack through the existing microphone and voice hardware of the voice recognition system, has the characteristics of low cost and high attack detection accuracy, and can be used for the security protection of the voice recognition system on smart devices such as mobile phones. , has a wide range of needs and application prospects.

Description of drawings

FIG. 1 is a schematic flowchart of a method for detecting a speech double-speed attack based on a prosodic feature and a random forest classifier according to an embodiment of the present invention.

Detailed ways

The technical solutions provided by each embodiment of the present invention will be described in detail below with reference to the accompanying drawings. The flow charts shown in the figures are merely illustrative and do not necessarily include all steps. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may be changed according to the actual situation.

A speech double-speed attack is a speech adversarial attack that simply speeds up or slows down the original audio, rather than adding distractions. In order to realize the recognition of both normal audio and double-speed confrontation audio without adding noise, the present invention uses the "unnatural" distortion brought by the double-speed operation, and proposes a voice double-speed attack detection method based on prosodic features and random forest classifiers. The microphone of the device receives the voice signal to extract specific features and mark them, use the marked features to train the random forest classifier, use the trained classifier to detect the voice double-speed attack on the voice signal to be tested, and output whether it is the double-speed voice confrontation result. . It mainly includes the following steps:

1) Feature extraction:

The overall process is shown in Figure 1. The input audio first goes through the feature extraction stage. The change of speech rate will cause the pronunciation difference between double-speed adversarial audio and normal audio. The key to using prosodic features for attack detection is to extract features with large differences from normal audio and double-speed adversarial audio. Prosody feature is one of the important forms of human language and emotional expression, including pitch, intonation, voice jitter, etc. The present invention adopts 4 kinds of jitter features, namely jitta, jitt, rap, ppq5; 4 kinds of vibrato features, namely Shim, ShdB , apq5, apq11; and two harmonic-to-noise ratio features, namely hnr05 and hnr15, as a combination of prosodic features to distinguish normal audio from double-speed confrontation audio, specifically:

1.1) Jitter

Jitter characterizes the frequency change between signal cycles, which is mainly caused by the lack of control over the vibration of the vocal cords. The jitter value represents the hoarseness of the voice of a specific object, and generally patients with a disease have strong jitter. The present invention screens out four kinds of jitter features, including jitt, jitta, rap, and ppq5. Let T _i denote the duration of the ith jitter in the audio, and N denote the total number of jitters in the audio.

The jitta feature is expressed as the mean absolute difference between successive jitter periods, and is calculated as:

The jitt characteristic is expressed as the average absolute difference between consecutive cycles divided by the average jitter cycle, and the calculation formula is:

The rap characteristic is the relative average jitter, expressed as the average absolute difference between a jitter and the average value of its two adjacent jitters divided by the average jitter period, denoted as jitt _rap , and the calculation formula is:

The ppq5 characteristic is expressed as the average absolute difference between a jitter and the average of its four adjacent jitters divided by the average jitter period, denoted as jitt _ppq5 , and the calculation formula is:

1.2) Vibrato

Vibrato characterizes the amplitude fluctuations of the sound wave signal, which is related to human breathing and noise, and is generally regarded as a pathological rhythmic characteristic. The present invention screens four vibrato features, including Shim, ShdB, apq5, and apq11. A _i represents the duration of the i-th vibrato in the audio, and M represents the total number of vibrato in the audio.

The Shim feature represents the mean absolute difference between the amplitudes of the vibrato divided by the mean amplitude of the vibrato, calculated as:

The ShdB feature is expressed as the mean absolute logarithm of the vibrato amplitude and is calculated as:

Similar to ppq5, apq5 represents the average absolute difference between the amplitude of a vibrato and the average of its four adjacent vibrato amplitudes divided by the average amplitude of the vibrato. The calculation formula is:

apq11 is the mean absolute difference between one vibrato amplitude and the average of its ten adjacent vibrato amplitudes divided by the mean vibrato amplitude, calculated as:

1.3) Harmonic noise ratio

The harmonic-to-noise ratio characterizes the ratio of periodic components to non-periodic components in an audio signal, reflecting the quality of human speech, such as soft or strong tone. The present invention screens two kinds of harmonic noise ratio characteristics, including hnr05 and hnr15.

hnr, the harmonic noise ratio, indicates the degree of sound wave periodicity in dB. The difference between hnr05 and hnr15 is the length of the analysis window. The calculation formula is:

where sig _per is the ratio of periodic signals, and sig _noise is the ratio of noise in the signal. The analysis window length represents the data length used to calculate the harmonic-to-noise ratio, and the analysis window length for hnr15 is 3 times that of hnr05.

2) Attack detection:

For the detection of double-speed attacks on speech, prosodic features can better characterize the pronunciation distortion caused by changes in speech rate, so as to detect double-speed adversarial audio.

The present invention uses the interface of Google TTS to convert text into speech, generates one hundred sentences of normal audio, and uses the particle swarm optimization algorithm to generate one hundred sentences of double-speed confrontation audio based on normal audio. As described in the above section, the present invention extracts a total of 10 There are four kinds of prosody features, including four kinds of jitter features, namely jitt, jitta, rap, and ppq5; four kinds of vibrato features, namely, Shim, ShdB, apq5, and apq11; and two kinds of harmonic-to-noise ratio features, namely, hnr05 and hnr15. Then use the random forest classifier to learn the 10-dimensional features of normal audio and double-speed adversarial audio. Each decision tree outputs the probability that the detection result is normal audio or double-speed adversarial audio. The random forest relies on the voting selection of the decision tree to decide the final classification. result.

In this embodiment, the detection result of random forest is compared with decision tree and SVM classifier, as shown in Table 1:

Table 1. Detection results of different classifiers for double-speed attack

Correct rate recall Accuracy F1 Score Equal error rate AUC random forest 0.921 0.897 0.903 0.916 0.192 0.838 decision tree 0.865 0.866 0.864 0.872 0.14 0.866 SVM 0.783 0.741 0.864 0.842 0.241 0.849

It can be seen that random forest, decision tree, and SVM all realize the recognition of normal audio and double-speed confrontation audio, and the correct rate of random forest reaches 92.1%.

The invention obtains the double-speed confrontation audio through the voice double-speed attack based on the particle swarm algorithm, and extracts three types of features that can effectively and truly reflect the difference in rhythm between the normal audio and the double-speed confrontation audio. According to the regular differences in pronunciation prosody between normal audio and double-speed adversarial audio, the features are input into the random forest classifier, and then normal audio and double-speed adversarial audio are detected. In the case of voice spoofing attacks, even if the attacker produces a voice that is very similar to the real user's voice, the voice will inevitably cause a certain degree of pronunciation difference when it passes through the double speed. Although the difference is very small, it is difficult for the traditional feature extraction method to perform However, the speech double-speed attack detection method based on prosodic features and random forest classifiers proposed by the present invention can effectively detect this double-speed confrontation audio without adding noise, and provides guidance for the protection of the speech recognition system.

It should be understood by those skilled in the art that the embodiments of the present invention shown in the above description and the accompanying drawings are only examples and do not limit the present invention. The objects of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been shown and described in the embodiments, and the embodiments of the present invention may be modified or modified in any way without departing from the principles.

Claims (6)

1. a speech speed attack detection method based on rhythm feature and random forest classifier, is characterized in that, comprises: Obtain audio datasets, including normal audio and double-speed adversarial audio; Extract the jitter features, vibrato features and harmonic-to-noise ratio features of all audios in the audio data set to form feature vectors; The random forest classifier is trained using the feature vectors of normal audio and double-speed adversarial audio, and the trained random forest classifier is used for speech double-speed attack detection. 2. the voice double-speed attack detection method based on prosodic feature and random forest classifier according to claim 1, is characterized in that, described double-speed confrontation audio is obtained by carrying out double-speed operation with normal audio, and does not add extra noise. 3. the voice double-speed attack detection method based on prosodic feature and random forest classifier according to claim 1, is characterized in that, described shaking feature comprises jitt feature, jitta feature, rap feature, ppq5 feature, and calculation formula is:

Among them, T _i represents the duration of the ith jitter in the audio, N represents the total number of jitters in the audio, and jitta, jitt, jitt _rap , and jitt _ppq5 are the jitt feature, jitta feature, rap feature, and ppq5 feature, respectively. 4. the voice double-speed attack detection method based on prosodic feature and random forest classifier according to claim 1, is characterized in that, described vibrato feature comprises Shim feature, ShdB feature, apq5 feature, apq11 feature, and calculation formula is:

Among them, A _i represents the duration of the ith vibrato in the audio, M represents the total number of vibrato in the audio, and shim, ShdB, apq5, and apq11 represent the Shim feature, ShdB feature, apq5 feature, and apq11 feature, respectively. 5. the speech double-speed attack detection method based on prosodic feature and random forest classifier according to claim 1, is characterized in that, described harmonic noise ratio characteristic calculation formula is:

Among them, sig _per is the ratio of the audio periodic signal, sig _noise is the ratio of the noise in the audio signal, and hnr represents the harmonic-to-noise ratio feature. By designing different analysis window lengths, two harmonic-to-noise ratio features hnr05 and hnr15 are obtained, so The analysis window length of hnr15 described is 3 times that of hnr05. 6. the method for detecting double-speed attacks based on prosodic feature and random forest classifier according to claim 1, it is characterized in that, when utilizing trained random forest classifier to carry out double-speed attack detection of voice, extract the jitter feature of audio to be detected , vibrato feature and harmonic noise ratio feature to form a feature vector, and the feature vector is used as the input of the trained random forest classifier to obtain the detection result.

Images