CN113921042A - Speech desensitization method, device, electronic device and storage medium - Google Patents

Abstract

The present invention provides a voice desensitization method, device, electronic equipment and storage medium, wherein the method includes: determining the voice data to be desensitized; inputting the amplitude spectrum of each voice frame in the voice data into a sensitive voice detection model to obtain a sensitive voice detection model. The speech mask of each speech frame output by the speech detection model; the sensitive speech detection model is based on the sample general speech and the general mask of each sample speech frame, and the sample sensitive word speech and the sensitivity mask of each sample speech frame Training is obtained; based on the speech mask of each speech frame, the sensitive information in the speech data is eliminated. The method, device, electronic device and storage medium provided by the present invention can output a voice mask based on the amplitude spectrum of the input voice frame by using the sensitive voice detection model to locate the sensitive word, desensitize the sensitive word, and overcome the sensitive information It is easy to leak or over-eliminate the voice and the problems of low efficiency and recognition rate, and realize the real-time and accurate desensitization of the voice.

Description

Speech desensitization method, device, electronic device and storage medium

technical field

The present invention relates to the technical field of artificial intelligence, and in particular, to a voice desensitization method, device, electronic device and storage medium.

Background technique

Nowadays, there are more and more application scenarios of real-time voice communication, such as telephone communication, voice communication tool communication and online conference communication. In the process of voice communication, a large amount of sensitive information is often included, such as personal ID number, name, address, etc. , price, registration information, etc. Therefore, it is necessary to shield these sensitive information to protect the security of voice information, and the existing method of shielding sensitive voice information is mainly to detect sensitive words after converting speech into text, desensitize and replace the sensitive words, and replace the replaced content. Converted to voice output to the receiver. However, this method cannot cope with real-time scenarios, sensitive information is easily leaked or excessively eliminates speech, and the recognition rate is low.

SUMMARY OF THE INVENTION

The present invention provides a voice desensitization method, device, electronic device and storage medium, which are used to solve the defects in the prior art that the real-time scene cannot be dealt with, the sensitive information is easily leaked or the voice is excessively eliminated, and the recognition rate is low.

The present invention provides a speech desensitization method, comprising:

Determine the voice data to be desensitized;

The amplitude spectrum of each speech frame in the speech data is input to the sensitive speech detection model, and the speech mask of each speech frame output by the sensitive speech detection model is obtained; the sensitive speech detection model is based on the sample general speech and wherein The general mask of each sample speech frame, and the sample sensitive word speech and the sensitive mask of each sample speech frame are obtained by training;

Based on the speech mask for each speech frame, sensitive information in the speech data is eliminated.

According to a speech desensitization method provided by the present invention, the amplitude spectrum of each speech frame in the speech data is input into the sensitive speech detection model, and the speech mask of each speech frame output by the sensitive speech detection model is obtained. code, including:

The amplitude spectrum of each speech frame in the described speech data is input to the sensitive speech detection model frame by frame, and the speech mask of each speech frame output by the described sensitive speech detection model frame by frame is obtained;

Wherein, the input voice frame and the output voice frame at the same time differ by a preset number of frames, the input voice frame is the voice frame corresponding to the amplitude spectrum of the input sensitive voice detection model, and the output voice frame is from the sensitive voice Detect the speech frame corresponding to the speech mask output in the model.

According to a speech desensitization method provided by the present invention, the amplitude spectrum of each speech frame in the speech data is input into the sensitive speech detection model frame by frame, and the frame-by-frame output of each speech frame by the sensitive speech detection model is obtained. Voice masks, including:

The amplitude spectrum of each voice frame in the voice data is input into the sensitive voice detection model frame by frame, and the sensitive voice detection model is based on the amplitude spectrum of each voice frame, the continuous preset frame number of voice frames after each voice frame. Amplitude spectrum, and the state vector of the frame before each speech frame, encode the state vector of each speech frame, and perform sensitive speech detection based on the state vector of each speech frame, and obtain the frame-by-frame output of the sensitive speech detection model. Voice mask.

According to a voice desensitization method provided by the present invention, the method for eliminating sensitive information in the voice data based on the voice mask of each voice frame includes:

Based on the voice mask of each voice frame, desensitization processing is performed on the amplitude spectrum of each voice frame in the voice data to obtain desensitized amplitude spectrum data;

Perform inverse transformation on the desensitized amplitude spectrum data to obtain desensitized speech data.

According to a speech desensitization method provided by the present invention, the desensitization process is performed on the amplitude spectrum of each speech frame in the speech data based on the speech mask of each speech frame, and the desensitized amplitude is obtained. Spectral data, including:

Based on the speech mask of each speech frame, locate the speech segment of the sensitive word from the speech data, and determine the desensitization mode of each speech segment of the sensitive word;

If the desensitization method is information desensitization, the amplitude spectrum of the speech frame with the specified number of frames after the sensitive word speech segment, or the amplitude spectrum of each speech frame in the sensitive word speech segment and the sensitive word The amplitude spectrum of the speech frame with the specified number of frames after the speech segment is desensitized;

If the desensitization method is sensitive word desensitization, the amplitude spectrum of each speech frame in the speech segment of the sensitive word is desensitized.

According to a voice desensitization method provided by the present invention, the voice segment of a sensitive word is located from the voice data based on the voice mask of each voice frame, including:

Determine that in the voice data, the voice frame whose voice mask is less than the preset voice mask threshold is used as a sensitive word voice frame;

A plurality of consecutive sensitive word speech frames whose frame number is greater than the preset frame number threshold are regarded as a sensitive word speech segment.

According to a speech desensitization method provided by the present invention, the specific steps for determining the desensitization mode of each sensitive word speech segment include:

From the end of the speech segment of the sensitive word, intercept the preset intercepted frames and several speech frames forward as the speech segment to be classified;

Inputting the speech segment to be classified into a speech classification model to obtain a desensitization mode of the sensitive word speech segment output by the speech classification model;

The speech classification model is obtained by training based on the speech segment of the sample sensitive word and its desensitization method label.

According to a speech desensitization method provided by the present invention, the sample sensitive word speech is obtained by adding noise to the original sensitive word speech based on sample noise, and the sensitivity mask is based on the amplitude spectrum of the sample noise and the original sensitive word. The magnitude spectrum of the word speech is determined.

The present invention also provides a voice desensitization device, comprising:

A determination module for determining the voice data to be desensitized;

The prediction module is used for inputting the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model to obtain the speech mask of each speech frame output by the sensitive speech detection model; the sensitive speech detection model is based on The sample general speech and the general mask of each sample speech frame, and the sample sensitive word speech and the sensitivity mask of each sample speech frame are trained;

An elimination module, configured to eliminate sensitive information in the speech data based on the speech mask of each speech frame.

The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, when the processor executes the program, the above-mentioned speech desensitization is implemented steps of the method.

The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of any of the above-mentioned speech desensitization methods.

The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned speech desensitization methods.

The present invention provides a speech desensitization method, device, electronic device and storage medium, by inputting the amplitude spectrum of each frame in the speech data to be desensitized into a sensitive speech detection model, to obtain the speech mask of each speech frame Then, according to the voice mask of each voice frame, the sensitive information in the voice data is eliminated, and the sensitive voice detection model is used to output the voice mask based on the amplitude spectrum of the input voice frame to locate the sensitive word, and remove the sensitive word. Sensitivity reduces the process of inter-conversion between speech and text, improves recognition efficiency, overcomes the problems of sensitive information being easily leaked or over-eliminated speech, and low efficiency and recognition rate, and realizes real-time and accurate desensitization of speech.

Description of drawings

In order to explain the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are the For some embodiments of the invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is one of the schematic flow charts provided by the present invention for speech desensitization method;

Fig. 2 is the schematic flowchart of obtaining the voice data after desensitization provided by the present invention;

3 is a schematic flowchart of a method for determining a sensitive speech segment provided by the present invention;

Fig. 4 is the schematic flow chart of determining the voice segment desensitization mode of sensitive words provided by the present invention;

Fig. 5 is the second schematic flow chart of the method for speech desensitization provided by the present invention;

Fig. 6 is the training method of the sensitive speech detection model provided by the present invention;

Fig. 7 is the processing flow schematic diagram of the sensitive speech detection model provided by the present invention;

8 is a schematic structural diagram of a voice desensitization device provided by the present invention;

FIG. 9 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The existing speech desensitization method mainly converts the speech data into text, and compares the converted text with the sensitive words in the preset sensitive word database. If it is detected that the text contains sensitive words. , then replace the speech paragraph of the sensitive word part with other speech paragraphs to complete the speech desensitization operation.

However, the voice-to-text method in the existing method is inefficient and cannot meet the desensitization requirements of real-time voice. The subsequent steps are highly dependent on the voice-to-text conversion. Once there is a problem in the conversion of text, the subsequent desensitization steps will all fail. Moreover, the voice-to-text operation is not carried out for sensitive words, which will lead to the problem of low recognition rate of sensitive words; and when sensitive words are detected, the voice paragraphs of the sensitive words are replaced with other voice paragraphs, and the replaced voice paragraphs need to be replaced. Aligning on the original voice segment, but aligning the voice data through text, is prone to errors and leads to the leakage of sensitive information. In order to reduce the chance of sensitive information leakage, the voice segments near the sensitive information will be excessively eliminated, affecting the actual experience. .

Therefore, how to achieve real-time and accurate speech desensitization is an urgent problem to be solved in the art.

Fig. 1 is one of the schematic flow charts of the method for speech desensitization provided by the present invention. As shown in Fig. 1, an embodiment of the present invention provides a method for speech desensitization, including:

Step 110, determine the voice data to be desensitized;

Specifically, the voice data to be desensitized may be real-time voice data, such as voice calls, online conferences, etc., or non-real-time voice data, such as recording, intercom-like voice messages, etc., which is not limited in this embodiment of the present invention . Among them, desensitization is to eliminate sensitive information in voice data, and sensitive information can be information that needs to be eliminated, such as uncivilized language or illegal language, etc., or positioning information, which is used to locate and eliminate the designation behind the sensitive information. Information, such as password, address, or identity information, etc., is not limited in this embodiment of the present invention.

Step 120, input the amplitude spectrum of each speech frame in this speech data to the sensitive speech detection model, obtain the speech mask of each speech frame output by this sensitive speech detection model; This sensitive speech detection model is based on the sample general speech and wherein The general mask of each sample speech frame, and the sample sensitive word speech and the sensitive mask of each sample speech frame are obtained by training;

Considering the conversion errors caused by ambient sounds, the speaker's local accent or semantics when converting speech to text, the speech data in the time domain is converted to the frequency domain for processing, and then the processed frequency domain signal is restored to the time domain. What it transforms is the representation of voice data, and conversion errors are not prone to occur. In the embodiment of the present invention, the voice data is not converted to text, but the voice data is transformed into the frequency domain for processing. Further, the amplitude spectrum, as one of the frequency-domain representation forms of the speech signal, can be used as a means for extracting speech features. Therefore, the embodiment of the present invention uses the amplitude spectrum of the speech data to detect and locate sensitive information.

Specifically, in this embodiment of the present invention, the amplitude spectrum of each voice frame in the voice data can be input into a pre-trained sensitive voice detection model, and the sensitive voice detection model can detect the amplitude spectrum of each voice frame in the voice data , to obtain the speech mask corresponding to the speech frame for use in step 130 . It should be noted that the amplitude spectrum of each voice frame in the voice data is obtained by dividing the voice data into frames by sliding window, and then performing time-frequency transformation on each frame of the obtained voice frame; the voice mask output by the sensitive voice detection model is obtained. The code is a judgment label for whether the voice frame is a voice frame containing sensitive information. For example, whether the voice frame is a voice frame containing sensitive information can be reflected in the form of "0" and "1", or it can be reflected in the form of probability. The possibility that the speech frame contains sensitive information is not specifically limited for comparison in the embodiments of the present invention.

Before performing step 120, a sensitive speech detection model needs to be pre-trained. Specifically, when the sensitive speech detection model is trained, the sample general speech and the sample sensitive word speech need to be used as samples, and the general speech of each sample speech frame in the sample general speech needs to be used as samples. mask, and the sensitivity mask of each sample speech frame in the sample sensitive word speech as the label. Among them, the sample general voice and the sample sensitive word voice are both sample voices. The difference is that the sample general voice does not contain sensitive words, that is, the sample general voice is the voice that does not need desensitization processing. The mask reflects that the corresponding sample speech frame does not contain sensitive information, so it is recorded as a general mask here, and the sample sensitive word speech is the speech that only contains sensitive words, that is, the sample sensitive word is the speech that needs to be silenced as a whole, and the sample sensitive word speech The mask of each speech frame in , reflects that the corresponding sample speech frame contains sensitive information, so it is recorded as a sensitive mask here.

During the training of the sensitive speech detection model, the amplitude spectrum of the sample general speech and the sample sensitive word speech can be input into the model under training respectively, so as to obtain the amplitude of the model for each sample speech frame in the sample general speech and the sample sensitive word speech. The prediction mask of the spectral output. On this basis, the prediction mask of each sample speech frame in the sample general speech is compared with the pre-labeled general mask, and the prediction of each sample speech frame output in the sample sensitive word speech is compared. The mask is compared with the pre-labeled sensitive mask to obtain the loss value of model training, and the model parameters are iteratively updated based on the loss value. In this process, the model can learn the relationship between the amplitude spectrum of the sample speech frame and the mask label. The corresponding relationship between them enables the trained sensitive speech detection model to have the ability to detect sensitive information based on the amplitude spectrum.

Step 130: Eliminate sensitive information in the voice data based on the voice mask of each voice frame.

Specifically, according to the voice mask of each voice frame output in step 120, the sensitive information in the voice data is eliminated. It should be noted that to eliminate the sensitive information in the voice data, the amplitude spectrum of the original voice frame can be desensitized according to the voice mask, and then the desensitized amplitude spectrum can be restored into voice data. The voice mask of the frame, locates a plurality of voice frames of continuous sensitive information to form a voice segment, so that the voice segment is eliminated as a whole or the voice segment with a specified number of frames after the voice segment is eliminated, which is not limited in this embodiment of the present invention. .

In a voice desensitization method provided by an embodiment of the present invention, a sensitive voice detection model is used to perform sensitive voice detection on the amplitude spectrum of an input voice frame, so as to locate the sensitive information in the voice data, thereby realizing voice desensitization. There is a process of inter-conversion between speech and text, which avoids the problems of easy leakage of sensitive information caused by speech transcription, excessive elimination of speech, and low transcription efficiency and recognition rate, and realizes real-time and accurate speech desensitization.

Based on the above embodiment, step 120 includes:

Input the amplitude spectrum of each speech frame in the speech data to the sensitive speech detection model frame by frame, and obtain the speech mask of each speech frame output by the sensitive speech detection model frame by frame;

The input speech frame and the output speech frame at the same time differ by a preset number of frames, the input speech frame is the speech frame corresponding to the amplitude spectrum of the input sensitive speech detection model, and the output speech frame is the speech mask output from the sensitive speech detection model the corresponding speech frame.

Considering the need for desensitization processing of real-time speech, based on the characteristics of low latency and high throughput of streaming processing, it is introduced into the embodiments of the present invention. Specifically, the amplitude spectrum of each speech frame in the speech data is input into the sensitive speech detection model frame by frame, and the speech mask corresponding to each speech frame output by the sensitive speech detection model frame by frame is obtained. , to achieve stream-sensitive voice detection for voice data and ensure the real-time performance of voice desensitization.

At the same time, considering that in streaming sensitive speech detection, the information content in the amplitude spectrum of a single speech frame is less, in order to improve the effect of model prediction, the information in the amplitude spectrum of multiple speech frames is needed to assist in sensitive speech detection. Therefore, in When predicting the current speech frame, you can set the time delay for the input and output of the model, so that when the model performs sensitive speech detection on the current speech frame, it can also refer to the speech frame according to the amplitude spectrum of the current speech frame. After that, the information of the amplitude spectrum of each speech frame is preset.

Specifically, the input speech frame and the output speech frame at the same moment differ by a preset number of frames. It should be noted that the preset frame number that differs between the input voice frame and the output voice frame at the same time is a preset frame delay. When a voice frame is input, the output of the sensitive voice detection model is the voice frame before the voice frame. A voice frame with a preset frame delay, for example: the preset frame delay is 2 frames, and the current input is the amplitude spectrum of the third voice frame, and the output is the voice mask of the first voice frame at the same time.

The method provided by the embodiment of the present invention realizes the sensitive information detection of the voice frame in the streaming voice data by inputting the voice frame frame by frame and outputting the sensitive information detection result frame by frame.

In the detection of sensitive information of streaming speech frames, the delay of the preset number of frames is applied, so that the sensitive speech detection model can combine the amplitude spectrum of the current frame with the current The sensitive information detection result of the previous frame is used to detect the sensitive information of the current frame, so as to improve the reliability of detection.

Based on the above embodiment, in step 120, the amplitude spectrum of each speech frame in the speech data is input into the sensitive speech detection model frame by frame, and the speech mask of each speech frame output by the sensitive speech detection model frame by frame is obtained, including:

inputting the amplitude spectrum of each speech frame in the speech data frame by frame into the sensitive speech detection model, and the sensitive speech detection model is based on the amplitude spectrum of each speech frame, the amplitude spectrum of the continuous preset number of speech frames after each speech frame, and The state vector of the frame before each speech frame, encode the state vector of each speech frame, and perform sensitive speech detection based on the state vector of each speech frame to obtain the speech mask of each speech frame output by the sensitive speech detection model frame by frame.

Specifically, considering the timing of speech itself, the information represented by the speech frame is often related to the information represented by the speech frames before and after the speech frame. When performing streaming sensitive speech detection based on the sensitive speech detection model, the sensitive speech detection model can realize sensitive speech detection based on the left field of view of the model through its own model structure, such as causal convolution, LSTM (Long Short-Term Memory, long short-term memory Network) and other model structures have a left view, that is, in the process of streaming, the model can encode the state vector of the speech frame according to the amplitude spectrum of any input speech frame and the state vector of the frame before the speech frame. , and then perform sensitive speech detection according to the state vector of each speech frame.

However, the effect of directly predicting the speech mask of the current frame by the streaming sensitive speech detection model without the right view is not ideal. Therefore, in the embodiment of the present invention, the amplitude spectrum of the continuous preset number of speech frames after each speech frame is used as The right view of the sensitive speech detection model assists the sensitive speech detection model to predict and output the speech mask of each speech frame.

Specifically, in the process of inputting the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model frame by frame, for any speech frame, the model can be based on the number of consecutive preset frames after the input speech frame. The amplitude spectrum of each voice frame, the amplitude spectrum of each voice frame itself, and the state vector of the frame before each voice frame, encode the state vector of each voice frame, and then perform sensitive voice detection according to the state vector of each voice frame to obtain the sensitive voice. The speech mask of each speech frame output by the detection model frame by frame.

It should be noted that the state vector is encoded by the sensitive voice detection model according to the amplitude spectrum of the voice frame and the amplitude spectrum of the contextual voice frame; when the sensitive voice detection model processes a voice frame, the subsequent prediction of the voice frame is required. Set the amplitude spectrum of several speech frames (as the right field of view of the model), the amplitude spectrum of the speech frame, and the state vector of the amplitude spectrum of the frame before the frame (as the left field of view of the model) for encoding to obtain the speech frame Then, according to the state vector of the voice frame, sensitive voice detection is performed to obtain the voice mask of the frame. Reliability and accuracy are also guaranteed.

For example, if the preset number of frames is 2, and the currently processed voice frame is the fourth voice frame, the sensitive voice detection model will use the last two voice frames of the currently processed voice frame as the right field of view of the model, that is, the fifth voice frame and the sixth voice frame. frame, then encode the state vector of the previous speech frame of the currently processed speech frame, that is, the state vector of the 3rd speech frame, the amplitude spectrum of the currently processed speech frame, that is, the 4th speech frame, and the amplitude spectrum of the 5th speech frame and the 6th speech frame, The state vector corresponding to the 4th speech frame is obtained, thereby obtaining the speech mask of the 4th speech frame. Here, the model outputs the speech mask of the 4th speech frame after the amplitude spectrum input model of the 6th speech frame, that is, There is a 2 frame delay.

Based on the above embodiment, the sensitive speech detection model may adopt a CNN (Convolutional Neural Network, convolutional neural network) + RNN (Recurrent Neural Network, recurrent neural network) structure. The CNN part of the model uses causal convolution, that is, the convolution layer without the right view of the model; the RNN part uses the LSTM structure, which also does not require the right view of the model, so that the entire model can maintain the streaming effect of one frame and one frame. .

Based on the above-mentioned embodiment, FIG. 2 is a schematic flowchart of obtaining desensitized voice data provided by the present invention. As shown in FIG. 2 , step 130 includes:

Step 131, based on the voice mask of each voice frame, desensitize the amplitude spectrum of each voice frame in the voice data to obtain desensitized amplitude spectrum data;

Specifically, based on the voice mask of each voice frame, desensitization processing is performed on the amplitude spectrum of each voice frame in the voice data to obtain desensitized amplitude spectrum data. It should be noted that, based on the voice mask of each voice frame, desensitization processing is performed on the amplitude spectrum of each voice frame in the voice data, wherein the desensitization processing may be to desensitize the original voice frame according to the voice mask of each voice frame. For example, when the output voice mask is 0 or close to 0, it is expressed as a sensitive voice frame, then the voice mask is multiplied by the original amplitude spectrum of the frame to obtain the amplitude spectrum of the desensitized voice frame.

However, considering that there will be some misjudgments in the output of the sensitive voice detection model, the desensitization processing based on the single-frame voice mask will cause misprocessing, which will eliminate the general voice and lead to poor user experience. Therefore, the specific desensitization processing It is also possible to combine the continuous multiple frames corresponding to the voice mask whose detection result is sensitive information into a voice segment, and then multiply the voice mask of the voice segment by the original amplitude spectrum of the voice segment to obtain desensitized amplitude spectrum data. .

Step 132: Perform inverse transformation on the desensitized amplitude spectrum data to obtain desensitized speech data.

Specifically, inverse transform is performed on the desensitized amplitude spectrum data to obtain desensitized speech data. It should be noted that the desensitized amplitude spectrum data is converted into speech data through inverse short-time Fourier transform, that is, desensitized speech data is obtained.

Based on the above embodiment, step 131 includes:

Based on the speech mask of each speech frame, the speech segment of the sensitive word is located from the speech data, and the desensitization method of each speech segment of the sensitive word is determined;

If the desensitization method is information desensitization, the amplitude spectrum of the speech frame with the specified number of frames after the speech segment of the sensitive word, or the amplitude spectrum of each speech frame in the speech segment of the sensitive word and the speech of the specified number of frames after the speech segment of the sensitive word The amplitude spectrum of the frame is desensitized;

If the desensitization method is sensitive word desensitization, the amplitude spectrum of each speech frame in the speech segment of the sensitive word is desensitized.

Considering that the method of desensitization may include desensitization of sensitive words directly, such as desensitization of uncivilized words, or based on the location of sensitive words, desensitization of the content behind the word, such as desensitization of speech The words such as account number and password are located, and the subsequent part of the word is desensitized. Based on the above content, the embodiment of the present invention locates a sensitive word, and determines a desensitization method for the sensitive word.

Specifically, according to the voice mask of each voice frame output by the sensitive information monitoring model, determine whether the voice frame is a sensitive voice frame and obtain the voice segment of the sensitive word based on the condition of locating the voice segment of the sensitive word, and according to the voice segment of the sensitive word The amplitude spectrum finds its corresponding desensitization method, wherein the desensitization methods include information desensitization and sensitive word desensitization. If the desensitization method is information desensitization, the amplitude spectrum of the speech frame with the specified number of frames after the speech segment of the sensitive word, or the amplitude spectrum of each speech frame in the speech segment of the sensitive word and the speech of the specified number of frames after the speech segment of the sensitive word The amplitude spectrum of the frame is desensitized; if the desensitization method is sensitive word desensitization, the amplitude spectrum of each speech frame in the speech segment of the sensitive word is desensitized. It should be noted that the condition for locating the speech segment of the sensitive word may be that the speech segment is determined to be a sensitive speech segment according to the proportion of the sensitive speech frames in the multiple speech frames reaching a preset ratio, or that the continuous sensitive speech frame reaches the predetermined ratio. If the number of frames is set, it is determined that the speech segment is a sensitive speech segment, which is not limited in this embodiment of the present invention; the desensitization method for determining the speech segment can be desensitized according to the preset correspondence between the amplitude spectrum of the speech segment and the desensitization method. If determined, the amplitude spectrum of the speech segment may also be input into the desensitization mode classification model, and the desensitization mode is output from the classification model, which is not limited in this embodiment of the present invention.

Based on the foregoing embodiment, FIG. 3 is a schematic flowchart of a method for determining a sensitive speech segment provided by the present invention. As shown in Figure 3, in step 131, based on the voice mask of each voice frame, the sensitive word voice segment is located from the voice data, including:

Step 310, determine that in the voice data, the voice frame whose voice mask is less than the preset voice mask threshold is used as the sensitive word voice frame;

Considering that the smaller the amplitude value of the amplitude spectrum is, the smaller the sound intensity is. Therefore, in order to eliminate the sensitive voice, the sensitive voice needs to be eliminated from the voice data, that is, the amplitude value in the amplitude spectrum of the sensitive voice needs to be changed to 0 or close to 0. . Therefore, a voice mask threshold is preset to distinguish general voice and sensitive voice. Since sensitive voice is a language that needs to be silenced, the voice mask of sensitive voice must be 0 or close to 0, so you can set the preset voice mask by setting the voice mask. The threshold value is selected, and the voice frame of the sensitive word is selected from the voice frames of the voice data by comparing the preset voice mask threshold value with the voice mask.

Specifically, it is determined that in the speech data, the speech frame smaller than the preset speech mask threshold is used as the speech frame of the sensitive word. It should be noted that the value of the speech mask may be the length of the vector, or may be the average value of each element in the vector, which is not limited in this embodiment of the present invention.

In step 320, a plurality of consecutive sensitive word speech frames whose frame number is greater than the preset frame number threshold are used as a sensitive word speech segment.

Considering that there is little data information in a single frame, the sensitive speech detection model will be falsely triggered, and the sensitive words will have a certain duration. Therefore, the speech segment smaller than a certain preset number of frames can be regarded as false triggering.

Specifically, a plurality of consecutive sensitive word speech frames with a frame number greater than a preset frame number threshold may be used as a sensitive word speech segment. It should be noted that during frame-by-frame processing, if a frame of sensitive word speech frame is detected after the general speech frame, the frame is used as the starting point, and the general speech frame detected after the starting point is used as the end point. If the number of frames in the speech segment interval of the end point) is greater than the preset frame number threshold, the speech segment is a sensitive word speech segment.

Based on the above embodiment, FIG. 4 is a schematic flowchart of a method for determining a voice segment desensitization method for a sensitive word provided by the present invention. As shown in FIG. 4 , a method for determining a voice segment desensitization method for a sensitive word provided by an embodiment of the present invention includes: :

Step 410, from the tail of the sensitive word speech segment forwardly intercepting several speech frames of preset interception frames, as the speech segment to be classified;

Considering that the real-time performance of speech desensitization needs to be ensured, in the embodiment of the present invention, a segment of speech frame is cut forward from the tail of the speech segment of the sensitive word, as the speech segment to be classified. Meanwhile, in order to facilitate the training of the speech classification model in step 420, the frame number of the speech segment of the sample is fixed. Here, the length of the speech segment to be predicted is also fixed to be the same as the frame number of the speech segment of the sample.

It should be noted that the end of the speech segment of the sensitive word is the last speech frame of the speech segment of the sensitive word; the preset number of intercepted speech frames can be set according to the requirements of the speech classification model in step 420 .

Step 420, input the speech segment to be classified into the speech classification model, and obtain the desensitization mode of the sensitive word speech segment output by the speech classification model;

The speech classification model is trained based on the speech segments of sample sensitive words and their desensitization method labels.

Specifically, the to-be-classified speech segment obtained in step 410 is input into a speech classification model trained by the sample-sensitive word speech segment and its desensitization method label, and the model predicts and outputs the desensitization method of the to-be-classified speech segment for The subsequent part desensitizes the voice data according to the desensitization method.

The method provided by the embodiment of the present invention intercepts a preset interception frame and several speech frames forward from the end of the speech segment of the sensitive word to determine the desensitization method, which helps to ensure the real-time processing efficiency of the sensitive information monitoring model, and does not generate time delay.

Based on the above embodiment, the sample sensitive word speech is obtained by adding noise to the original sensitive word speech based on sample noise, and the sensitivity mask is determined based on the amplitude spectrum of the sample noise and the amplitude spectrum of the original sensitive word speech.

Considering that sensitive words will appear in various environments, in order to train the sensitive speech model more accurately, it is necessary to simulate the appearance of sensitive words in a noisy environment. The embodiment of the present invention performs noise processing on the original sensitive word speech.

For the sample sensitive word speech obtained by adding noise, since the sample sensitive word speech is no longer pure sensitive word speech, the corresponding sensitive mask also needs to be determined based on the amplitude spectrum of the sample noise and the amplitude spectrum of the original sensitive word speech.

For example, the sensitivity mask can be determined by the following formula:

Among them, S is the speech amplitude spectrum of sensitive words, and N is the speech amplitude spectrum of background noise. The loss function of the model training is MSE (Mean Square Error, Mean Square Error) loss function, which is used to reduce the error between the predicted speech mask and the real speech mask.

Based on the foregoing embodiment, FIG. 5 is the second schematic flowchart of the method for speech desensitization provided by the present invention. As shown in FIG. 5 , a speech desensitization method provided by an embodiment of the present invention includes:

Step 510, performing sliding window framing on the voice to obtain a set of voice frames of the voice data;

Step 520, carry out STFT (Short-time Fourier transform, short-time Fourier transform) to each speech frame in the speech frame set, obtain the amplitude spectrum of each speech frame;

Step 530: Input the amplitude spectrum of each frame of the voice into the sensitive voice detection model frame by frame, and obtain the voice mask of each frame, the voice mask of each frame of the general voice is close to 1, and the sensitive words or similar sensitive words The speech of the word will have a speech mask with a value close to 0, and determine the speech segment of the sensitive word according to step 310 and step 320;

Step 540, post-processing the speech segment of the sensitive word, which may include directly multiplying the original STFT amplitude spectrum corresponding to the speech mask in the speech mask of the sensitive word to obtain the STFT amplitude spectrum that eliminates the sensitive word and only retains the background sound, and then Use ISTFT (Inverse Short-Time Fourier Transformation, Inverse Short-Time Fourier Transformation) to inversely transform back the speech data, which can naturally filter sensitive words; Sensitive words are pre-processed, such as replacing with silenced audio, etc., and further processing of complex sensitive information that may appear after the sensitive words. For example, to prevent the sensitive words from being too long and affecting the comparison efficiency, you can set short sensitive words. After the end of the word, the 1s-long audio frame after the end frame is replaced, and the 1s-long mute frame prepared in advance is replaced by the original 1s-long audio frame after the end frame. Audio to achieve mute effect, to achieve the purpose of active hiding of sensitive words and sensitive information after sensitive words.

In a voice desensitization method provided by an embodiment of the present invention, a sensitive voice detection model is used to perform sensitive voice detection on the amplitude spectrum of an input voice frame, so as to locate the sensitive information in the voice data, thereby realizing voice desensitization. There is a process of inter-conversion between speech and text, which avoids the problems of easy leakage of sensitive information caused by speech transcription, excessive elimination of speech, and low transcription efficiency and recognition rate, and realizes real-time and accurate speech desensitization.

Based on the above embodiment, FIG. 6 is a training method for a sensitive speech detection model provided by the present invention. As shown in FIG. 6 , an embodiment of the present invention provides a training method for a sensitive speech detection model, including:

Step 610: Perform sliding window and frame processing on the original voice, and perform STFT transformation. For example, for voice data with a sampling rate of 16k, the frame window length is 1024, and the window shift is 512. Perform STFT transformation to obtain the amplitude spectrum, and convert each frame of voice. is a 513-dimensional frequency domain feature vector;

Step 620: Input the amplitude spectrum and label of each frame of the original speech into the initial sensitive speech detection model frame by frame, and output the corresponding speech mask. For general speech data, the output speech mask value is all 1, and the speech mask is multiplied by the original speech amplitude spectrum, and the value of the amplitude spectrum will not be changed, that is, the general speech will not be modified; The speech data is randomly processed with noise to simulate the appearance of sensitive words in complex environments. The model inputs the amplitude spectrum of the sensitive word voice after adding noise, and the output voice mask is multiplied by the amplitude spectrum after adding noise to obtain the amplitude of the original noise. In other words, the speech mask only blocks out the sensitive word components, while retaining the original background sound as much as possible. The formula for obtaining the speech mask as the model output training label is as follows:

Among them, S is the speech amplitude spectrum of sensitive words, and N is the speech amplitude spectrum of background noise. The loss function trained by the model is the MSE loss function, which is used to reduce the error between the predicted speech mask and the real speech mask.

At the same time, in order to improve the detection effect of the sensitive speech detection model, a number of frame delays are applied to the labels trained by the sensitive speech detection model, which is equivalent to giving the classification results of the historical frames at the current frame moment, so that even if the model does not have a right view, For the given historical frame results, it is still equivalent to having a part of the right field of view (that is, the historical frame feature to the current frame feature part), and both sides of the field of view exist for the CNN model. Fig. 7 is the processing flow schematic diagram of the sensitive speech detection model provided by the present invention, as shown in Fig. 7:

Assuming that the delay is 2 frames, that is, the delay of 64ms, when the second speech frame is input, the sensitive speech detection model outputs the 0th speech frame, and then passes through the MSE loss function to train and converge, and complete the training of all frames. Delayed alignment label sequence number.

The speech desensitization device provided by the present invention is described below, and the speech desensitization device described below and the speech desensitization method described above can be referred to each other correspondingly.

FIG. 8 is a schematic structural diagram of a speech desensitization device provided by the present invention. As shown in FIG. 8 , the device includes: a determination module 810 , a prediction module 820 , and an elimination module 830 .

in,

A determination module 810, configured to determine the voice data to be desensitized;

The prediction module 820 is used for inputting the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model to obtain the speech mask of each speech frame output by the sensitive speech detection model; the sensitive speech detection model is based on the sample general speech and wherein The general mask of each sample speech frame, and the sample sensitive word speech and the sensitive mask of each sample speech frame are obtained by training;

The elimination module 830 is configured to eliminate sensitive information in the speech data based on the speech mask of each speech frame.

In the embodiment of the present invention, the determination module 810 is used to determine the speech data to be desensitized; the prediction module 820 is used to input the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model to obtain the sensitive speech detection model. The speech mask of each speech frame output by the model; the sensitive speech detection model is trained based on the sample general speech and the general mask of each sample speech frame, and the sample sensitive word speech and the sensitive mask of each sample speech frame The elimination module 830 is used to eliminate the sensitive information in the speech data based on the speech mask of each speech frame, and realizes the use of the sensitive speech detection model to output the speech mask based on the amplitude spectrum of the input speech frame to locate the sensitive word, and Desensitizing the sensitive word reduces the process of inter-conversion between speech and text, improves the recognition efficiency, overcomes the problems of sensitive information being easily leaked or excessive elimination of speech and low efficiency and recognition rate, and realizes real-time and accurate speech desensitization.

Based on any of the above embodiments, the prediction module 820 includes:

The streaming prediction submodule is used to input the amplitude spectrum of each speech frame in the speech data to the sensitive speech detection model frame by frame, and obtain the speech mask of each speech frame output by the sensitive speech detection model frame by frame;

The input speech frame and the output speech frame at the same time differ by a preset number of frames, the input speech frame is the speech frame corresponding to the amplitude spectrum of the input sensitive speech detection model, and the output speech frame is the speech mask output from the sensitive speech detection model the corresponding speech frame.

Based on any of the above-mentioned embodiments, in the streaming prediction sub-module, the amplitude spectrum of each speech frame in the speech data is input into the sensitive speech detection model frame by frame, and the speech mask of each speech frame output by the sensitive speech detection model frame by frame is obtained, including :

inputting the amplitude spectrum of each speech frame in the speech data frame by frame into the sensitive speech detection model, and the sensitive speech detection model is based on the amplitude spectrum of each speech frame, the amplitude spectrum of the continuous preset number of speech frames after each speech frame, and The state vector of the frame before each speech frame, encode the state vector of each speech frame, and perform sensitive speech detection based on the state vector of each speech frame to obtain the speech mask of each speech frame output by the sensitive speech detection model frame by frame.

Based on any of the above embodiments, the elimination module 830 includes:

The desensitization sub-module is used to desensitize the amplitude spectrum of each voice frame in the voice data based on the voice mask of each voice frame to obtain desensitized amplitude spectrum data;

The inverse transformation sub-module is used to inversely transform the desensitized amplitude spectrum data to obtain desensitized speech data.

Based on any of the above-mentioned embodiments, the desensitization sub-module includes:

The positioning sub-module is used to locate the speech segment of the sensitive word from the speech data based on the speech mask of each speech frame, and determine the desensitization method of the speech segment of each sensitive word;

The desensitization processing sub-module is used to desensitize the amplitude spectrum of the speech frame with the specified number of frames after the speech segment of the sensitive word, or the amplitude spectrum of each speech frame in the speech segment of the sensitive word and the sensitivity word if the desensitization method is information desensitization. The amplitude spectrum of the speech frame with the specified number of frames after the speech segment is desensitized;

If the desensitization method is sensitive word desensitization, the amplitude spectrum of each speech frame in the speech segment of the sensitive word is desensitized.

Based on any of the above embodiments, the positioning sub-module includes:

The sensitive frame determination submodule is used to determine in the voice data, the voice frame whose voice mask is smaller than the preset voice mask threshold as the voice frame of the sensitive word;

The sensitive speech segment determination sub-module is used to use a plurality of consecutive sensitive word speech frames whose frame number is greater than the preset frame number threshold as a sensitive word speech segment.

Based on any of the foregoing embodiments, the positioning submodule further includes:

Determine the speech segment submodule to be classified, which is used for intercepting a preset number of speech frames forward from the tail of the speech segment of the sensitive word, as the speech segment to be classified;

The mode prediction submodule is used to input the speech segment to be classified into the speech classification model, and obtain the desensitization method of the speech segment of the sensitive word output by the speech classification model;

The classification model training module is used to train the speech classification model based on the sample-sensitive word speech segment and its desensitization method label.

Based on any of the above embodiments, the sample sensitive word speech in the prediction module 820 is obtained by adding noise to the original sensitive word speech based on sample noise, and the sensitivity mask is determined based on the amplitude spectrum of the sample noise and the amplitude spectrum of the original sensitive word speech.

FIG. 9 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG. 9 , the electronic device may include: a processor (processor) 910, a communication interface (Communications Interface) 920, a memory (memory) 930, and a communication bus 940, The processor 910 , the communication interface 920 , and the memory 930 communicate with each other through the communication bus 940 . The processor 910 can call the logic instructions in the memory 930 to execute the speech desensitization method, the method includes: determining the speech data to be desensitized; inputting the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model to obtain The speech mask of each speech frame output by the sensitive speech detection model; the sensitive speech detection model is based on the sample general speech and the general mask of each sample speech frame, and the sample sensitive word speech and the sensitive mask of each sample speech frame. It is obtained by code training; based on the speech mask of each speech frame, the sensitive information in the speech data is eliminated.

In addition, the above-mentioned logic instructions in the memory 930 may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as an independent product. Based on such understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

In another aspect, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can Execute the speech desensitization method provided by the above methods, the method includes: determining the speech data to be desensitized; inputting the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model, and obtaining each output of the sensitive speech detection model. A speech mask of a speech frame; the sensitive speech detection model is trained based on the sample general speech and the general mask of each sample speech frame, and the sample sensitive word speech and the sensitivity mask of each sample speech frame; based on each sample speech frame Speech mask for speech frames to remove sensitive information from speech data.

In another aspect, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, the computer program is implemented by a processor to execute the speech desensitization method provided by the above methods, and the method includes : determine the speech data to be desensitized; input the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model to obtain the speech mask of each speech frame output by the sensitive speech detection model; the sensitive speech detection model is based on the sample general The voice and the general mask of each sample voice frame, and the sample sensitive word voice and the sensitivity mask of each sample voice frame are obtained by training; based on the voice mask of each voice frame, the sensitive information in the voice data is eliminated.

The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (11)

1. a speech desensitization method, is characterized in that, comprises: Determine the voice data to be desensitized; The amplitude spectrum of each speech frame in the speech data is input to the sensitive speech detection model, and the speech mask of each speech frame output by the sensitive speech detection model is obtained; the sensitive speech detection model is based on the sample general speech and wherein The general mask of each sample speech frame, and the sample sensitive word speech and the sensitive mask of each sample speech frame are obtained by training; Based on the speech mask for each speech frame, sensitive information in the speech data is eliminated. 2. The speech desensitization method according to claim 1, wherein the amplitude spectrum of each speech frame in the speech data is input to a sensitive speech detection model, and each output of the sensitive speech detection model is obtained. A speech mask for a speech frame, including: The amplitude spectrum of each speech frame in the described speech data is input to the sensitive speech detection model frame by frame, and the speech mask of each speech frame output by the described sensitive speech detection model frame by frame is obtained; Wherein, the input voice frame and the output voice frame at the same time differ by a preset number of frames, the input voice frame is the voice frame corresponding to the amplitude spectrum of the input sensitive voice detection model, and the output voice frame is from the sensitive voice Detect the speech frame corresponding to the speech mask output in the model. 3. speech desensitization method according to claim 2, is characterized in that, described in described speech data, the amplitude spectrum of each speech frame is input to sensitive speech detection model frame by frame, obtain described sensitive speech detection model frame by frame The speech mask of each speech frame output, including: The amplitude spectrum of each voice frame in the voice data is input into the sensitive voice detection model frame by frame, and the sensitive voice detection model is based on the amplitude spectrum of each voice frame, the continuous preset frame number of voice frames after each voice frame. Amplitude spectrum, and the state vector of the frame before each speech frame, encode the state vector of each speech frame, and perform sensitive speech detection based on the state vector of each speech frame, and obtain the frame-by-frame output of the sensitive speech detection model. Voice mask. 4. The voice desensitization method according to claim 1, wherein the voice mask based on each voice frame, to eliminate sensitive information in the voice data, comprises: Based on the voice mask of each voice frame, desensitization processing is performed on the amplitude spectrum of each voice frame in the voice data to obtain desensitized amplitude spectrum data; Perform inverse transformation on the desensitized amplitude spectrum data to obtain desensitized speech data. 5. speech desensitization method according to claim 4, described based on the speech mask of each speech frame, carry out desensitization processing to the amplitude spectrum of each speech frame in the described speech data, obtain after desensitization. Amplitude spectral data, including: Based on the speech mask of each speech frame, locate the speech segment of the sensitive word from the speech data, and determine the desensitization mode of each speech segment of the sensitive word; If the desensitization method is information desensitization, the amplitude spectrum of the speech frame with the specified number of frames after the sensitive word speech segment, or the amplitude spectrum of each speech frame in the sensitive word speech segment and the sensitive word The amplitude spectrum of the speech frame with the specified number of frames after the speech segment is desensitized; If the desensitization method is sensitive word desensitization, the amplitude spectrum of each speech frame in the speech segment of the sensitive word is desensitized. 6. The speech desensitization method according to claim 5, wherein the speech mask based on each speech frame, locates a sensitive word speech segment from the speech data, comprising: Determine that in the voice data, the voice frame whose voice mask is less than the preset voice mask threshold is used as a sensitive word voice frame; A plurality of consecutive sensitive word speech frames whose frame number is greater than the preset frame number threshold are regarded as a sensitive word speech segment. 7. speech desensitization method according to claim 5, the described desensitization mode that determines each sensitive word speech segment, concrete steps comprise: From the end of the speech segment of the sensitive word, intercept the preset intercepted frames and several speech frames forward as the speech segment to be classified; Inputting the speech segment to be classified into a speech classification model to obtain a desensitization mode of the sensitive word speech segment output by the speech classification model; The speech classification model is obtained by training based on the speech segment of the sample-sensitive word and its desensitization method label. 8. The speech desensitization method according to any one of claims 1 to 7, wherein the sample sensitive word speech is obtained by adding noise to the original sensitive word speech based on sample noise, and the sensitive mask is based on the sample noise. The magnitude spectrum and the magnitude spectrum of the original sensitive word speech are determined. 9. A voice desensitization device, characterized in that, comprising: A determination module for determining the voice data to be desensitized; The prediction module is used for inputting the amplitude spectrum of each speech frame in the speech data into the sensitive speech detection model to obtain the speech mask of each speech frame output by the sensitive speech detection model; the sensitive speech detection model is based on The sample general speech and the general mask of each sample speech frame, and the sample sensitive word speech and the sensitivity mask of each sample speech frame are trained; An elimination module, configured to eliminate sensitive information in the speech data based on the speech mask of each speech frame. 10. An electronic device, comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the program as claimed in claim 1 when the processor executes the program Steps of the speech desensitization method described in any one of to 8. 11. A non-transitory computer-readable storage medium on which a computer program is stored, wherein the computer program implements the speech desensitization method according to any one of claims 1 to 8 when the computer program is executed by a processor A step of.

Images