CN110782901B - Method, storage medium and device for identifying voice of network telephone - Google Patents

Abstract

The invention provides a method, a storage medium and a device for recognizing voice of an Internet phone. The method comprises the steps of: converting a filtered voice signal into a standardized Mel-scale spectrogram, a standardized inverse Mel-scale spectrogram and a stack-type spectrogram respectively. waveform frame signal; using the pile-shaped waveform frame signal as the network input, extract the time domain information of the speech signal through the first convolutional neural network structure; respectively use the standardized Mel scale spectrogram and the standardized inverse Mel scale language The spectrogram is used as the network input, and the frequency domain information of the speech signal is extracted through the second convolutional neural network structure; the time domain information and frequency domain information of the speech signal are input into the classification module after training, and the classification result is output. The invention can not only effectively identify the VoIP voice of the fixed source and fixed terminal, but also can quickly and efficiently identify the VoIP voice generated by the unknown source and the unknown terminal.

Description

A method, storage medium and device for recognizing voice of Internet phone

technical field

The present invention relates to the field of network phone identification, and in particular, to a method, storage medium and device for identifying network phone voice.

Background technique

With the development of the Internet and the maturity of audio compression technology, people's communication methods have become diversified. The emergence of VoIP Internet telephony technology enables people to communicate more conveniently and at a lower cost. Therefore, VoIP technology has attracted a large number of users, and VoIP is gradually replacing traditional landline and mobile phones as one of the main communication methods for people. Compared with traditional fixed phones and mobile phones, Internet phone users do not have a fixed phone number and do not need to use a calling card to dial. The user only needs to dial the phone number of the other party through a specific VoIP software to achieve the purpose of communicating with the other party. Because of this, some VoIP software allows users to set their own phone number at will, and the modified phone number does not need to be verified. Therefore, criminals use this loophole to disguise their own phone numbers by setting their own phone numbers to specific phone numbers (such as the phone number of the public security bureau, the phone number of a bank or the phone number of a government department) through some Internet phone number changing software. identity to commit fraud. Therefore, identifying whether a call is an Internet phone can help the called party to identify the identity of the caller, so that the called party can avoid possible fraud to a certain extent.

SUMMARY OF THE INVENTION

In view of the above-mentioned deficiencies of the prior art, the purpose of the present invention is to provide a method, storage medium and device for recognizing VoIP voice, aiming to solve the problem that the prior art cannot efficiently identify VoIP voice of unknown source.

The technical scheme adopted by the present invention for solving the above-mentioned technical problems is as follows:

A method for recognizing voice of Internet phone, comprising the steps of:

Decompressing, resampling and high-pass filtering the received voice signal to obtain a filtered voice signal;

Converting the filtered speech signal into a standardized Mel-scale spectrogram, a standardized inverse Mel-scale spectrogram and a stack-shaped waveform frame signal respectively;

Taking the pile-shaped waveform frame signal as the network input, extracting the time domain information of the speech signal through the first convolutional neural network structure;

Using the standardized Mel-scale spectrogram and the standardized anti-Mel-scale spectrogram as network input respectively, the frequency domain information of the speech signal is extracted by the second convolutional neural network structure;

Input the time domain information and frequency domain information of the speech signal into the classification module after training, and output the classification result.

The method for recognizing the voice of the Internet phone, wherein the steps of decompressing, resampling and high-pass filtering the received voice signal to obtain the filtered voice signal include:

After decompressing the received voice signal, resampling it into an 8kHz, 16-bit waveform signal;

The resampled waveform signal is subjected to high-pass filtering by using a second-order differential filter to obtain the filtered voice signal: x[n]=-0.5*s[n-1]+s[n]-0.5*s[n+ 1], where n represents the sample points in the time domain signal.

The method for recognizing VoIP voice, wherein the step of converting the filtered voice signal into a standardized mel-scale spectrogram, a standardized inverse mel-scale spectrogram and a stack waveform frame signal respectively comprises:

Perform frame segmentation processing on the filtered speech signal to obtain the nth sample of the ith frame: y _i [n]=w[n]*x[(i-1)*S+n], wherein,

其中，Convert the framed filtered speech signal to a normalized mel-scale spectrogram:

in,

为梅尔刻度频率，f为赫兹刻度频率，K_b为边界点，f_L＝0为最低频率，f_H＝F_s/2为最高频率，F_s为采样频率，ΔF为频率分辨率，f_k为第k个离散傅里叶变换数据点的频率，K_m(m∈{1，2，…，M})是第m个梅尔子带滤波器；

is the Mel scale frequency, f is the Hz scale frequency, K _b is the boundary point, f _L = 0 is the lowest frequency, f _H = F _s /2 is the highest frequency, F _s is the sampling frequency, ΔF is the frequency resolution, f _k is the frequency of the kth discrete Fourier transform data point, and _Km (m∈{1, 2,...,M}) is the mth Mel subband filter;

其中，Convert the framed filtered speech signal to a normalized inverse Mel-scale spectrogram:

in,

为反梅尔刻度；

is the inverse Mel scale;

Convert the framed filtered speech signal to a stacked waveform frame signal:

The method for recognizing VoIP voice, wherein the step of extracting the time domain information of the voice signal through the first convolutional neural network structure using the stack waveform frame signal as the network input includes:

A first convolutional neural network structure is constructed in advance, and the first convolutional neural network structure includes 6 groups of convolution modules sequentially connected in series from the input end to the output end, and the first 5 groups of convolution modules starting from the input end all include A convolutional layer, a max pooling layer, a linear rectification function, and two batch normalization layers. The last set of convolutional modules includes a convolutional layer, a batch normalization layer, a linear rectification function, and A global average pooling layer;

The first convolutional neural network structure is trained on network parameters through the main classifier connected to the output end of the first convolutional neural network structure and the auxiliary classifier connected to the fourth group of convolutional modules. The total cost function of the two classifiers: LossA=α*loss ₀ +β*loss ₁ , where loss ₀ and Loss ₁ represent the cross-entropy loss function of the main classifier and the auxiliary classifier, respectively, α and β are the weights, Satisfy α+β=1;

After the training of the first convolutional neural network structure is completed, the main classifier and the auxiliary classifier are removed, the pile-shaped waveform frame signal is input into the first convolutional neural network structure, and the time of extracting the speech signal domain information.

The method for recognizing VoIP voice, wherein the Mel-scale spectrogram and the inverse Mel-scale spectrogram are respectively used as network inputs, and the frequency domain information of the speech signal is extracted through the second convolutional neural network structure The steps include:

A second convolutional neural network structure is constructed in advance, and the second convolutional neural network structure includes 6 groups of convolution modules sequentially connected in series from the input end to the output end, and the first 5 groups of convolution modules starting from the input end all include A convolutional layer, a max pooling layer, a 2D convolution kernel function, and two batch normalization layers. The last set of convolution modules includes a convolutional layer, a batch normalization layer, and a linear rectification layer. function, and a global average pooling layer;

The second convolutional neural network structure is trained on network parameters through the main classifier connected to the output end of the second convolutional neural network structure and the auxiliary classifier connected to the third group of convolutional modules. The total cost function of the two classifiers: LossB=γ*loss ₀ +δ*loss ₁ , where loss ₀ and loss ₁ represent the cross-entropy loss function of the main classifier and the auxiliary classifier, respectively, and γ and δ are the weights, Satisfy γ+δ=1;

After the second convolutional neural network structure training is completed, the main classifier and the auxiliary classifier are removed, and the standardized Mel-scale spectrogram and the standardized inverse Mel-scale spectrogram are respectively used as network inputs, The frequency domain information of the speech signal is extracted through the second convolutional neural network structure.

In the method for recognizing VoIP voice, the main classifier includes a fully connected layer and a softmax function.

In the method for recognizing VoIP voice, the classification module is a fully connected neural network classifier, and the fully connected neural network classifier includes a first fully connected layer, a second fully connected layer and a softmax function.

The method for recognizing the voice of the Internet phone, wherein, the time domain information and frequency domain information of the voice signal are input into the classification module after training, and the step of outputting the classification result includes:

Input the time domain information and frequency domain information of the speech signal into the fully connected neural network classifier, and set the number of nodes of the first fully connected layer and the second fully connected layer to 768 and 2 respectively. ;

If the fully-connected neural network classifier outputs [0, 1], it is determined that the voice signal is an VoIP voice;

If the fully connected neural network classifier outputs [1,0], it is determined that the voice signal is a mobile phone voice.

A storage medium, which includes storing a plurality of instructions, the instructions are suitable for being loaded by a processor and specifically executing the steps of the method for recognizing voice over an Internet phone according to the present invention.

A device for recognizing voice of Internet phone, which includes a processor, suitable for implementing various instructions; and a storage medium, suitable for storing a plurality of instructions, and the instructions are suitable for being loaded by the processor and executing the recognition of the Internet phone of the present invention. The steps of the method of speech. .

Beneficial effects: The present invention provides and proposes a method for recognizing the voice of an Internet phone, using the trained first convolutional neural network structure and the second convolutional neural network structure to extract the multi-domain depth features of the voice information, and using the extracted features Recognize voice over VoIP. Compared with the existing method, the present invention can not only effectively identify the VoIP voice of the fixed source fixed terminal, but also can quickly and efficiently identify the VoIP voice generated by the unknown source and the unknown terminal.

Description of drawings

FIG. 1 is a flowchart of a preferred embodiment of a method for recognizing voice over an Internet phone provided by an embodiment of the present invention.

Figure 2a is a spectrogram of a mobile phone call recording at a sampling rate of 48kHz.

Figure 2b is the spectrogram of the VoIP call recording under the condition of 16kHz sampling rate.

Figure 2c is the spectrogram of the mobile phone call recording under the condition of 8kHz resampling rate.

Figure 2d is the spectrogram of the VoIP call recording under the condition of 8kHz resampling rate.

Figure 3a is a histogram of the zero-crossing rate of the mobile phone call recording before and after high-pass filtering.

Figure 3b is a histogram of the zero-crossing rate of the VoIP call recording before and after high-pass filtering.

FIG. 4 is a flow chart of framing and format conversion expression of a speech signal.

5 is a regularized log spectrogram, a regularized log mel scale spectrogram, and a regularized log inverse Mel scale spectrogram of high-pass filtered mobile phone and VoIP call recordings.

FIG. 6a is a schematic structural diagram of a first convolutional neural network structure.

FIG. 6b is a schematic structural diagram of a second convolutional neural network structure.

FIG. 7 is a structural diagram of a fully connected convolutional neural network classifier designed by the present invention.

FIG. 8 is a structural block diagram of an apparatus for recognizing voice over an Internet phone according to the present invention.

FIG. 9 is a schematic diagram showing the detection accuracy of speech segments of different lengths in scene 0 and scene 5.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer and clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

In the existing literature, a VoIP fingerprint is proposed for VoIP identification. The designed VoIP fingerprint includes two parts. The first part consists of the packet loss rate of the voice frame, the correlation between packets, the statistical characteristics of the noise spectrum, and the The second part is the path traversal signature formed by inputting the features extracted from the first part into the decision tree. Since the method relies on the packet loss feature, as network quality improves, packet loss may become a less effective feature. In addition, this method assumes that each call source has a fixed and unique identification fingerprint, so this method can only be applied to detect VoIP calls made from a certain fixed source to another designated terminal (such as a fixed telephone call to a fixed telephone) , instead of being able to identify various VoIP calls from unknown sources.

The existing literature also proposes a method for detecting VoIP speech by identifying codecs. The method first re-encodes the speech with a known codec to form multi-dimensional features including noise spectrum and time-domain histogram of the speech signal. model, and then compare the obtained feature model with the reference feature model of the candidate codec, so as to identify which codec processing a piece of speech has undergone, and then judge whether the voice to be detected has passed through the VoIP network by the type of the codec. . However, since the audio data is transmitted in different telephone networks and various devices, the encoded code stream of the voice signal will be transcoded as it passes through different types of communication networks. The last codec used in the PSTN public telephone network may obscure most traces of the codec used in the VoIP network. In addition, both VoIP networks and PSTN networks may use the same codec, such as G.711. Therefore, identifying codecs alone is not enough to identify VoIP speech.

Based on the problems existing in the prior art, the present embodiment provides a method for recognizing voice over an Internet phone, as shown in FIG. 1 , which includes the steps:

Decompressing, resampling and high-pass filtering the received voice signal to obtain a filtered voice signal;

Converting the filtered speech signal into a standardized Mel-scale spectrogram, a standardized inverse Mel-scale spectrogram and a stack-shaped waveform frame signal respectively;

Taking the pile-shaped waveform frame signal as the network input, extracting the time domain information of the speech signal through the first convolutional neural network structure;

Using the standardized Mel-scale spectrogram and the standardized anti-Mel-scale spectrogram as network input respectively, the frequency domain information of the speech signal is extracted by the second convolutional neural network structure;

Input the time domain information and frequency domain information of the speech signal into the classification module after training, and output the classification result.

In this embodiment, the trained first convolutional neural network structure and the second convolutional neural network structure are used to extract multi-domain depth features of speech information, and the extracted features are used to recognize the voice of the Internet phone. Compared with the existing method, the present invention can not only effectively identify the VoIP voice of the fixed source fixed terminal, but also can quickly and efficiently identify the VoIP voice generated by the unknown source and the unknown terminal.

In some embodiments, the step of performing decompression, resampling and high-pass filtering on the received voice signal to obtain the filtered voice signal includes: after decompressing the received voice signal, resampling to 8 kHz, 16-bit waveform signal; use a second-order differential filter to perform high-pass filtering on the resampled waveform signal to obtain a filtered voice signal: x[n]=-0.5*s[n-1]+s[n]- 0.5*s[n+1], where n represents the sample point in the time domain signal.

Specifically, when using a mobile phone to record the received voice signal, different mobile phones will use different sampling rates to sample the voice signal by default. For example, the mobile phone will use sampling rates such as 16kHz, 44.1kHz, 48kHz, etc. The captured speech contains many useless high frequency signals. Due to the limited bandwidth of VoIP and PSTN networks, this embodiment chooses to resample the decompressed voice signal into an 8kHz, 16-bit waveform signal, which can not only retain most of the voice content, but also solve the problem of insufficient bandwidth. Figures 2a-2d show the spectrogram of the 48kHz mobile phone call recording signal, the spectrogram of the 16kHz Internet phone call recording signal, and the spectrogram after resampling them with a sampling rate of 8kHz, as can be seen from the figures The high frequency parts of the 48kHz and 16kHz speech signals contain very little information, while the resampled signal still retains most of the information in the original speech.

In this embodiment, high-pass filtering is also performed on the resampled speech signal. Since the human ear is not sensitive to the high-frequency part of the speech signal, it is difficult for the human ear to distinguish the voice of the Internet phone and the voice of the mobile phone call. Therefore, this embodiment uses a high-pass filter to suppress the low-frequency part of the speech signal and amplify the high-frequency information in the speech. Figures 3a-3b show the zero-crossing rates of 12,000 25-millisecond cell phone calls and VoIP recordings before and after passing through the high-pass filter. It can be found from Figures 3a-3b that after the high-pass filtering period, the difference in high-frequency components between the mobile phone call recording and the Internet phone recording will be amplified. For a signal s[n], this embodiment uses a second-order differential filter to perform high-pass filtering, and the high-pass filtered signal x[n] can be obtained by the following formula: x[n]=-0.5*s[n -1]+s[n]-0.5*s[n+1], where n represents a sample point in the time-domain signal.

In some implementation manners, in order to facilitate the use of deep learning to extract features later, in this embodiment, after decompressing, resampling, and high-pass filtering the speech signal, it continues to be framed and converted into Mel. The three forms of spectrum, inverse Mel spectrum, and heap waveform frame, the process is shown in Figure 4.

然后计算分帧后语音的梅尔刻度语谱图、反梅尔刻度语谱图、以及堆式波形帧信号。Specifically, in this embodiment, N is used to represent the frame length of the audio frame, and S is used to represent the frame shift, then the n-th sample of the i-th frame can be obtained by the following formula: y _i [n]=w[n]*x [(i-1)*S+n], where w[n] is the Hamming window function, defined by the following formula

Then, the Mel-scale spectrogram, the inverse Mel-scale spectrogram, and the stack waveform frame signal of the framed speech are calculated.

表示从赫兹刻度到梅尔刻度的转换，使用

表示反变换，则与赫兹刻度频率f相比，所述梅尔刻度频率通过以下公式计算：

而反梅尔刻度语谱可以通过以下公式计算：

其中，f_L,f_H,和F_s分别表示最低频率、最高频率、以及采样频率，在本实施例中，将采样频率设置为8000Hz、最低频率设置为0、最高频率设置为采样频率的一半，即4000Hz。N为离散傅里叶变换的点数，其值与帧长点数相同。对于第i帧语音信号，我们先求它的离散傅里叶变换：

其中，第k个离散傅里叶变换数据点的频率f_k＝kΔF，其中，ΔF为频率分辨率，

通过以下公式计算每一帧的频谱能量：E_i[k]＝|Y_i[k]|²，由于频谱是对称的，因此本实施例只选取一半频谱，即

接下来本实施例使用M个梅尔三角滤波器对能量谱进行滤波，在滤波器组中的第m个三角滤波器定义为：This example uses

To represent the conversion from Hertz scale to Mel scale, use

represents the inverse transformation, then compared with the Hertz scale frequency f, the Mel scale frequency is calculated by the following formula:

The inverse Mel scale spectrum can be calculated by the following formula:

Wherein, f _L , f _H , and F _s represent the lowest frequency, the highest frequency, and the sampling frequency, respectively. In this embodiment, the sampling frequency is set to 8000 Hz, the lowest frequency is set to 0, and the highest frequency is set to half of the sampling frequency , ie 4000Hz. N is the number of discrete Fourier transform points, and its value is the same as the number of points in the frame length. For the i-th frame speech signal, we first find its discrete Fourier transform:

Among them, the frequency of the k-th discrete Fourier transform data point f _k =kΔF, where ΔF is the frequency resolution,

The spectral energy of each frame is calculated by the following formula: E _i [k]=|Y _i [k]| ² , since the spectrum is symmetrical, only half of the spectrum is selected in this embodiment, that is,

Next, this embodiment uses M Mel triangular filters to filter the energy spectrum, and the mth triangular filter in the filter bank is defined as:

其中，K_b为边界点，

f_L＝0为最低频率，f_H＝F_s/2为最高频率，在本实施例中，K_m(m∈{1，2，…，M})是第m个梅尔子带滤波器的中心频率，本实施例通过以下公式计算梅尔刻度语谱图，

最后通过对梅尔刻度语谱图进行标准化处理就得到最终的标准化梅尔刻度语谱图：

其中，

Among them, K _b is the boundary point,

f _L =0 is the lowest frequency, f _H =F _s /2 is the highest frequency, in this embodiment, K _m (m∈{1,2,...,M}) is the mth Mel subband filter The center frequency of , the present embodiment calculates the Mel scale spectrogram by the following formula,

Finally, the final standardized Mel-scale spectrogram is obtained by normalizing the Mel-scale spectrogram:

in,

In some embodiments, different from calculating the Mel-scale spectrogram, this implementation uses an inverse Mel-triangular filter bank to filter the energy spectrum when the spherical inverse Mel-scale spectrogram is used, wherein the mth inverse Mel-scale spectrogram is used to filter the energy spectrum. The definition of the mel filter is:

通过以下公式计算反梅尔刻度语谱图：

The inverse Mel-scale spectrogram is calculated by the following formula:

最后对所述反梅尔刻度语谱图进行标准化处理得到最终的标准化反梅尔刻度语谱图：

Finally, standardize the anti-Mel scale spectrogram to obtain the final standardized anti-Mel scale spectrogram:

其中，

in,

Figure 5 shows the normalized spectrogram of different scales of the high-pass filtered signal. From the figure, we can see that compared with the mobile phone call recording, the energy of the high-frequency part of the Internet phone call recording is weaker than that of the mobile phone call. . Compared with the spectrogram on the Hertz scale, the spectrogram on the Mel-scale and the inverse Mel-scale can amplify the difference between the phone call recording and the VoIP call recording, and reduce the difference between different VoIP recordings.

维的堆式波形帧矩阵In some embodiments, compared to mel-scale spectrograms and inverse mel-scale spectrograms, the stacked waveform frames preserve the phase information of the speech signal, so using the stacked waveform frames as input to the deep network can capture Information that cannot be captured in the spectrogram can be obtained by arranging L-frame waveform frames column by column in this embodiment.

dimensional heaped waveform frame matrix

In some embodiments, two different convolutional neural network structures are designed, and the frequency domain information and time domain information of the speech signal are extracted respectively through the first convolutional neural network structure and the second convolutional neural network structure, and finally the extracted The depth feature is a 2304-dimensional vector.

In some embodiments, the step of extracting the time domain information of the speech signal through the first convolutional neural network structure using the stack waveform frame signal as the network input includes: as shown in FIG. 6a, constructing a first volume in advance Convolutional neural network structure, the first convolutional neural network structure includes 6 groups of convolution modules connected in series from the input end to the output end, and the first 5 groups of convolution modules starting from the input end all include a convolution layer, A max pooling layer, a linear rectification function (ReLU), and two batch normalization layers (BN), the last set of convolution modules includes a convolutional layer, a batch normalization layer, a linear rectification function, and a global average pooling layer; in this embodiment, the first three groups of convolution modules are designed as one-dimensional convolutions, the purpose is to extract the intra-frame features of each frame of the stack waveform frame, and the fourth convolution module It is designed as a two-dimensional convolution to extract intra-frame features and fuse the information extracted between frames. The last two convolution modules are also designed as one-dimensional convolutions to fuse inter-frame features.

In this embodiment, a main classifier consisting of a fully connected layer and a softmax function is connected to the output end of the first convolutional neural network, and an auxiliary classifier is connected to the output of the fourth group of convolution modules, Perform network parameter training on the first convolutional neural network structure through the main classifier and the auxiliary classifier, and the total cost function of the two classifiers: LossA=α*loss ₀ +β*loss ₁ , where, loss ₀ and loss ₁ represent the cross-entropy loss function of the main classifier and the auxiliary classifier, respectively, α and β are the weights, satisfying α+β=1; when the first convolutional neural network structure is trained, remove The main classifier and the auxiliary classifier input the stack waveform frame signal into the first convolutional neural network structure (SWF-net) to extract the time domain information of the speech signal.

In some embodiments, the Mel-scale spectrogram and the inverse Mel-scale spectrogram are respectively used as network inputs, and the step of extracting the frequency domain information of the speech signal through the second convolutional neural network structure includes: As shown in Fig. 6b, a second convolutional neural network structure is pre-built. The second convolutional neural network structure includes 6 groups of convolution modules sequentially connected in series from the input end to the output end. The first 5 groups starting from the input end Each convolution module includes a convolution layer, a maximum pooling layer, a two-dimensional convolution kernel function, and two batch normalization layers. The last group of convolution modules includes a convolution layer and a batch normalization layer. layer, a linear rectification function, and a global average pooling layer. In this embodiment, there are two differences between the second convolutional neural network structure and the first convolutional neural network structure. The first difference is that the dimension of the kernel function in the convolution module is different, because for language In terms of spectrogram, two adjacent data points have a certain correlation, so this embodiment directly uses a two-dimensional convolution kernel in the first five convolution modules in the second convolutional neural network structure. Extract features, and design a specific pooling size and convolution step size to make the second convolutional neural network structure and the first convolutional neural network structure have the same output dimension; the second difference is that the auxiliary The position of the classifier, the auxiliary classifier in this embodiment is connected at the output of the third convolution module. Perform network parameter training on the second convolutional neural network structure through the main classifier and the auxiliary classifier, and the total cost function of the two classifiers: LossB=γ*loss ₀ +δ*loss ₁ , where, loss ₀ and loss ₁ represent the cross-entropy loss function of the main classifier and the auxiliary classifier, respectively, γ and δ are weights, satisfying γ+δ=1; when the second convolutional neural network structure is trained, remove For the main classifier and the auxiliary classifier, the corresponding convolutional neural network model names are MS-net and IMS-net; the standardized Mel-scale spectrogram and the standardized inverse Mel-scale spectrogram are respectively used as the network Input, extract the frequency domain information of the speech signal through the second convolutional neural network structure.

In some implementations, after the depth feature is extracted, this embodiment uses a classification module to fuse the extracted features and make a final decision. In this stage, many classifiers can be used as the classification module, and this embodiment uses the fully connected neural network classifier as the classification module. As shown in Figure 7, the fully connected neural network classifier consists of two fully connected layers and a softmax function. The number of nodes in the first fully connected layer and the second fully connected layer is 768 and 2. After inputting the extracted 2304-dimensional feature vector into the trained fully connected neural network classifier, the fully connected neural network classifier will output [0,1] or [1,0]; in this embodiment, output [0, 1] means that the classifier judges the voice signal to be tested as a VoIP call recording, and output [1,0] means that the classifier judges the voice signal to be tested as a mobile phone call recording.

In some implementations, a storage medium is also provided, wherein a plurality of instructions are stored, and the instructions are adapted to be loaded by a processor and specifically execute the steps of the method for recognizing voice over an Internet phone of the present invention.

In some embodiments, an apparatus for recognizing network voice signals is also provided, wherein, as shown in FIG. 8 , it includes a processor 10 adapted to implement various instructions; and a storage medium 20 adapted to store a plurality of instructions, the The instructions are adapted to be loaded by the processor 10 and to execute the steps of the method of recognizing voice over VoIP.

Test the recognition performance of the present invention to the network voice signal by existing database VPCID below:

In this embodiment, the frame length N is set to 240, the frame shift S is set to 120, and the number M of triangular filters is set to 48. Frame number L is set to 132. A 48x132 mel-scale spectrogram, a 48x132 inverse mel-scale spectrogram, and a 240x132 stack waveform frame are obtained. In the evaluation index, the present embodiment selects the true rate, the true negative rate, and the accuracy rate, wherein the voice of the Internet phone is a positive sample, and the voice of a mobile phone call is a negative sample.

1. Verify the effect of the auxiliary classifier:

Table 1 shows the detection performance for 2-second speech segments when using classifiers with different weights in SFW-net, and Table 2 shows the detection performance when using classifiers with different weights in MS-net and IMS-net. Detection performance for 2-second long speech clips.

Table 1 Detection performance of classifiers using different weights in SFW-net

Table 2 Detection performance of classifiers using different weights in MS-net and IMS-net

As can be seen from Table 1-Table 2, the detection performance of using the auxiliary classifier is better than that without using the auxiliary classifier, and the weight of the auxiliary classifier may be optimized.

2. The overall effect of using different classification modules:

The classification module of the present invention can be composed of different classifiers or even some decision-making methods, and good performance can be achieved by using different classifiers or decision-making methods. Table 3 shows the use of the Universal Background Model-Gaussian Mixture Model (UBM-GMM), the ensemble classifier (FLD Ensemble) classifier as the classification module, the use of the Majority Voting mechanism (Majority Voting) as the classification module, and the use of fully connected neural network classification. The detection performance of the NN-based classifier as a classification module for 2-second long speech segments.

Table 3 Detection performance using different classifiers

It can be seen from the table that different classification methods can achieve very good detection performance. The performance of our fully connected neural network classifier is better than that of existing classifiers and decision-making methods.

3. Detect the effect of voice signals from different sources:

Since the present invention starts from the actual call recording, the design process of the algorithm does not depend on data from a certain source. Therefore, the present invention can more effectively detect call voices from different sources. Therefore, the present invention sets different scenarios, and Table 4 shows the degree of mismatch between the sources in the training data and the test data in each scenario. The tick-marked factor indicates that the training data and test data are from different sources in the factor. Table 5 shows the detection performance of the present invention for 2-second speech segments in different scenarios. It can be seen from Table 5 that the method proposed by the present invention has good detection performance for speeches from many different sources.

Table 4 Mismatch factors in each detection scenario

Table 5 Detection performance of the present invention under various mismatch factor scenarios

4. Detect the effect of speech clips of different lengths:

In the process of extracting features, the algorithm designed in the present invention extracts the intra-frame features of the short-term audio frame and performs inter-frame feature fusion. Therefore the present invention can be used to detect speech segments of different lengths. FIG. 9 shows the detection accuracy rates of the present invention for 1-second, 2-second, 4-second, 6-second, and 8-second-long speech segments in Scenario 0 and Scenario 5. It can be seen from the figure that the detection performance of the present invention for speech segments longer than 6 seconds basically reaches saturation. And it can also have a good detection accuracy for speech segments shorter than 6 seconds.

To sum up, based on the statistical characteristics of the voice signal, the present invention finds that the energy of the voice signal of the Internet phone distributed in the high frequency part is weaker than the energy of the voice signal of the mobile phone distributed in the high frequency part, so the problem of detecting the voice signal of the Internet phone is weak. In the above, the present invention first performs high-pass filtering on the speech signal; the present invention analyzes the spectrograms of different scales, and finds that compared with the linear scale spectrogram, the use of the nonlinear scale spectrogram can amplify the difference between different types of call recording signals. Therefore, in the algorithm design, the present invention uses the Mel-scale spectrogram and the inverse Mel-scale spectrogram as the network input, and compared with the existing technology, The present invention abandons the method of manually designing features, and uses deep learning to extract features from the network input by itself. The first convolutional neural network structure designed by the present invention adopts the method of first extracting intra-frame features and then fusing inter-frame features to extract depth Features; the designed second convolutional neural network structure directly extracts the adjacent data points of the spectrogram to extract the features of the frequency domain, and the information extracted by the three trained subnetworks can be effectively fused together. Combining the technologies together, the present invention can effectively detect VoIP call recordings from the same source, and can more robustly detect VoIP call recordings from different sources.

It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above descriptions, and all these improvements and transformations should belong to the protection scope of the appended claims of the present invention.

Claims (7)

1. A method for recognizing voice of a network telephone, comprising the steps of:

decompressing, resampling and high-pass filtering the received voice signal to obtain a filtered voice signal;

respectively converting the filtered voice signals into a standardized Mel scale spectrogram, a standardized inverse Mel scale spectrogram and a stacked waveform frame signal;

taking the pile-up waveform frame signal as network input, and extracting time domain information of a voice signal through a first convolutional neural network structure;

respectively taking the standardized Mel scale spectrogram and the standardized inverse Mel scale spectrogram as network inputs, and extracting frequency domain information of the voice signal through a second convolutional neural network structure;

inputting the time domain information and the frequency domain information of the voice signal into a trained classification module, and outputting a classification result;

the step of converting the filtered speech signal into a normalized mel scale spectrogram, a normalized anti-mel scale spectrogram and a stacked waveform frame signal respectively comprises:

performing framing processing on the filtered voice signal to obtain an nth sample of an ith frame: y is_i[n]＝w[n]*x[(i-1)*S+n]Wherein

converting the filtered speech signal into a standardized Mel-time scaleDegree language spectrogram:

wherein,

is the Mel scale frequency, f is the Hertz scale frequency, K_bAs a boundary point, f_L0 is the lowest frequency, f_H＝F_s2 is the highest frequency, F_sFor the sampling frequency,. DELTA.F for the frequency resolution, F_kFor the frequency of the kth discrete Fourier transform data point, K_m(M ∈ {1, 2, …, M }) is the mth Mel-band filter;

converting the filtered voice signal subjected to framing into a standardized anti-Mel scale spectrogram:

wherein,

is reverse Mel scale;

converting the filtered voice signal subjected to framing into a pile-up waveform frame signal:

the step of extracting the time domain information of the speech signal by using the heap waveform frame signal as a network input through a first convolutional neural network structure comprises:

the method comprises the steps that a first convolution neural network structure is constructed in advance, the first convolution neural network structure comprises 6 groups of convolution modules which are sequentially connected in series from an input end to an output end, the first 5 groups of convolution modules from the input end comprise a convolution layer, a maximum pooling layer, a linear rectification function and two batch normalization layers, and the last group of convolution modules comprise a convolution layer, a batch normalization layer, a linear rectification function and a global average pooling layer;

performing network parameter training on the first convolutional neural network structure through a main classifier connected to the output end of the first convolutional neural network structure and an auxiliary classifier connected to a fourth group of convolutional modules, wherein the total cost functions of the two classifiers are as follows: LossA ═ α loss₀+βloss₁Wherein, loss₀And loss₁Representing primary and secondary classifiers respectivelyThe cross entropy loss function of the classifier, wherein alpha and beta are weights, and alpha + beta is 1;

after the training of the first convolutional neural network structure is finished, removing the main classifier and the auxiliary classifier, inputting the stacked waveform frame signal into the first convolutional neural network structure, and extracting time domain information of a voice signal;

the step of extracting the frequency domain information of the voice signal through a second convolutional neural network structure by respectively taking the normalized Mel scale spectrogram and the normalized inverse Mel scale spectrogram as network inputs comprises:

a second convolutional neural network structure is constructed in advance, the second convolutional neural network structure comprises 6 groups of convolutional modules which are sequentially connected in series from an input end to an output end, the first 5 groups of convolutional modules from the input end comprise a convolutional layer, a maximum pooling layer, a two-dimensional convolutional kernel function and two batch normalization layers, and the last group of convolutional modules comprise a convolutional layer, a batch normalization layer, a linear rectification function and a global average pooling layer;

performing network parameter training on the second convolutional neural network structure through a main classifier connected to the output end of the second convolutional neural network structure and an auxiliary classifier connected to a third group of convolutional modules, wherein the total cost functions of the two classifiers are as follows: LossB ═ γ loss₀+δ*loss₁Wherein, loss₀And loss₁Respectively representing cross entropy loss functions of a main classifier and an auxiliary classifier, wherein gamma and delta are weights and satisfy that gamma + delta is 1;

and after the second convolutional neural network structure is trained, removing the main classifier and the auxiliary classifier, respectively taking the standardized Mel scale spectrogram and the standardized reverse Mel scale spectrogram as network inputs, and extracting frequency domain information of the voice signal through the second convolutional neural network structure.

2. The method of claim 1, wherein the step of decompressing, resampling and high-pass filtering the received voice signal to obtain a filtered voice signal comprises:

decompressing the received voice signal, and resampling to 8kHz and 16 bit waveform signals;

and carrying out high-pass filtering on the resampled waveform signal by adopting a second-order differential filter to obtain a filtered voice signal: x [ n ] — 0.5 × s [ n-1] + s [ n ] -0.5 × s [ n +1], where n represents a sample point in the time domain signal.

3. The method of claim 1, wherein the primary classifier comprises a full connectivity layer and a softmax function.

4. The method for recognizing voice over internet phone of claim 1, wherein the classification module is a fully-connected neural network classifier comprising a first fully-connected layer, a second fully-connected layer and a softmax function.

5. The method of claim 4, wherein the time domain information and the frequency domain information of the voice signal are input into a trained classification module, and the step of outputting the classification result comprises:

inputting the time domain information and the frequency domain information of the voice signal into the fully-connected neural network classifier, and respectively setting the node numbers of the first layer of fully-connected layer and the second layer of fully-connected layer to 768 and 2;

if the full-connection neural network classifier outputs [0,1], judging that the voice signal is the voice of the network telephone;

and if the full-connection neural network classifier outputs [1,0], judging that the voice signal is the voice of the mobile phone.

6. A storage medium comprising a plurality of instructions stored thereon, the instructions adapted to be loaded by a processor and to carry out the steps of the method of recognizing voice over internet phone of any of claims 1-5.

7. An apparatus for recognizing voice of a network telephone, comprising a processor adapted to implement instructions; and a storage medium adapted to store a plurality of instructions adapted to be loaded by the processor and to perform the steps of the method of recognizing voice over internet phone of any of claims 1-5.

Images