CN114400019A - Model generation method, abnormality detection device, and electronic apparatus - Google Patents

Abstract

The embodiments of the present application disclose a model generation method, an abnormality detection method, an apparatus, and an electronic device. The method includes: acquiring a training data set, the training data set including respective first audio features of multiple audio information of the target device, the multiple audio information including normal audio information and abnormal audio information; Set to train the generator network to be trained to use the converged generator network to be trained as the initial anomaly detection model; adjust the initial anomaly detection model through the discriminator network to use the adjusted generator network as the target anomaly detection model Model. Through the above method, it is possible to perform anomaly detection on the device to be detected by inputting the first audio feature of the device to be detected into the target anomaly detection model, which saves manpower and improves the efficiency of anomaly detection.

Description

Model generation method, abnormality detection method, device, and electronic device

technical field

The present application relates to the field of computer technology, and more particularly, to a model generation method, anomaly detection method, apparatus, and electronic device.

Background technique

With the continuous growth of power demand, the economical and reliability requirements of the power system for the power operating equipment continue to increase. Due to the load effect of long-term operation and the influence of the natural environment (temperature, air pressure, humidity and pollution, etc.), the aging and wear of power operating equipment will cause the performance and reliability of power operating equipment to gradually decrease, and there are potential safety hazards. Therefore, it is very necessary to monitor and detect the operating state of the power operating equipment.

However, manual inspection is required in the related electric power operation equipment detection methods, which is inefficient.

SUMMARY OF THE INVENTION

In view of the above problems, the present application proposes a model generation method, anomaly detection method, apparatus, electronic device, and storage medium, so as to improve the above problems.

In a first aspect, the present application provides a model generation method, which is applied to an electronic device. The method includes: acquiring a training data set, where the training data set includes respective first audio features of multiple audio information of the target device, and the The plurality of audio information includes normal audio information and abnormal audio information; the generator network to be trained is trained by the training data set, so that the convergent generator network to be trained is used as the initial abnormal detection model; The initial anomaly detection model is adjusted to use the adjusted generator network as the target anomaly detection model.

In a second aspect, the present application provides an anomaly detection method, which is applied to an electronic device. The method includes: acquiring audio to be detected; performing frame segmentation, windowing, and fast Fourier transform on the audio to be detected to obtain The first audio feature corresponding to the audio to be detected, the first audio feature is the spectrogram corresponding to the audio to be detected; input the first audio feature into the target anomaly detection model obtained by the above method, and obtain the Describe the detection results output by the target anomaly detection model.

In a third aspect, the present application provides a model generation apparatus, which runs on an electronic device, the apparatus includes: a data set acquisition unit, configured to acquire a training data set, wherein the training data set includes a plurality of audio information of the target device, respectively. The first audio feature, the plurality of audio information includes normal audio information and abnormal audio information; the initial abnormal detection model acquisition unit is used to train the generator network to be trained by the training data set, so that the convergent to be trained generator network is trained. The generator network is used as the initial abnormality detection model; the target abnormality detection model acquisition unit is used to adjust the initial abnormality detection model through the discriminator network, so as to use the adjusted generator network as the target abnormality detection model.

In a fourth aspect, the present application provides an abnormality detection device that runs on an electronic device, the device comprising: a to-be-detected audio acquisition unit for acquiring to-be-detected audio; a first audio feature acquisition unit for The detected audio is subjected to framing, windowing, and fast Fourier transform to obtain the first audio feature corresponding to the audio to be detected, and the first audio feature is the spectrogram corresponding to the audio to be detected; the detection result is obtained The unit is configured to input the first audio feature into the target abnormality detection model obtained by the above method, and obtain the detection result output by the target abnormality detection model.

In a fifth aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, The one or more programs are configured to perform the methods described above.

In a sixth aspect, the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, wherein the above-mentioned method is executed when the program code is executed.

In a model generation method, anomaly detection method, device, electronic device and storage medium provided by the present application, after acquiring a training data set including a first audio feature of normal audio information and abnormal audio information, the training data set is used for training. The generator network is trained to use the converged generator network to be trained as the initial anomaly detection model, and then the initial anomaly detection model is adjusted through the discriminator network to use the adjusted generator network as the target anomaly detection model. Through the above method, after the target anomaly detection model is obtained by training the first audio features of the normal audio information and the abnormal audio information, in the process of anomaly detection of the device to be detected, the first audio feature of the device to be detected can be input into the target The anomaly detection model performs anomaly detection on the equipment to be detected, which saves manpower and improves the efficiency of anomaly detection. Moreover, by adjusting the initial anomaly detection model through the discriminator network, the target anomaly detection model can also have better performance in the absence of abnormal audio training data, that is, it can distinguish between normal audio and abnormal audio. higher accuracy.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 shows a flowchart of a model generation method proposed by an embodiment of the present application;

FIG. 2 shows a flowchart of an embodiment of S110 in FIG. 1 of the present application;

FIG. 3 shows a schematic diagram of the flow of a method for acquiring an audio data set proposed by the present application;

FIG. 4 shows a schematic diagram of a flow of a method for acquiring a first audio feature proposed by the present application;

FIG. 5 shows a schematic diagram of a generator network to be trained proposed by the present application;

FIG. 6 shows a flowchart of a model generation method proposed by another embodiment of the present application;

FIG. 7 shows a flowchart of a model generation method proposed by another embodiment of the present application;

8 shows a schematic diagram of a to-be-trained anomaly detection model proposed by the present application;

FIG. 9 shows a flowchart of an abnormality detection method proposed by an embodiment of the present application;

FIG. 10 shows a structural block diagram of a model generation apparatus proposed by an embodiment of the present application;

FIG. 11 shows a structural block diagram of an abnormality detection apparatus proposed by an embodiment of the present application;

FIG. 12 shows a structural block diagram of an electronic device proposed by the present application;

FIG. 13 is a storage unit for storing or carrying a program code for implementing a parameter acquisition method according to an embodiment of the present application according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

With the continuous growth of power demand, the economical and reliability requirements of the power system for the power operating equipment continue to increase. Due to the load effect of long-term operation and the influence of the natural environment (temperature, air pressure, humidity and pollution, etc.), the aging and wear of power operating equipment will cause the performance and reliability of power operating equipment to gradually decrease, and there are potential safety hazards. Therefore, it is very necessary to monitor and detect the operating state of the power operating equipment.

The inventor found in the research on abnormality detection of related electric power operation equipment that the traditional abnormal detection method of electric power operation equipment requires manual investigation and is inefficient. The anomaly detection method of power operating equipment based on deep neural network requires a large amount of abnormal data to train network parameters. However, in the actual production environment, the amount of abnormal data is small, which makes the performance of deep neural network poor.

Therefore, the inventor proposes a model generation method, anomaly detection method, device and electronic device in the present application. After acquiring the training data set including the first audio features of normal audio information and abnormal audio information, the Set the generator network to be trained to train the convergent generator network to be trained as the initial anomaly detection model, and then adjust the initial anomaly detection model through the discriminator network to use the adjusted generator network as the target anomaly detection model . Through the above method, after the target anomaly detection model is obtained by training the first audio features of the normal audio information and the abnormal audio information, in the process of anomaly detection of the device to be detected, the first audio feature of the device to be detected can be input into the target The anomaly detection model performs anomaly detection on the equipment to be detected, which saves manpower and improves the efficiency of anomaly detection. Moreover, by adjusting the initial anomaly detection model through the discriminator network, the target anomaly detection model can also have better performance in the absence of abnormal audio training data, that is, it can distinguish between normal audio and abnormal audio. higher accuracy.

Referring to FIG. 1, a model generation method provided by the present application is applied to an electronic device, and the method includes:

S110: Acquire a training data set, where the training data set includes respective first audio features of multiple pieces of audio information of the target device, where the multiple pieces of audio information include normal audio information and abnormal audio information.

In this embodiment of the present application, the target device may be a device selected for abnormality detection. Wherein, the target device may be a power running device, such as a generator, an electric motor, a transformer, and the like.

Among them, as shown in Figure 2, the training data set is obtained, including:

S111: Acquire multiple pieces of audio information of the target device.

Among them, as a way, the sound generated by the target device during operation due to its own internal structure or hardware conditions can be sampled multiple times through an audio collection device (such as a tape recorder, etc.), and the multiple sampled audios can be used as Multiple audio information of the target device, wherein the multiple audio information can include normal audio information (audio when the target device is running normally) and abnormal audio information (audio when the target device is abnormal, for example: the transformer is over-circuited due to an external short circuit. current, sound when overloaded due to a load exceeding the rated capacity for a long time). Exemplarily, the target device is a transformer, and the audio collection device samples multiple audio frequencies when the transformer operates normally and when abnormality occurs, so as to obtain multiple audio information with a sampling rate of 16 kHz and a sampling precision of 16 bits.

It should be noted that the sampling rate and sampling accuracy are generally related to the hardware conditions of the audio capture device. The higher the sampling rate, the more sampling points the audio capture device collects per second. For example, when the sampling rate is 16khz , it can be shown that the audio collection device collects 16,000 sampling points per second; the higher the sampling precision, the larger the representation range of each sampling point data, for example, when the sampling precision is 16bit, it can be shown that each sample The range represented by point data is -32768 (2 ^(16-1) ) to +32767.

As shown in FIG. 3 , after the audio collection device collects multiple audio information of the target device, the multiple audio information of the target device may be stored and marked, so as to use the marked multiple audio information as an audio data set. Exemplarily, the tag of the collected normal audio information may be set to 1, and the tag of the collected abnormal audio information may be set to 0.

It should be noted that there may be various ways to label the multiple audio information of the target device, which may be manual labeling, automatic labeling using a classification model, or manual calibration after automatic labeling using a classification model.

S112: Perform framing, windowing, and fast Fourier transform on the audio information to obtain a spectrogram corresponding to the audio information.

In one way, an audio data set including multiple audio information of the target device can be preprocessed and the preprocessed result is stored as a training data set, where the training data set includes language corresponding to each of the multiple audio information. Spectrogram to convert audio information into image information and input it into deep neural network for training. Among them, the preprocessing can include framing, windowing, and fast Fourier transform. As shown in Figure 4, the marked audio can be divided into frames and windowed, and then each window can be subjected to fast Fourier transform to obtain Obtain the spectrogram corresponding to the labeled audio. Exemplarily, when the duration of each audio collected by the audio collection device is 1s, the sampling rate is 16kHZ, the frame length is 25ms, the frame shift is 10ms, and the window is a Hanning window, each audio information corresponds to 16,000 pieces of audio information. Sampling point data, each frame in each audio information corresponds to 400 sampling point data. When the window length is the same as the frame length, after multiplying the Hanning window function on each frame of data, 400 windowed data can be obtained. Sampling point data, 512 spectral lines corresponding to one frame of data can be obtained by performing 512-point fast Fourier transform on the windowed sampling point data. After this calculation is completed, the Hanning window can be moved backward by 10ms (160 sampling point data), perform the next calculation until all frames are calculated, and the corresponding spectrogram of each audio information can be obtained.

As another method, the audio data set including multiple audio information of the target device can be processed in real time by framing, windowing, and fast Fourier transform, so as to obtain the corresponding spectrograms of the multiple audio information, so as to obtain the corresponding spectrograms. Convert audio information into image information and input it into deep neural network for training in real time.

It should be noted that the frame length, frame shift, window function and the number of fast Fourier transform points may be determined according to actual requirements. Exemplarily, considering the real-time nature of the anomaly detection model and the resolution of the audio information, the number of fast Fourier transform points can be set to 512, so that both rich audio information can be extracted and the model can maintain a faster speed. Calculate speed.

Furthermore, it should be noted that since in actual production and life, there are few abnormal situations in the power operation equipment, so that the number of spectrograms corresponding to normal audio in the training data set may be more than the number of spectrograms corresponding to abnormal audio. Therefore, the training data set in this embodiment of the present application may be unbalanced.

S113: Use the spectrogram as the first audio feature of the audio information.

Wherein, as an approach, multiple spectrograms of the target device may be used as the respective first audio features of multiple audio information of the target device.

S120: Train the generator network to be trained by using the training data set, so as to use the converged generator network to be trained as an initial anomaly detection model.

The generator network (Generator, G) to be trained may be used to generate a second audio feature conforming to the distribution of the first audio feature based on the first audio feature, and perform feature extraction on the second audio feature. In this embodiment of the present application, the generator network to be trained may include a first audio feature reconstruction network and a feature extraction network, wherein the first audio feature reconstruction network may include a first encoder, LSTM (Long Short-Term Memory, long short term memory network) and a decoder, the feature extraction network may include a second encoder. As a way, the first audio feature can be input into the first encoder, and the first audio feature can be encoded (Coding) through nonlinear transformation to obtain the low-dimensional feature of the first audio feature; then the encoded feature ( The low-dimensional features of the first audio feature) are input into LSTM, because LSTM has a good ability to extract temporal information and the audio features in this application are related to time, so LSTM can be used to further extract the encoded features to obtain More effective latent representation (low-dimensional feature of the first audio feature), so that the generator network to be trained can learn the characteristics of normal audio and abnormal audio in the time dimension, thereby improving the generator network to be trained to normal audio and abnormal audio. The ability to distinguish audio; then the potential representation can be input into the decoder, the potential representation can be decoded (Decoding) through inverse mapping, and the first audio feature can be reconstructed. At this time, the reconstructed first audio feature can be called the first audio feature. Two audio features, wherein the size of the first audio feature is the same as that of the second audio feature; after obtaining the second audio feature, the second audio feature can be input into the second encoder, and further feature extraction and extraction are performed on the second audio feature. Dimensionality reduction.

Optionally, the generator network to be trained may further include a fully connected layer, and after the fully connected layer, a softmax activation function may be used to output anomaly detection results of audio information corresponding to the first audio feature. By training the generator network to be trained with the training data set including the plurality of first audio features, a converged generator network to be trained can be obtained, and the converged generator network to be trained can be used as an initial anomaly detection model.

Optionally, in order to reduce the loss of feature information, in the generator network to be trained, a residual connection may be used between the intermediate features of the same size of the first encoder and the decoder. Exemplarily, as shown in Figure 5, in the first encoder part, an intermediate feature can be obtained after the first audio feature undergoes a 3×3 convolution operation. Similarly, in the decoder part, the output of the LSTM undergoes a 3×3 convolution operation. After the deconvolution operation of ×3, an output feature can be obtained, and an intermediate feature of the decoder can be obtained by adding the output feature and the above-mentioned first encoded intermediate feature, and the decoder intermediate feature and the above-mentioned first encoder intermediate feature Same size.

It should be noted that the structures of the first encoder and the second encoder may be the same. Exemplarily, as shown in FIG. 5 , the first encoder and the second encoder of the generator network to be trained may respectively include three 3×3 two-dimensional convolutional layers, and the decoder may include three 3×3 convolutional layers. 2D deconvolution layer, in addition, the generator network to be trained can also include an LSTM layer and a fully connected layer.

Furthermore, it should be noted that the network layer depth of the encoder, decoder, LSTM layer and fully connected layer in the generator network to be trained and the parameters corresponding to each layer of the network (for example, the size of the convolution layer, the deconvolution layer layer size, etc.) can be flexibly set according to different target devices, the size of the first audio feature, and the like. Optionally, in order to improve the performance of the model, an attention mechanism can be introduced into the generator network to be trained.

S130: Adjust the initial anomaly detection model through a discriminator network, so as to use the adjusted generator network as a target anomaly detection model.

The discriminator network (Discriminator, D) can be used to discriminate whether the distributions of the first audio feature and the second audio feature in the initial anomaly detection model are consistent. In this embodiment of the present application, the discriminator network may include a third encoder. Optionally, the network structure of the third encoder may be the same as that of the first encoder or the second encoder.

As an approach, since the training purpose of the generator network in the initial anomaly detection model may be to generate a second audio feature conforming to the first audio feature distribution (true distribution), the training purpose of the discriminator network may be to correctly discriminate the first audio Whether the distribution of the feature and the second audio feature is consistent, so adjusting the network parameters (for example, weights, etc.) of the initial anomaly detection model through the discriminator network can make the generator network in the initial anomaly detection model compete with the discriminator network. In the process of learning the data distribution of the first audio features corresponding to the normal audio information and the abnormal audio information, the adjusted generator network can be used as a target abnormality detection model to perform abnormality detection on the input audio features.

As another method, it can be determined whether to adjust the initial anomaly detection model through the discriminator network according to the proportion of abnormal audio information in the training data set. The anomaly detection model is adjusted to use the adjusted generator network as the target anomaly detection model. Exemplarily, assuming that the threshold is A, and the proportion of abnormal audio information in the training data set is B, if B is less than A, it indicates that the training samples of abnormal audio information are insufficient, which may make the initial abnormal detection model unable to learn the corresponding abnormal audio information. The data distribution of the first audio feature of , causes the initial anomaly detection model to be unable to accurately distinguish between normal audio and abnormal audio. The generator network in the detection model learns the data distribution of the first audio features corresponding to the normal audio information and abnormal audio information in the process of confrontation with the discriminator network, so that the adjusted generator network can be used as a target abnormality detection. The model performs anomaly detection on the input audio features.

In a model generation method provided by this embodiment, after acquiring a training data set including a first audio feature of normal audio information and abnormal audio information, the generator network to be trained is trained by the training data set, so as to convert the converged to-be-trained generator network. The generator network is trained as the initial anomaly detection model, and then the initial anomaly detection model is adjusted through the discriminator network to use the adjusted generator network as the target anomaly detection model. Through the above method, after the target anomaly detection model is obtained by training the first audio features of the normal audio information and the abnormal audio information, in the process of anomaly detection of the device to be detected, the first audio feature of the device to be detected can be input into the target The anomaly detection model performs anomaly detection on the equipment to be detected, which saves manpower and improves the efficiency of anomaly detection. Moreover, by adjusting the initial anomaly detection model through the discriminator network, the target anomaly detection model can also have better performance in the absence of abnormal audio training data, that is, it can distinguish between normal audio and abnormal audio. higher accuracy.

Please refer to FIG. 6 , a model generation method provided by the present application is applied to an electronic device, and the method includes:

S210: Acquire a training data set, where the training data set includes respective first audio features of multiple pieces of audio information of the target device, where the multiple pieces of audio information include normal audio information and abnormal audio information.

S220: Input the training data set into the generator network to be trained, and obtain the output of the generator network to be trained.

Wherein, as a method, the first audio features corresponding to the normal audio information and the abnormal audio information can be input into the generator network to be trained to obtain the output of the generator network to be trained. In this manner, there may be multiple normal audio information and multiple abnormal audio information, and each normal or abnormal audio information corresponds to a first audio feature.

S230: Train the generator network to be trained based on the output, the first loss function and the second loss function, so as to use the converged generator network to be trained as an initial anomaly detection model, wherein the first loss function is the absolute value of the difference between the output result of the first encoder and the output result of the second encoder, the second loss function is the absolute value of the difference between the first audio feature and the second audio feature, the The second audio feature is the output result of the decoder.

Wherein, in this embodiment of the present application, the first loss function can be used to minimize the distance between the output features of the first encoder and the output features of the second encoder in the generator network (Generator, G) to be trained, so that The generator network to be trained can learn the respective encoding feature distributions of normal audio information and abnormal audio information. As a way, the first loss function can be the absolute value of the difference between the output result of the first encoder and the output result of the second encoder, and the calculation formula of the first loss function is as follows:

Loss_g ₁ =‖z ₁ -z ₂ ‖

Wherein, z ₁ may represent the output result of the first encoder, and z ₂ may represent the output result of the second encoder.

Furthermore, in this embodiment of the present application, the second loss function may be used to minimize the distance between the first audio feature and the second audio feature in the generator network to be trained (Generator, G), so that the generator network to be trained The distribution of texture features corresponding to normal audio information and abnormal audio information can be learned. As a way, the second loss function may be the absolute value of the difference between the first audio feature and the output result of the decoder, and the calculation formula of the second loss function is as follows:

Loss_g ₂ =‖xG(x)‖

Wherein, x may represent the first audio feature, and G(x) may represent the output result of the decoder.

As a way, the weighted sum of the first loss function and the second loss function can be used as the loss function of the generator network to be trained, and the generator network to be trained is based on the output of the generator network to be trained and the loss function of the generator network to be trained. The network is trained to use the converged generator network to be trained as the initial anomaly detection model. The calculation formula of the loss function of the generator network to be trained is as follows:

Loss_G=xLoss_g ₁ +yLoss_g ₂

The sum of x and y is 1, and the values of x and y can be set based on experience, or can be obtained by training as trainable parameters of the generator network to be trained.

S240: Adjust the initial anomaly detection model through a discriminator network, so as to use the adjusted generator network as a target anomaly detection model.

In the model generation method provided by this embodiment, after the target abnormality detection model is obtained by training the normal audio information and the first audio feature of the abnormal audio information, in the process of abnormality detection of the device to be detected, it is possible to The first audio feature of the device to be detected is input into the target abnormality detection model to perform abnormality detection on the device to be detected, which saves manpower and improves the efficiency of abnormality detection. Moreover, by adjusting the initial anomaly detection model through the discriminator network, the target anomaly detection model can also have better performance in the absence of abnormal audio training data, that is, it can distinguish between normal audio and abnormal audio. higher accuracy. Moreover, in this embodiment, the generator network to be trained is trained through the output of the generator network to be trained, the first loss function and the second loss function, so that the converged generator network to be trained is used as the initial abnormality detection model, thereby The initial anomaly detection model can obtain the respective feature distributions of the normal audio information and the abnormal audio information, which improves the ability of the initial anomaly detection model to discriminate between the normal audio and the abnormal audio.

Please refer to FIG. 7 , a model generation method provided by the present application is applied to an electronic device, and the method includes:

S310: Acquire a training data set, where the training data set includes respective first audio features of multiple pieces of audio information of the target device, where the multiple pieces of audio information include normal audio information and abnormal audio information.

S320: Train the generator network to be trained by using the training data set, so as to use the converged generator network to be trained as an initial anomaly detection model.

S330: Obtain an abnormality detection model to be trained, where the abnormality detection model to be trained includes the initial abnormality detection model and the third encoder.

Wherein, as one way, as shown in FIG. 8 , the anomaly detection model to be trained may include a first encoder, a decoder, a second encoder and a third encoder, wherein the third encoder may be used as a discriminator network ( Discriminator, D), the input of the third encoder can be the first audio feature and the output of the decoder (the second audio feature).

Optionally, the structures of the first encoder, the second encoder and the third encoder may be the same.

S340: Input the training data set into the anomaly detection model to be trained to obtain an output of the anomaly detection model to be trained.

Wherein, as a method, the plurality of first audio features corresponding to the normal audio information and the abnormal audio information may be input into the abnormality detection model to be trained to obtain the output of the abnormality detection model to be trained.

S350: Adjust the anomaly detection model to be trained based on the output, the first loss function, the second loss function and the third loss function to obtain a converged anomaly detection model to be trained, wherein the first The three loss function is the absolute value of the difference between the third audio feature and the fourth audio feature, the third audio feature is the output result of the third encoder corresponding to the first audio feature, and the fourth audio feature is the The third encoder output result corresponding to the second audio feature.

Wherein, in the embodiment of the present application, the third loss function may be used to minimize the distance between the output feature of the discriminator network corresponding to the first audio feature and the output feature of the discriminator network corresponding to the second audio feature, so that the abnormality to be trained The detection model learns features that can fool the discriminator network (Discriminator, D), even if the discriminator network cannot confirm whether the second audio feature is a generated feature. As a way, the third loss function may be the absolute value of the difference between the third audio feature and the fourth audio feature, and the calculation formula of the third loss function is as follows:

Loss_d=‖D(x)-D(G(x))‖

Among them, D(x) can represent the third audio feature, and the third audio feature can be the output result obtained by inputting the first audio feature into the discriminator network; D(G(x)) can represent the fourth audio feature, the fourth audio feature The audio feature may be an output result obtained by inputting the second audio feature into the discriminator network.

As a way, the weighted sum of the first loss function, the second loss function and the third loss function can be used as the loss function of the anomaly detection model to be trained, and the output of the anomaly detection model to be trained and the loss of the anomaly detection model to be trained can be used as the loss function. The function trains the to-be-trained generator network to use the converged to-be-trained generator network as the initial anomaly detection model. The calculation formula of the loss function of the generator network to be trained is as follows:

Loss=xLoss_g ₁ +yLoss-g ₂ +zLoss_d

The sum of x, y, and z is 1, and the values of x, y, and z may be set based on experience, or may be obtained through training as trainable parameters of the anomaly detection model to be trained.

S360: Use the generator network in the converged anomaly detection model to be trained as a target anomaly detection model.

In the model generation method provided by this embodiment, after the target abnormality detection model is obtained by training the normal audio information and the first audio feature of the abnormal audio information, in the process of abnormality detection of the device to be detected, it is possible to The first audio feature of the device to be detected is input into the target abnormality detection model to perform abnormality detection on the device to be detected, which saves manpower and improves the efficiency of abnormality detection. Moreover, by adjusting the initial anomaly detection model through the discriminator network, the target anomaly detection model can also have better performance in the absence of abnormal audio training data, that is, it can distinguish between normal audio and abnormal audio. higher accuracy. In addition, in this embodiment, the discriminator network is used to discriminate the authenticity of the audio features of the generator network in the anomaly detection model to be trained (the label of the first audio feature is true, and the label of the second audio feature is false). Improve the generator network's ability to acquire normal audio features, thereby increasing the difference between the first audio features corresponding to normal audio and abnormal audio after feature extraction through the generator network, further improving the abnormality detection model to be trained. The performance of the generator network, that is, the performance of the target anomaly detection model.

Please refer to FIG. 9, an anomaly detection method provided by this application is applied to an electronic device, and the method includes:

S410: Acquire the audio to be detected.

Wherein, the audio frequency to be detected may be the sound produced by the power operating equipment (generator, motor, transformer, etc.) during operation, and the sound may be produced by the power equipment due to its own internal structure or hardware conditions. As a way, the audio to be detected can be obtained periodically through the audio collection device, so that the power running device can be detected in real time, so that when the power running device is abnormal, it can be discovered and maintained in time to avoid the occurrence of potential safety hazards. Exemplarily, the audio to be detected may be acquired by the audio collection device every 2s.

S420: Perform framing, windowing, and fast Fourier transform on the audio to be detected to obtain a first audio feature corresponding to the audio to be detected, where the first audio feature is a language corresponding to the audio to be detected Spectrum.

In this embodiment of the present application, the spectrogram may be a two-dimensional image, and the size of the spectrogram may be related to the number of points in the fast Fourier transform and the number of frames of the audio to be detected after sub-framing and windowing. Among them, the size of the first dimension of the spectrogram can be obtained by the formula: the number of fast Fourier transform points/2+1, and 1 can represent the DC component; the size of the second dimension of the spectrogram can be obtained by the formula: (sampling rate×audio duration -Sampling rate×frame length)/(sampling rate×frame shift)+1, where the unit of frame length and frame shift is s. Exemplarily, when the sampling rate is 16kHZ, the frame length is 25ms, the frame shift is 10ms, and the number of fast Fourier transform points is 512, a 257×198 spectrogram of the audio to be detected with a duration of 2s can be obtained.

S430: Input the first audio feature into a target abnormality detection model, and obtain a detection result output by the target abnormality detection model.

Among them, as a way, the spectrogram corresponding to the audio to be detected can be input into the target abnormality detection model, and the target abnormality detection model can output whether the audio to be detected is abnormal audio, if it is abnormal audio, it indicates the audio to be detected. If the corresponding power running device is abnormal, it is necessary to troubleshoot the power device; if the audio is normal, it means that the power running device corresponding to the audio to be detected is in a normal working state.

In an abnormality detection method provided by this embodiment, in the process of performing abnormality detection on the device to be detected, the first audio feature of the device to be detected can be input into the target abnormality detection model to perform abnormality detection on the device to be detected, thereby saving energy. It saves manpower and improves the efficiency of anomaly detection.

Please refer to FIG. 10 , a model generation apparatus 600 provided by the present application runs on an electronic device, and the apparatus 600 includes:

The data set acquisition unit 610 is configured to acquire a training data set, where the training data set includes respective first audio features of multiple audio information of the target device, and the multiple audio information includes normal audio information and abnormal audio information.

The initial abnormality detection model obtaining unit 620 is configured to train the generator network to be trained by using the training data set, so as to use the converged generator network to be trained as the initial abnormality detection model.

The target abnormality detection model obtaining unit 630 is configured to adjust the initial abnormality detection model through the discriminator network, so as to use the adjusted generator network as the target abnormality detection model.

Wherein, as a way, the data set acquisition unit 610 is specifically configured to acquire a plurality of audio information of the target device; perform framing, windowing, and fast Fourier transform on the audio information, so as to obtain the corresponding audio information of the audio information. Spectrogram; using the spectrogram as the first audio feature of the audio information.

In one way, the generator network includes a first audio feature reconstruction network and a feature extraction network, wherein the first audio feature reconstruction network includes a first encoder, an LSTM and a decoder, and the feature extraction network Including a second encoder, the initial anomaly detection model obtaining unit 620 is specifically configured to input the training data set into the generator network to be trained, and obtain the output of the generator network to be trained; based on the output, the first loss function and a second loss function to train the generator network to be trained to obtain an initial anomaly detection model, wherein the first loss function is the output result of the first encoder and the output result of the second encoder The absolute value of the difference of the second loss function is the absolute value of the difference between the first audio feature and the second audio feature, and the second audio feature is the output result of the decoder.

In one way, the discriminator network includes a third encoder, and the target anomaly detection model acquiring unit 630 is specifically configured to acquire an anomaly detection model to be trained, and the anomaly detection model to be trained includes the initial anomaly detection model and the a third encoder; input the training data set into the anomaly detection model to be trained, and obtain the output of the anomaly detection model to be trained; based on the output, the first loss function, the second loss function and A third loss function adjusts the anomaly detection model to be trained to obtain a converged anomaly detection model to be trained, wherein the third loss function is the absolute value of the difference between the third audio feature and the fourth audio feature, and the The third audio feature is the output result of the third encoder corresponding to the first audio feature, and the fourth audio feature is the output result of the third encoder corresponding to the second audio feature; The generator network in the detection model is used as the target anomaly detection model.

Optionally, the structures of the first encoder, the second encoder and the third encoder are the same.

Please refer to FIG. 11 , an abnormality detection apparatus 800 provided by the present application operates on an electronic device, and the apparatus 800 includes:

The detection audio acquisition unit 810 is configured to acquire the audio to be detected.

The first audio feature acquisition unit 820 is used to perform framing, windowing, and fast Fourier transform on the audio to be detected, so as to obtain a first audio feature corresponding to the audio to be detected, where the first audio feature is The spectrogram corresponding to the audio to be detected.

The detection result obtaining unit 830 is configured to input the first audio feature into the target abnormality detection model, and obtain the detection result output by the target abnormality detection model.

Wherein, as a way, the detection audio acquisition unit 810 is specifically configured to periodically acquire the audio to be detected.

An electronic device provided by the present application will be described below with reference to FIG. 12 .

Referring to FIG. 12 , based on the above-mentioned model generation method, anomaly detection method, and apparatus, an embodiment of the present application further provides another electronic device 100 that can execute the aforementioned model generation method and anomaly detection method. The electronic device 100 includes one or more (only one is shown in the figure) a processor 102 and a memory 104 that are coupled to each other. Wherein, the memory 104 stores a program that can execute the content in the foregoing embodiments, and the processor 102 can execute the program stored in the memory 104 .

The processor 102 may include one or more processing cores. The processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 104, and calling the data stored in the memory 104. Various functions of the electronic device 100 and processing data. Optionally, the processor 102 may use at least one of digital signal processing (Digital Signal Processing, DSP), field-programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). implemented in hardware. The processor 102 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 102, and is implemented by a communication chip alone.

The memory 104 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 104 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 104 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data created by the terminal 100 during use (such as phone book, audio and video data, chat record data) and the like.

Please refer to FIG. 13 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program codes are stored in the computer-readable storage medium 1000, and the program codes can be invoked by a processor to execute the methods described in the above method embodiments.

The computer-readable storage medium 1000 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. Computer readable storage medium 1000 has storage space for program code 1010 to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. Program code 1010 may be compressed, for example, in a suitable form.

To sum up, a model generation method, anomaly detection method, device and electronic device provided by the present application, after acquiring the training data set including the first audio features of normal audio information and abnormal audio information, through the training data set The generator network to be trained is trained to use the converged generator network to be trained as the initial anomaly detection model, and then the initial anomaly detection model is adjusted through the discriminator network to use the adjusted generator network as the target anomaly detection model. Through the above method, after the target anomaly detection model is obtained by training the first audio features of the normal audio information and the abnormal audio information, in the process of anomaly detection of the device to be detected, the first audio feature of the device to be detected can be input into the target The anomaly detection model performs anomaly detection on the equipment to be detected, which saves manpower and improves the efficiency of anomaly detection. Moreover, by adjusting the initial anomaly detection model through the discriminator network, the target anomaly detection model can also have better performance in the absence of abnormal audio training data, that is, it can distinguish between normal audio and abnormal audio. higher accuracy.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (11)

1. A model generation method applied to an electronic device, the method comprising:

acquiring a training data set, wherein the training data set comprises respective first audio features of a plurality of pieces of audio information of a target device, and the plurality of pieces of audio information comprise normal audio information and abnormal audio information;

training a generator network to be trained through the training data set to take the converged generator network to be trained as an initial anomaly detection model;

and adjusting the initial anomaly detection model through a discriminator network to take the adjusted generator network as a target anomaly detection model.

2. The method of claim 1, wherein the generator network to be trained comprises a first audio feature reconstruction network and a feature extraction network, wherein the first audio feature reconstruction network comprises a first encoder, an LSTM, and a decoder, and wherein the feature extraction network comprises a second encoder;

training the generator network to be trained through the training data set to take the converged generator network to be trained as an initial anomaly detection model, including:

inputting the training data set into the generator network to be trained to obtain the output of the generator network to be trained;

training the generator network to be trained based on the output, a first loss function and a second loss function to take the converged generator network to be trained as an initial anomaly detection model, wherein the first loss function is an absolute value of a difference between the first encoder output result and the second encoder output result, the second loss function is an absolute value of a difference between the first audio feature and a second audio feature, and the second audio feature is an output result of the decoder.

3. The method of claim 2, wherein the network of discriminators includes a third encoder, and wherein adjusting the initial anomaly detection model by the network of discriminators to have the adjusted generator network as a target anomaly detection model comprises:

acquiring an anomaly detection model to be trained, wherein the anomaly detection model to be trained comprises the initial anomaly detection model and the third encoder;

inputting the training data set into the anomaly detection model to be trained to obtain the output of the anomaly detection model to be trained;

adjusting the anomaly detection model to be trained based on the output, the first loss function, the second loss function and a third loss function to obtain a converged anomaly detection model to be trained, wherein the third loss function is an absolute value of a difference between a third audio feature and a fourth audio feature, the third audio feature is a third encoder output result corresponding to the first audio feature, and the fourth audio feature is a third encoder output result corresponding to the second audio feature;

and taking a generator network in the converged anomaly detection model to be trained as a target anomaly detection model.

4. The method of claim 3, wherein the first encoder, the second encoder, and the third encoder are identical in structure.

5. The method of claim 1, wherein obtaining a training data set that includes first audio features of respective audio information of a target device, the audio information including normal audio information and abnormal audio information, comprises:

acquiring a plurality of audio information of a target device;

performing framing, windowing and fast Fourier transform on the audio information to obtain a spectrogram corresponding to the audio information;

and taking the spectrogram as a first audio feature of the audio information.

6. An abnormality detection method applied to an electronic device, the method comprising:

acquiring audio to be detected;

performing framing, windowing and fast Fourier transform on the audio to be detected to obtain a first audio characteristic corresponding to the audio to be detected, wherein the first audio characteristic is a spectrogram corresponding to the audio to be detected;

inputting the first audio characteristic into a target abnormity detection model obtained by the method of any one of claims 1 to 5, and obtaining a detection result output by the target abnormity detection model.

7. The method according to claim 6, applied to an electronic device, wherein the acquiring the audio to be detected comprises:

the audio to be detected is acquired periodically.

8. An apparatus for model generation, operable on an electronic device, the apparatus comprising:

the data set acquisition unit is used for acquiring a training data set, wherein the training data set comprises first audio features of a plurality of pieces of audio information of the target equipment, and the plurality of pieces of audio information comprise normal audio information and abnormal audio information;

an initial anomaly detection model obtaining unit, configured to train a generator network to be trained through the training data set, so as to use the converged generator network to be trained as an initial anomaly detection model;

and the target anomaly detection model acquisition unit is used for adjusting the initial anomaly detection model through the discriminator network so as to take the adjusted generator network as a target anomaly detection model.

9. An anomaly detection apparatus, operable with an electronic device, the apparatus comprising:

the audio acquisition unit to be detected is used for acquiring audio to be detected;

the first audio characteristic acquisition unit is used for performing framing, windowing and fast Fourier transform on the audio to be detected to obtain a first audio characteristic corresponding to the audio to be detected, wherein the first audio characteristic is a spectrogram corresponding to the audio to be detected;

a detection result obtaining unit, configured to input the first audio feature into the target anomaly detection model obtained by the method according to any one of claims 1 to 5, and obtain a detection result output by the target anomaly detection model.

10. An electronic device comprising one or more processors and memory;

one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-7.

11. A computer-readable storage medium, having program code stored therein, wherein the method of any of claims 1-7 is performed when the program code is run.

Images