CN110602078B - Application encryption traffic generation method and system based on generation countermeasure network - Google Patents

Abstract

The invention discloses a method and system for generating application encrypted traffic based on a generative confrontation network, which extracts encrypted traffic packets (including data packet headers) of real applications into decimal data and intercepts them to a fixed length (possible digits are filled with 0) , separated by commas, each line is a piece of traffic data, which is sent to the generative confrontation network (GAN) for feature extraction, after the generator and discriminator of the GAN network tend to be stable. Inputting a small amount of encrypted traffic of real applications into the generator of GAN can generate any number of encrypted traffic containing the characteristics of the application traffic. This method cleverly abstracts the features of encrypted traffic through GAN, without decrypting the traffic itself, reducing the work of decryption while effectively protecting user privacy and greatly reducing the cost of obtaining samples. This method is applicable to all encrypted traffic recognition scenarios based on deep learning, where few encrypted traffic samples are difficult to obtain and the recognition rate is low.

Description

A method and system for generating application encrypted traffic based on generative confrontation network

technical field

The invention relates to a method for generating application encryption traffic based on a generative confrontation network (GAN), and belongs to the technical field of data encryption.

Background technique

Traffic classification and identification are the basis for improving network management and security monitoring, improving service quality, and also the prerequisite for network behavior such as network design and planning. With the enhancement of user privacy protection and security awareness, technologies such as SSL, SSH, and VPN have been more and more widely used, resulting in an increasing proportion of encrypted traffic in network transmission.

Due to the use of application layer encryption, traditional port matching and DPI can no longer accurately identify application traffic; compared with machine learning, deep learning can better express the essential characteristics of data, but its training relies on a large number of labeled samples. Accuracy directly leads to the recognition rate of the training results.

The main idea of the generative confrontation network (GAN) algorithm is to use the two networks of the generation network and the discriminant network to converge to the optimal solution through a minimax game. GAN has shown its great advantages in the generation of images, sounds and texts. In order to learn the distribution of the generator G(z), the input noise variable p _z (z) is pre-defined and the distribution D(x) of the discriminator is defined, which means that x comes from the real data p _data (x) instead of G( z) probability. By training the discriminator D, the probability of distinguishing the real sample from the real sample and the sample generated by the generator G is as far as possible, so that log(1-D(G(z))) is minimized. To sum up, D and G use the value function V(D,G) to perform the following two-person maximin game, that is, the loss of GAN is shown in the following formula (1):

The generator G takes random noise as input and generates samples. The discriminator D accepts data from both the generator and real samples and tries to distinguish between the two. These two networks are playing a game, the generator G is constantly learning to generate more realistic samples, and the discriminator D is constantly learning to better distinguish between generated samples and real samples. The two networks are trained simultaneously, and this game is expected to make generated samples indistinguishable from real samples. The output of the generator G is a synthesized sample X _fake =G(z). The discriminator D receives synthetic samples or real data samples from the generator, and outputs a probability distribution of possible sources P(F|X)=D(X) as shown in Equation (2). The discriminator is trained to maximize logD(x), while the generator G is trained to minimize log(1-D(G(z))) in Eq.

L=E[logP(F=real|X _real )]+E[logP(F=fake|X _fake )] (2)

Discriminator's loss logD(x) Generator's loss log(1-D(G(z)))

About training:

Each round of training contains two parts minG and maxD,

Train the discriminator:

First look at the maxD part. The training is generally to keep G (generator) unchanged and train D. The training goal of D is to correctly distinguish fake/true. If we use 1/0 to represent true/fake, then for the first item E, because the input is sampled from real data, we expect D(x) to approach 1, which is the first item is larger. And z is a random input, and G(z) represents the generated samples. For the generated samples, we hope that the discriminant's discriminant result D(G(z)) is closer to 0, the better, that is, to maximize the total value, and also It is the meaning of maxD.

Training generator:

The second part keeps D unchanged and trains G. At this time, only the second item E is useful, because we want to confuse D. It is better to hope that the output of D(G(z)) is close to 1, that is, the smaller the item, the better. Well, this is minG. Of course, the discriminator is not so easy to fool, so at this time, the discriminator will generate a relatively large error, and the error will update G, then G will become better.

Repeat the above two steps

After N rounds of training iterations, the discriminator and generator losses stabilize.

In addition, it is very difficult to obtain and mark the traffic of encryption applications. To achieve the sample size required for training a better model, direct collection is difficult and costly.

Contents of the invention

Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a method and system for generating application-encrypted traffic based on Generative Adversarial Networks. The present invention uses real data of a small amount of applications to generate The confrontation mechanism extracts the encrypted traffic characteristics of each application, triggers the generator of the generative adversarial network (GAN) by generating Gaussian noise, and generates new data with the encrypted traffic characteristics of the application. Because the new data has the characteristics of application encryption, it can participate in the subsequent identification model training, which greatly reduces the difficulty of obtaining deep learning samples and the high cost of labeling.

Technical scheme: in order to achieve the above object, the technical scheme adopted in the present invention is:

A method for generating application encrypted traffic based on generative confrontation network, comprising the following steps:

Step 1. Obtain the encrypted traffic data of the real application, extract the encrypted traffic data of the real application, convert it into decimal, and intercept it to 784 bits. If it is insufficient, use 0 to fill it up. Use "," to divide each byte. Put only one piece of traffic data in one line, and put the traffic of the same application in the same csv file in the above format.

Step 2, build a GAN, which includes a generator and a discriminator connected together, send the csv file to the discriminator in the GAN for feature extraction, and use Gaussian noise to trigger the generator. The generated data is sent to the discriminator together, and the generator and the discriminator are trained at the same time until the loss of the generator and the discriminator tends to be stable

Step 3: Use a specified number of Gaussian noises to trigger the generator, and the generator generates a corresponding amount of generated traffic that contains the encrypted traffic characteristics of the application. The format of the generated traffic is the same as the encrypted traffic data of the real application, with a length of 784 bytes. One row is a piece of traffic data, and finally the generated data is tagged with an application label. The label is specified manually. Each row of data generated by application 1 corresponds to a tag 1, and each row of data generated by application 2 corresponds to a tag 2. By analogy, the generated data and label data are stored in two csv files to achieve the purpose of generating a labeled data set.

Preferably: by repeating steps 1-3 multiple times, the generated data of encrypted traffic data of different real applications and the corresponding tag data can be obtained.

Preferably: the encrypted traffic data of the real application includes a data packet header.

A generation system that adopts an application encryption traffic generation method based on a generation confrontation network, including a generation confrontation network GAN, an input module, an output module and a Gaussian noise generator, and the generation confrontation network GAN includes a generator and a discriminator connected together , the Gaussian noise generator is connected to the generator, where:

The input module is used to obtain the encrypted traffic data of the real application, extract the encrypted traffic data of the real application, convert it into a decimal system, and intercept it to 784 bits, fill it with 0 if it is insufficient, and use ", ", put only one piece of traffic data in each line, and put the traffic of the same application in the same csv file in the above format.

The Gaussian noise generator is used to trigger the generator by generating a set of random Gaussian noise data.

The generator corrects the weights of each neuron in the network according to the discrimination results sent by the discriminator through the Gaussian noise data entering the neural network, and fits the generated traffic containing features.

Discriminator: The neural network trained by the encrypted traffic data of the real application input by the input module makes a judgment on whether the generated traffic generated by the generator contains application characteristics, and feeds back the judgment result to the generator until the generator and the discriminator The loss tends to be stable, and the discriminator can no longer distinguish whether the input data is real data or the data generated by the generator, and the training of the generative confrontation network GAN is completed.

The output module is used to output generated traffic generated by the trained generator.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention is based on a Generative Adversarial Network (GAN) application encrypted traffic generation method, which uses a small amount of encrypted traffic of real applications to generate any number of encrypted traffic containing the characteristics of the application traffic. The method does not need to collect a large number of Sample data, only a few thousand pieces of data are needed to generate better results. This method cleverly abstracts the features in the encrypted traffic through the Generative Adversarial Network (GAN), and does not need to decrypt the traffic itself, which reduces the work of decryption while effectively protecting user privacy and greatly reducing the cost of obtaining samples. This method is applicable to all encrypted traffic recognition scenarios based on deep learning, where there are few encrypted traffic samples, difficult acquisition, and low recognition rate.

The present invention can simply and quickly generate encrypted traffic with better application characteristics by means of a small amount of encrypted traffic data of real applications, thereby greatly reducing the cost of traffic collection and marking.

Description of drawings

Figure 1 shows the content and format of real data and generated data.

Figure 2 is the generation process and the neural network structure of the generator and the discriminator.

Figure 3 is the recognition of real data on the recognition model trained on real data (4 applications as examples)

Figure 4 shows the recognition of real data on a model that is jointly trained with generated data and real data (four applications as examples)

detailed description

Below in conjunction with accompanying drawing and specific embodiment, further illustrate the present invention, should be understood that these examples are only for illustrating the present invention and are not intended to limit the scope of the present invention, after having read the present invention, those skilled in the art will understand various aspects of the present invention All modifications of the valence form fall within the scope defined by the appended claims of the present application.

A method for generating application encrypted traffic based on generative confrontation networks, as shown in Figure 1, extracts traffic data (including protocol headers) from encrypted traffic packets of real applications, converts them into decimals, and intercepts them to 784 bits ( Use "," to separate each byte, put only one flow data in each line, put the flow of the same application in the same csv file in the above form, and send it to the generative confrontation network ( GAN) for feature extraction, and then use the specified number of Gaussian noise to trigger the generator of the generation confrontation network, generate the corresponding number of new data with its application encryption features and generate the corresponding label data (one line of generated data corresponds to one label) , the label is manually specified, each line of the generated data of application 1 corresponds to a label 1, each line of the generated data of application 2 corresponds to a label 2, and so on, the generated data and label data are stored in two csv files, including the following steps :

Step 1. Obtain the encrypted traffic data (including the protocol header) of the real application, extract the encrypted traffic data of the real application, convert it into decimal, and intercept it to 784 bits (if it is insufficient, fill it with 0), and use 0 between each byte "," split, each line only puts one piece of traffic data, and puts the traffic of the same application in the same csv file in the above format. Extract the encrypted traffic packet (including the data packet header) of the real application into decimal data and intercept it to a fixed length (fill the gap with 0), separated by commas, each line is a traffic data, and send it to the generated confrontation network (GAN) is used to train the discriminator so that the discriminator can better identify the encrypted traffic characteristics of real applications.

Step 2, build a GAN, which includes a generator and a discriminator connected together, send the csv file to the discriminator in the GAN for feature extraction, and use Gaussian noise to trigger the generator. The generated data is sent to the discriminator together, and the generator and the discriminator are trained at the same time until the loss of the generator and the discriminator tends to be stable, that is, the loss is less than the predetermined loss threshold.

Step 3: Use a specified number of Gaussian noises to trigger the generator, and the generator generates a corresponding amount of generated traffic that contains the encrypted traffic characteristics of the application. The format of the generated traffic is the same as the encrypted traffic data of the real application, with a length of 784 bytes. One row is a piece of traffic data, and finally the generated data is tagged with an application label. The label is specified manually. Each row of data generated by application 1 corresponds to a tag 1, and each row of data generated by application 2 corresponds to a tag 2. By analogy, the generated data and label data are stored in two csv files. The generated traffic is passed to the discriminator, and the discriminator distinguishes the generated traffic and the encrypted traffic characteristics of the real application according to the preset threshold, and feeds back the discrimination result to the generator, and the generator updates it according to the discrimination result, and iterates until the discriminator The generated traffic and the encrypted traffic characteristics of the real application are discriminated to reach the preset threshold, and the training of the generated confrontation network GAN is completed.

Step 4, use the specified number of Gaussian noises to trigger the trained generator, and the trained generator will generate a corresponding number of generated traffic that contains the encrypted traffic characteristics of the application characteristics, as shown in Figure 1, to achieve the generation of marked Purpose of the dataset.

Repeat steps 1-3 multiple times to obtain the generated data and corresponding label data of encrypted traffic data of different real applications.

Preferably: the encrypted traffic data of the real application includes a data packet header.

A generation system that adopts an application encryption traffic generation method based on a generation confrontation network, as shown in Figure 2, includes a generation confrontation network GAN, an input module, an output module and a Gaussian noise generator, and the generation confrontation network GAN includes a A generator and a discriminator together, the Gaussian noise generator connected to the generator, where:

The input module is used to obtain the encrypted traffic data of the real application, extract the encrypted traffic data of the real application, convert it into a decimal system, and intercept it to 784 bits, fill it with 0 if it is insufficient, and use ", ", put only one piece of traffic data in each line, and put the traffic of the same application in the same csv file in the above format.

The Gaussian noise generator is used to trigger the generator by generating a set of random Gaussian noise data.

The generator corrects the weights of each neuron in the network according to the discrimination results sent by the discriminator through the Gaussian noise data entering the neural network, and fits the generated traffic containing features.

Discriminator: The neural network trained by the encrypted traffic data of the real application input by the input module makes a judgment on whether the generated traffic generated by the generator contains application characteristics, and feeds back the judgment result to the generator until the generator and the discriminator The loss tends to be stable, and the discriminator can no longer distinguish whether the input data is real data or the data generated by the generator, and the training of the generative confrontation network GAN is completed.

The present invention utilizes the real encrypted traffic of the public data set or the collected application, and iteratively generates the discriminator and generator of the confrontational network (GAN) through training, and finally enables the generator to better grasp the traffic characteristics of the application. For Gaussian noise, the generator can generate a corresponding amount of encrypted traffic containing application traffic characteristics to expand the data set and label set. Use public datasets or collected real data to randomly generate a specified number of Gaussian noises, send them into the iteratively trained Generative Adversarial Network (GAN) model, and use the generator to generate a corresponding number of noises containing the characteristics of the application Encrypt traffic data.

Using the specified number of Gaussian noises, the generator can generate a corresponding amount of generated data containing the characteristics of the encrypted traffic of the application, and can achieve a better recognition rate when training the model with real data, so as to expand the data set and reduce data collection. , Mark the cost, and improve the accuracy of the recognition model.

In order to prove that the data generated by this method has the characteristics of real applications, we conducted a generation experiment with 4 common applications (aimchat, hangouts, icq, netflix) in the "ISCX VPN-non VPNtraffic dataset", by combining the real data with the generated Mixing data together for training can effectively improve the recognition rate. Figure 3 shows the recognition of real data on the recognition model trained on real data (for example, 4 applications). Figure 4 shows the model jointly trained with generated data and real data. The recognition situation of the real data (4 kinds of applications as an example). As shown in Figure 3 and Figure 4, the data generated by this method can effectively improve the recognition rate of the application, which proves that the data generated by this method can expand the data set and reduce the difficulty of obtaining data and labeling.

The present invention utilizes a small amount of encrypted flow of real applications to generate any number of encrypted flows containing the characteristics of the application flow. This method does not need to collect a large number of sample data, and only a few thousand pieces of data can generate better results. This method cleverly abstracts the features of encrypted traffic through GAN, without decrypting the traffic itself, reducing the work of decryption, effectively protecting user privacy and greatly reducing the cost of obtaining samples. This method is applicable to all encrypted traffic recognition scenarios based on deep learning, where few encrypted traffic samples are difficult to obtain and the recognition rate is low.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications are also possible. It should be regarded as the protection scope of the present invention.

Claims (3)

1. A method for generating application encrypted traffic based on generating an adversarial network, characterized in that, comprising the following steps: Step 1. Get the encrypted traffic data of the real application and extract the encrypted traffic data of the real application. There is no need to decrypt the encrypted traffic data itself, convert it into decimal, and intercept it to 784 bits. Sections are separated by ",", and only one flow data is placed in each line, and the flow of the same application is placed in the same csv file in the above format; Step 2, build a GAN, which includes a generator and a discriminator connected together, send the csv file to the discriminator in the GAN for feature extraction, and use Gaussian noise to trigger the generator. The generated data is sent to the discriminator together, and the generator and the discriminator are trained at the same time until the loss of the generator and the discriminator tends to be stable; the Gaussian noise generator generates a set of random Gaussian noise data to trigger the generator; Step 3, use the specified amount of Gaussian noise to trigger the generator, and the generator generates a corresponding amount of generated traffic that contains the characteristics of the encrypted traffic of the application. The format of the generated traffic is the same as the encrypted traffic data of the real application, with a length of 784 bytes. Each line is A piece of traffic data, and finally mark the generated data with an application label. The label is specified manually. Each line of the generated data of application 1 corresponds to a label 1, and each line of the generated data of application 2 corresponds to a label 2. By analogy, the generated data and labels The data is stored in two csv files; by generating a specified amount of Gaussian noise, the generator generates a corresponding amount of encrypted traffic containing application traffic characteristics to achieve the purpose of expanding the data set and label set; Repeat steps 1-3 multiple times to obtain the generated data and corresponding label data of encrypted traffic data of different real applications. 2. The method for generating application encrypted traffic based on Generative Adversarial Networks according to claim 1, characterized in that: the encrypted traffic data of the real application includes a data packet header. 3. A generation system that adopts the generation method of application encryption traffic based on generating confrontation network according to claim 1, characterized in that: comprising generating confrontation network GAN, input module, output module and Gaussian noise generator, said generating confrontation network GAN includes a generator and a discriminator connected together, the Gaussian noise generator is connected to the generator, where: The input module is used to obtain the encrypted traffic data of the real application, extract the encrypted traffic data of the real application, convert it into a decimal system, and intercept it to 784 bits, fill it with 0 if it is insufficient, and use ", "Split, put only one piece of traffic data in each line, and put the traffic of the same application in the same csv file in the above format; The Gaussian noise generator is used to trigger the generator by generating a set of random Gaussian noise data; The generator corrects the weights of each neuron in the network according to the discriminant results sent by the discriminator through the Gaussian noise data entering the neural network, and fits the generated traffic containing features; Discriminator: The neural network trained by the encrypted traffic data of the real application input by the input module makes a judgment on whether the generated traffic generated by the generator contains application characteristics, and feeds back the judgment result to the generator until the generator and the discriminator The loss tends to be stable, and the discriminator can no longer distinguish whether the input data is real data or the data generated by the generator, and the training of the generated confrontation network GAN is completed; The output module is used to output generated traffic generated by the trained generator.

Images