CN101635658A - Method and system for detecting abnormality of network secret stealing behavior - Google Patents

Abstract

The present invention provides a method for abnormal detection of network theft, including: receiving network data packets, and performing protocol restoration on the obtained network data packets; making a correlation judgment between the network data packets and the target of interest, and discarding Relevant network data packets, perform the next step on the relevant network data packets; determine the connection to which the network data packets related to the concerned target belong; wherein, the connection includes a TCP connection or a UDP connection; Carry out statistics on the detection feature vectors on all connections related to the concerned target; calculate the value of each detection feature in the detection feature vector corresponding to the connection related to the concerned target according to the statistical results; judge the corresponding connection according to the value of the detection feature Whether there is an exception. The invention does not need to analyze and extract fixed features, and can discover unknown network theft behaviors in time; does not need to process the data load of network data packets, improves detection efficiency, and is suitable for application deployment in large-scale network environments.

Description

Abnormal detection method and system for network theft

technical field

The invention relates to the field of network security, in particular to an abnormal detection method and system for network theft.

Background technique

In recent years, with the development of network intrusion technology and the proliferation of malicious codes such as Trojan horses, viruses, and bots, network and information security issues have become increasingly prominent, especially network confidentiality and theft (synthetic network theft) incidents have emerged in an endless stream, some of which have significant impacts. security incidents, endangering social economy and national security. Therefore, the research work on network stealing behavior and stealing behavior detection (synthetic network stealing behavior detection) is particularly important.

Similar to network intrusion detection, network theft detection methods can also be divided into two categories: misuse detection and anomaly detection. Misuse detection mainly extracts its corresponding fixed features (usually character string features or binary string features) through in-depth analysis of stolen secrets, and combines features such as secret-related and sensitive keywords to perform feature pattern matching on network data. If the matching is successful, it is considered that there is a secret theft. The general principle of anomaly detection is to pre-establish the behavior profile of normal network activities through the statistical analysis of historical network data, and then count the current network behavior during detection. The behavior is cyber theft.

The network intrusion detection in the prior art often adopts the misuse detection method, which is more suitable for detecting known behaviors, and has the advantages of high efficiency and accuracy under the premise of selecting appropriate keywords or features. However, due to the inherent deficiencies of misuse detection methods, the existing techniques have the following problems:

1) Lack of ability to discover unknown theft of secrets;

2) Since the misuse detection method can only detect plaintext information, the existing methods lack the ability to detect encrypted information;

3) In practice, due to the variation of stealing behavior and the emergence of new behaviors, it is difficult to extract fixed features, or it cannot keep up with the change of stealing behavior, so the detection effect is not good.

Contents of the invention

The purpose of the present invention is to overcome the defects of the existing anomaly detection method for network theft behavior, such as lack of ability to discover unknown theft events and poor detection effect, so as to provide an efficient and fast abnormal detection method for network theft behavior.

In order to achieve the above object, the present invention provides a method for abnormal detection of network theft secret behavior, comprising:

Step 1), receiving the network data packet, and performing protocol restoration on the obtained network data packet;

Step 2), making a correlation judgment between the network data packet and the target of concern, discarding the network data packet without correlation, and performing the next step for the network data packet with correlation;

Step 3), determine the connection to which the network data packet related to the concerned target belongs; wherein, the connection includes a TCP connection or a UDP connection;

Step 4), making statistics on the detection feature vectors on all connections related to the target of interest; wherein,

The detection feature vector includes the latest access interval feature last_interval used to represent the time interval from the latest access to the current connection to the time of detection, the traffic feature traffic used to represent the average flow of the current connection within the detection time interval, and used to represent The periodic behavior of the current connection in the time interval of the most recent visits. period, which is used to indicate the same sip connection with the same sip as the current connection and the number of connections that have successively accessed within the detection time interval. The successive access feature sip_count, the connection feature dip_count of the same dip used to indicate the number of connections with the same dip as the current connection within the detection time interval;

Step 5), according to the statistical results, calculate the value of each detection feature in the detection feature vector corresponding to the connection related to the target of interest;

Step 6), judging whether the corresponding connection is abnormal according to the value of the detection feature.

In the above-mentioned technical solution, a learning stage is also included before the step 1), and the learning stage includes steps 1)-step 4), so as to obtain a large amount of statistics of the detection feature vectors on all connections related to the concerned target result.

In the above technical solution, in said step 2), said correlation judgment includes:

Matching the source IP address or the destination IP address of the network data packet with the IP address of the target of interest, if one of the source IP address and the destination IP address matches the IP address of the target of concern, Then there is a correlation between the network data packet and the target of interest; otherwise, there is no correlation.

In the above-mentioned technical scheme, described step 3) comprises:

Step 3-1), when the network data packet related to the concerned target is a TCP synchronization packet, if the network data packet does not belong to an existing connection, then a new TCP connection is established for the network data packet, if the The network packet belongs to an existing connection, and a new access to the existing connection is started;

Step 3-2), when the network data packet related to the concerned target is a TCP asynchronous packet, if the network data packet does not belong to an existing connection, then discard the network data packet, if the network data packet belongs to a If there is already a connection, keep the network data packet;

Step 3-3), when the network data packet related to the concerned target is a UDP packet, if the network data packet does not belong to an existing connection, a new UDP connection is established for it, if the network data packet belongs to An existing connection, when the existing connection meets the division rules of UDP connections, start a new access to the current existing connection, and then do subsequent statistical processing; when the existing connection does not meet the division rules of UDP connections, Do the follow-up statistical processing directly.

In the above technical solution, the division rules of the UDP connection include:

(1), for the UDP packet whose destination port is 53, each UDP packet is a UDP connection;

(2) For all other UDP packets, set the UDP connection timer and division threshold, set the timer to 0 when the connection is established or each access starts, and determine whether the timer exceeds the preset value when processing each UDP packet If the division threshold is exceeded, the connection division will be performed, a new access of the current connection will be started, and its timer will be reset to 0. If it does not exceed, no division is required.

In the above-mentioned technical scheme, in described step 4),

The statistics of the latest access interval feature last_interval include: use the variable last_time to record the time when the current connection is established or the time when the last visit occurred; wherein, when the connection is established, let last_time be 0, and when a new visit occurs, let last_time is the current time;

The statistics of the traffic characteristic traffic include: using the variable sum to count the sum of all data packet lengths currently connected in the detection interval T ₀ ;

The statistics of the periodic feature period of the behavior include: using a circular queue with a length of n to record the time interval of the latest n accesses of the current connection;

The statistics of the successive access feature sip_count to the same sip connection include: adopt the CountingBloom Filter method to count the sip_count feature; or,

Let start_time be the time when the current connection is established or access occurs, and flag is the flag indicating whether the current connection has been counted. The statistical elements of the Counting Bloom Filter method include the counter count and the end time end_time, both of which are initialized to 0; when the current connection access ends, If flag is equal to 1, then return, otherwise find the statistical element corresponding to the currently connected sip through the Counting Bloom Filter method, if start_time is greater than end_time, then set flag=1, count+1, end_time=current access end time, otherwise if If start_time is less than or equal to end_time, return;

The statistics of the connection feature dip_count of the same dip includes: using the Counting BloomFilter method to count the current connected dips.

In the above-mentioned technical scheme, in described step 5), the calculation to detection characteristic value comprises:

The value of the latest access interval feature last_interval is the difference between the current time and the variable last_time;

The value of the traffic characteristic traffic is the product of the variable sum and 8 and the quotient of the detection interval _T0 ;

The calculation formula of the value of the periodic characteristic period of the behavior is:

$\mathrm{<span\; class="notranslate">\; period</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ~</span>\frac{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; ;</span>$

Wherein, n is the length of the circular queue; Max represents the maximum value in the set; m represents the packet length; p (m) represents the uniformity that the average number of each packet element presents when the packet length is m, its size Calculated using the following formula:

为分组的个数， $\stackrel{\&OverBar;}{q}=\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}{q}_j$ 为队列元素的平均数， $p_i=\underset{j=m\&times;(i-1)+1}{\overset{\mathrm{Min}(m\&times;i,n)}{\mathrm{\&Sigma;}}}{q}_j/m$ 为第i分组元素的平均数；Among them, Min means to take the minimum value in the set,

is the number of groups, $\stackrel{\&OverBar;}{q}=\underset{j=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{q}_j$ is the average number of queue elements, $p_i=\underset{j=m\&times;(i-1)+1}{\overset{\mathrm{Min}(m\&times;i,\mathrm{no})}{\mathrm{\&Sigma;}}}{q}_j/m$ is the average number of i grouping elements;

The value of the successive access feature sip_count of the same sip connection is the count value of the statistical element in the Counting Bloom Filter method;

The value of the connection feature dip_countsip_count of the same dip is the count value of the statistical element in the Counting BloomFilter method.

In above-mentioned technical scheme, described step 6) comprises:

Step 6-1), judging whether there is an abnormality in the detection feature;

In step 6-2), it is judged whether there is an abnormality in the corresponding connection according to the abnormality judgment result of the detection feature.

In the above technical solution, in the step 6-1), a method of comparing the detection feature with a corresponding detection threshold or a model judgment method is used to judge whether the detection feature is abnormal.

In the above technical solution, in the step 6-2), it is judged whether there is an abnormality in the connection by using a method of setting a judgment standard or a method of model judgment.

In the above-mentioned technical scheme, described judging standard comprises: if the most recent visit interval characteristic last_interval and traffic characteristic traffic are abnormal, or if the most recent visit interval characteristic last-interval and behavior periodicity characteristic period are abnormal, or connect successively with sip access characteristic sip_count If there is an exception, or if the latest access interval feature last_interval and the connection feature of the same dip are abnormal at the same time, the connection will behave abnormally.

In the above-mentioned technical scheme, in described step 6-2), also include:

Extract the secret information and the secret information; wherein, sip is a secret stealer, dip is a secret stealer, if the direction of the current connection is in, then the IP address of the concerned target is suspected of being a secret stealer, if the direction of the current connection is out, then the The IP address of the target of concern is suspected to be a leaker.

The present invention also provides an abnormality detection system for network theft, including a data packet recovery module, a correlation judgment module, a connection judgment module, a detection feature vector statistics module, a detection feature value calculation module, and an abnormality judgment module; wherein,

The data packet restoration module receives the network data packet, and performs protocol restoration on the obtained network data packet;

The correlation judgment module makes a correlation judgment between the network data packet and the target of interest, discards the network data packets without correlation, and sends the relevant network data packets to the connection judgment module;

The connection determination module determines the connection to which the network data packet related to the concerned target belongs; wherein, the connection includes a TCP connection or a UDP connection;

The detection feature vector statistics module performs statistics on the detection feature vectors on all connections related to the target of interest; wherein,

The detection feature vector includes the latest access interval feature last_interval used to represent the time interval from the latest access to the current connection to the time of detection, the traffic feature traffic used to represent the average flow of the current connection within the detection time interval, and used to represent The periodic behavior of the current connection in the time interval of the most recent visits. period, which is used to indicate the same sip connection with the same sip as the current connection and the number of connections that have successively accessed within the detection time interval. The successive access feature sip_count, the connection feature dip_count of the same dip used to indicate the number of connections with the same dip as the current connection within the detection time interval;

The detection feature value calculation module calculates the value of each detection feature in the detection feature vector corresponding to the connection related to the target of interest according to the statistical results;

The abnormality judging module judges whether the corresponding connection is abnormal according to the value of the detection feature.

The advantages of the present invention are:

1. The present invention does not need to analyze and extract fixed features, and can promptly discover unknown network theft.

2. The present invention does not need to process the data load of the network data packet, improves the detection efficiency, and is suitable for application deployment in a large-scale network environment.

Description of drawings

FIG. 1 is a flow chart of the abnormal detection method for network theft of secrets according to the present invention.

Detailed ways

The present invention will be described below in conjunction with the accompanying drawings and specific embodiments.

Through practical experience and theoretical analysis, it can be known that network theft generally follows the following necessary processes in its life cycle:

1) Invading the network information system by means of implanting, disseminating or network intrusion of malicious code stealing;

2) Collect confidential information;

3) Return the confidential information.

The present invention aims at the above-mentioned characteristics of the network stealing behavior process, based on the application requirements of network detection, focusing on the necessary step of "returning secret-related information" in the network stealing behavior process, and extracting its key behavior through in-depth analysis of its behavior characteristics Features, by comparing with the learned normal behavior profile of the network to find its abnormal behavior, so as to detect suspicious stealing behavior events.

Next, with reference to FIG. 1 , how the present invention realizes the detection of the event of loss of confidentiality or theft of confidentiality will be described.

To realize the method of the present invention, some detection parameters involved in the realization of the method of the present invention must be initialized first. For example, set the timer T to 0, set the detection interval to T ₀ , set the IP address of the target concerned, set the division rules of UDP connections, and set the parameters required by the detection method, such as threshold-based anomaly detection. The threshold of the required detection features and the learning time, etc. The above-mentioned detection parameters will be involved in the following description, therefore, specific values of the detection parameters will be described below.

After setting the initial values of the detection parameters through the initialization operation, it is also necessary to obtain the data related to the detection. In the process of obtaining data related to detection, it is first necessary to receive network data packets on the Internet, and perform protocol restoration on the received network data packets, so as to obtain the information of the network layer header and the transport layer header of the network data packets. In this embodiment, the network data packets are IP packets, specifically including TCP packets and UDP packets. The protocol restoration is TCP/IP protocol restoration, and the information that can be obtained from the network layer header and the transport layer header includes source IP address, destination IP address, protocol, packet length, TCP flag information, etc. The above-mentioned operations of receiving network data packets and performing protocol restoration are all existing technologies known to those skilled in the art, so repeated descriptions are not repeated here.

It has been mentioned in the foregoing description that the method of the present invention is to detect whether a secret theft event has occurred in the target concerned. Therefore, it is only necessary to pay attention to the network data packets related to the concerned target during the detection process. However, in the aforementioned process of receiving network data packets, not all received network data packets are related to the concerned target. Based on the above reasons, it is necessary to make a correlation determination on the received network data packets. The correlation determination process includes: matching the source IP address or the destination IP address of the received network data packet with the IP addresses of all concerned targets set in the initialization process, if the source IP address and the destination IP address Once it matches the IP address of the target of interest, it is considered that there is a correlation between the network data packet and the target of interest. Otherwise, there is no correlation. Since there are usually multiple IP addresses of the target concerned, in order to improve the matching efficiency, a matching method based on a hash table in the prior art may be used to achieve efficient matching of IP addresses.

After the network data packets are determined to be relevant, the network data packets that are not relevant are discarded. For related network data packets, the connection to which these network data packets belong is further determined. As mentioned above, the network data packets in this embodiment include TCP packets and UDP packets, therefore, the connections corresponding to these network data packets include TCP connections and UDP connections. Whether it is a TCP connection or a UDP connection, a 4-tuple <sip, dip, protocol, direction> can be used to describe it uniformly. In this 4-tuple, sip indicates the IP address of the connection initiator, dip indicates the IP address of the connection receiver, protocol indicates the protocol type of the current connection, specifically including TCP and UDP, and direction indicates the direction of the connection to the target of interest , including in and out, in means the incoming connection, and out means the outgoing connection. Among them, sip, dip and protocol are unique identifiers of a connection. In the correlation determination mentioned above, if the source IP address of the network data packet matches the target of interest, the value of direction is out; if the destination IP address of the network data packet matches the target of interest, the value of direction is in.

Although both the TCP connection and the UDP connection can be uniformly described by the above 4-tuple, there are obvious differences in the process of determining the TCP connection from the TCP packet and the UDP connection from the UDP packet. Therefore, they are described separately below.

1. When the network data packet investigated is a TCP synchronous packet (that is, a TCP packet whose flag is Syn), if the data packet does not belong to an existing connection, a new TCP connection is established for it. The source IP address of the data packet is the IP address sip of the connection initiator in the TCP connection 4-tuple, the destination IP address of the data packet is the IP address dip of the connection receiver in the 4-tuple, and the protocol protocol of the connection is TCP. The implementation of establishing a TCP connection by using the TCP synchronization packet belongs to known technologies, and specific implementation details may refer to prior art documents. If the data packet belongs to an existing connection, it indicates that a new access to the current existing connection is to be started.

2. When the network data packet under investigation is other TCP packets of non-synchronous packets, if the data packet does not belong to an existing connection, it will be discarded; if the data packet belongs to an existing connection, it will be kept for subsequent statistics deal with.

3. When the network data packet under investigation is a UDP packet, if the data packet does not belong to an existing connection, a new UDP connection is established for it. The source IP address of the data packet is the IP address sip of the connection initiator in the aforementioned 4-tuple, the destination IP address of the data packet is the IP address dip of the connection receiver in the aforementioned 4-tuple, and the protocol protocol of the connection is UDP. If the data packet belongs to an existing connection, it is judged whether the connection satisfies the pre-defined division rules of UDP connections. If it is satisfied, a new visit of the current connection is started, and then subsequent statistical processing is performed. If it is not satisfied, it can be directly Do follow-up statistical processing. In this embodiment, the division rule of the UDP connection can be defined as: (1) for the UDP packet whose destination port is 53, each UDP packet is a UDP connection; (2) for all other UDP packets, set the UDP connection Timer and division threshold, the division threshold described in an example can be set to 10 seconds, the timer is set to 0 when the connection is established or each access starts, and when processing each UDP packet, it is judged whether the timer exceeds the preset If the set division threshold is exceeded, the connection division will be performed, a new access of the current connection will be started, and its timer will be reset to 0. If it does not exceed, no division is required.

In the above-mentioned determination process, the method for judging whether a data packet belongs to a connection is as follows: if the source IP address, destination IP address, and protocol of the data packet under investigation are the same as the sip, dip and protocol of an existing connection respectively, then the The packet belongs to this connection, if not the same, it does not belong to this connection.

After the connection to which it belongs is obtained from the network data packet, the detection feature vectors on each connection can be counted. The detection feature vector includes a plurality of detection features, and the detection features reflect the network traffic characteristics and/or behavior characteristics presented when a network theft occurs. By detecting the abnormality of the characteristics, it can be found whether there is network theft. The selection of detection features is not only related to the type of anomaly to be detected, but also related to the accuracy and efficiency of anomaly detection. In one embodiment, for the characteristics of network theft, the present invention designs a detection feature vector <last_interval, traffic, period, sip_count, dip_count> consisting of 5 detection features for each connection, the detection feature vector The meanings of each detection feature in are as follows:

a. The latest access interval feature last_interval: Indicates the time interval from the last access time of the current connection to the detection time, in seconds. When the theft occurs, the IP address of the stealer is often not the set of IP addresses that the stealer usually visits habitually, so there may be IP addresses that have never been visited in the access IP list of the stealer. On the other hand, in order to achieve concealment The purpose of stealing secrets often shows the characteristics of low frequency of visits. A connection that satisfies the above characteristics has a larger value of the latest access interval characteristic, so the possibility of its theft is greater. Therefore, the recent access interval feature is selected as a detection feature of the theft.

b. Traffic characteristic traffic: indicates the average traffic of the current connection within the detection interval T ₀ , and the unit is bps (bits per second). Stealing behavior often has the characteristics of downloading a large amount of secret-related information in a short period of time in the process of stealing secrets, so as to achieve the purpose of stealing secret-related information as soon as possible. Therefore, traffic characteristics are selected as a detection feature of secret-stealing behavior.

c. Behavior periodic feature period: It indicates the periodicity of the time interval between the last n accesses of the current connection until the detection, where n is the threshold, which should be a multiple of 3, and can usually be set to 6, 9 or 12. The magnitude of the periodicity is a fuzzy number in the range 0 to 1. Application practice shows that the vast majority of stealing behaviors are generated by malicious codes such as stealing Trojan horses or zombie programs. As a kind of computer program, these malicious codes often produce certain periodic and regular access behaviors during the stealing process, so , select the periodic feature of behavior as a detection feature of stealing secret behavior.

d. Sequential access feature sip_count of the same sip connection: Indicates the number of connections that have the same sip as the current connection within the detection time interval T ₀ and accesses occur successively. Malicious codes such as secret-stealing Trojans or bots usually access multiple IP addresses in sequence to achieve domain name resolution, obtain the IP address of the stealing control server, or obtain stealing commands. Therefore, the sequential access feature of the same sip connection is selected as a detection feature.

e. Connection feature dip_count of the same dip: indicates the number of connections with the same dip as the current connection within the detection time interval T ₀ . When malicious codes such as secret-stealing Trojans or bots communicate with the control server, the number of connections with the same dip (IP address of the control server) will be significantly different from the normal behavior. Therefore, the connection feature of the same dip is selected as a detection feature.

When counting the above-mentioned detection features in the detection feature vector, there are different statistical methods, including:

a. For the last_interval feature: use the variable last_time to record the time when the current connection was established or the time when the last access occurred. When a connection is established, let last_time be 0, and when a new access occurs, let last_time be the current time.

b. For traffic characteristics: use the variable sum to count the sum of the lengths of all data packets of the current connection within the detection interval T ₀ , that is, the total number of bytes of the current connection.

c. For the period feature: use a circular queue with a length of n to record the time interval of the latest n accesses of the current connection.

d. For the sip_count feature: statistics on the sip_count feature can be realized based on the published Counting Bloom Filter (CBF) technology. In addition to the above-mentioned prior art, the following statistical algorithms can also be used in this embodiment:

Let start_time be the time when the current connection is established or access occurs, and flag is the flag indicating whether the current connection has been counted. The statistical elements of CBF are the counter count and the end time end_time, both of which are initialized to 0. When the current connection access ends, that is, when the TCP end packet (that is, the TCP packet whose flag is Fin) is received or the division rule of the UDP connection mentioned above is satisfied, if the flag is equal to 1, return, otherwise, find the current connection through CBF technology For the statistical element corresponding to the connected sip, if start_time is greater than end_time, then set flag=1, count+1, end_time=current access end time, otherwise, if start_time is less than or equal to end_time, then return.

e. For the dip_count feature: when a connection is established or a new visit occurs, the disclosed Counting Bloom Filter technology can be used to count the dips of the current connection.

The results obtained through the above processes of receiving network data packets, determining the connection from the network data packets, and counting the detection characteristics of each connection are data related to detection. In a preferred implementation, in order to improve the final effect of anomaly detection, the above-mentioned receiving network data packets, determining the connection to which the network data packets belong, and counting the detection characteristics of each connection can be performed repeatedly within a relatively long period of time process to obtain richer statistical results. This repeated execution process is also called a learning process, and the time taken by the learning process is determined by the learning time set during the aforementioned initialization. Generally speaking, the longer the learning time, the better the final detection effect. For example, in an example, it is found through experiments that when the learning time is set to 24×7 hours, the final detection effect is better. In practice, it is necessary to achieve a good balance between the learning time and the detection effect, and the learning time can generally be set to 24 to 48 hours. In addition, although the existence of the learning process is conducive to improving the final effect of anomaly detection, in other embodiments, the learning process can also be omitted in the entire anomaly detection process, and only the detection results of the anomaly detection process The false positive rate will increase.

After receiving the network data packet, determining the connection according to the network data packet, and counting the detection characteristics of each connection, anomaly detection can be performed on each connection related to the target of interest. Taking one of the connections as an example, the value of each detection feature in the detection feature vector corresponding to the connection is firstly calculated from the statistical results obtained above. The calculation method is as follows:

a. last_interval=now_time-last_time, where now_time is the current time.

b. traffic=sum×8/T ₀ .

c. The calculation of the period can be realized by using related methods in the prior art, such as a self-similarity determination method and the like. In one embodiment, the following calculation method is used:

The statistical results related to the period feature include a circular queue with a length of n, assuming that the elements in the queue are q _j (j=1~n), then $\mathrm{period}=\mathrm{Max}\left\{p\left(m\right)\right|m=1~\frac{\mathrm{no}}{3}\},$ Among them, Max means to take the maximum value in the set, m means the grouping length (that is, when the queue elements are divided into several groups, the number of elements contained in each group), p(m) means when the grouping length is m, the average number of grouping elements The uniformity presented. The calculation formula of p(m) is as follows:

为分组的个数， $\stackrel{\&OverBar;}{q}=\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}{q}_j$ 为队列元素的平均数， $p_i=\underset{j=m\&times;(i-1)+1}{\overset{\mathrm{Min}(m\&times;i,n)}{\mathrm{\&Sigma;}}}{q}_j/m$ 为第i分组元素的平均数。Among them, Min means to take the minimum value in the set,

is the number of groups, $\stackrel{\&OverBar;}{q}=\underset{j=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{q}_j$ is the average number of queue elements, $p_i=\underset{j=m\&times;(i-1)+1}{\overset{\mathrm{Min}(m\&times;i,\mathrm{no})}{\mathrm{\&Sigma;}}}{q}_j/m$ is the average number of elements in the i-th group.

The time complexity of this calculation method is O(n ² ). In order to improve processing efficiency, in practice, only the case of m=1 can be considered, and the time complexity at this time is O(n).

d. sip_count is the count value of the statistical element in the corresponding CBF.

e. dip_count is the count value of the statistical element in the corresponding CBF.

After obtaining the values of each detection characteristic of the connection, it is possible to detect whether there is an abnormality in the connection. There are many ways to implement anomaly detection, such as threshold-based detection methods, model-based detection methods (such as Bayesian network-based) and so on. In the following, the threshold-based detection method and the model-based detection method are taken as examples to illustrate how to implement anomaly detection.

(1) In an embodiment based on the threshold value detection method, it is first judged whether there is an abnormality in the detection feature, and then it is judged whether there is an abnormality in the entire connection according to the abnormality judgment result of the detection feature. Detection thresholds can be set for each detection feature (each detection threshold can be an individual threshold, or a threshold range or a set of thresholds), and then the detection feature is compared with the corresponding detection threshold, if a detection feature is greater than If it is less than or belongs to or deviates from the corresponding detection threshold of the detection feature, it is considered that the feature is abnormal. The detection threshold can be set respectively for each detection feature according to practical experience and application environment. For example, in an example, the detection thresholds of last_interval, traffic, period, sip_count, and dip_count can be respectively set to 86400 seconds (24 hours), 300k bps, 0.95, [3, 5], 128, when last_interval, traffic, period, and dip count are greater than the corresponding detection threshold, and sip_count does not belong to the corresponding threshold range, the corresponding detection feature is considered abnormal. In addition, the detection threshold can also be set by pre-training and learning the network data of the application environment by using disclosed technologies and methods.

After obtaining the abnormality judgment results of each detection feature constituting the detection feature vector, a further judgment can be made on whether the entire connection is abnormal. In one embodiment, the following judgment criteria are set: 1) if the last_interval and traffic features are abnormal; 2) if the last_interval and period features are abnormal; 3) if the sip_count feature is abnormal; 4) if the last_interval and dip_count features are abnormal. As long as any of the above rules are met, the connection is considered to be misbehaving. Among them, sip is the stealer, and dip is the stealer. If the direction of the current connection is in, the IP address of the target of interest is suspected to be the stealer. If the direction is out, the IP address of the target of interest is suspected to be the stealer. If there is no abnormal detection feature in the connection, or if there is an abnormal detection feature but does not meet the criteria for connection abnormality, then it is considered that the current connection does not have abnormal behavior.

(2) In an embodiment of a model-based detection method, the anomaly detection process is described by taking the disclosed Bayesian network-based method as an example.

First, a training sample set with event conclusions is compiled based on the detection features, and an example of the training sample set is given in Table 1 below.

last_interval traffic period sip_count dip_count conclusion 10000 seconds 100k bps 0.8 1 53 normal behavior 50000 seconds 89k bps 0.95 3 twenty three abnormal behavior ... ... ... ... ... ...

Table 1

After obtaining the training sample set, the training sample set is input into the corresponding model, so as to calculate the parameters of the model through training. After the model training is completed, the theft can be detected. During detection, collect, restore, and count the network data as described above to generate detection features, and then input the detection features into the trained model. The model can usually calculate a probability value, such as 0.5, 0.9, etc., with a large A connection with a probability value (eg, exceeding a predetermined threshold) is generally considered to be anomalous.

In addition to the above-mentioned methods based on Bayesian networks, other model-based detection methods can also be used, such as detection methods based on credibility models, detection methods based on direct push reliability machines, and detection methods based on K-nearest neighbors. , detection method based on support vector machine and so on.

After the conclusion of whether there is an abnormality in the connection is obtained, follow-up processing can be performed on the conclusion. If there is an abnormality in a certain connection, an alarm event of stealing secrets will be generated and written into the event library. If there is no abnormality in a certain connection, continue to perform abnormality detection on the next connection. After the above-mentioned anomaly detection is performed on all connections related to the target of interest, the variable sum of the detection feature traffic corresponding to all connections should be set to zero, and the traffic statistics should be restarted, and the timer T should be set to 0.

The present invention also provides an abnormality detection system for network theft, including a data packet recovery module, a correlation judgment module, a connection judgment module, a detection feature vector statistics module, a detection feature value calculation module, and an abnormality judgment module; wherein,

The data packet restoration module receives the network data packet, and performs protocol restoration on the obtained network data packet;

The correlation judgment module makes a correlation judgment between the network data packet and the target of interest, discards the network data packets without correlation, and sends the relevant network data packets to the connection judgment module;

The connection determination module determines the connection to which the network data packet related to the concerned target belongs; wherein, the connection includes a TCP connection or a UDP connection;

The detection feature vector statistics module performs statistics on the detection feature vectors on all connections related to the target of interest; wherein,

The detection feature vector includes the latest access interval feature last_interval used to represent the time interval from the latest access to the current connection to the time of detection, the traffic feature traffic used to represent the average flow of the current connection within the detection time interval, and used to represent The periodic behavior of the current connection in the time interval of the most recent visits. period, which is used to indicate the same sip connection with the same sip as the current connection and the number of connections that have successively accessed within the detection time interval. The successive access feature sip_count, the connection feature dip_count of the same dip used to indicate the number of connections with the same dip as the current connection within the detection time interval;

The detection feature value calculation module calculates the value of each detection feature in the detection feature vector corresponding to the connection related to the target of interest according to the statistical results;

The abnormality judging module judges whether the corresponding connection is abnormal according to the value of the detection feature. The anomaly detection method of the present invention does not need to analyze and extract fixed features, and can discover unknown network theft behaviors in time.

The anomaly detection method of the present invention adopts the access behavior of the key step of "returning secret-related information" as the detection feature of the present invention, does not need to process the data load of the network data packet, improves the detection efficiency, and is suitable for large-scale network environments. Application deployment. The invention also provides effective information support and technical means for further tracking and defense of unknown network theft.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than limit them. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent replacements to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all of them should be included in the scope of the present invention. within the scope of the claims.

Claims (13)

1. An anomaly detection method for network theft, comprising: Step 1), receiving the network data packet, and performing protocol restoration on the obtained network data packet; Step 2), making a correlation judgment between the network data packet and the target of concern, discarding the network data packet without correlation, and performing the next step for the network data packet with correlation; Step 3), determine the connection to which the network data packet related to the concerned target belongs; wherein, the connection includes a TCP connection or a UDP connection; Step 4), making statistics on the detection feature vectors on all connections related to the target of interest; wherein, The detection feature vector includes the latest access interval feature last_interval used to represent the time interval from the latest access to the current connection to the time of detection, the traffic feature traffic used to represent the average flow of the current connection within the detection time interval, and used to represent The periodic behavior of the current connection in the time interval of the most recent visits. period, which is used to indicate the same sip connection with the same sip as the current connection and the number of connections that have successively accessed within the detection time interval. The successive access feature sip_count, the connection feature dip_count of the same dip used to indicate the number of connections with the same dip as the current connection within the detection time interval; Step 5), according to the statistical results, calculate the value of each detection feature in the detection feature vector corresponding to the connection related to the target of interest; Step 6), judging whether the corresponding connection is abnormal according to the value of the detection feature. 2. The abnormal detection method of network theft secret behavior according to claim 1, is characterized in that, before described step 1), also comprises learning phase, and described learning phase comprises step 1)-step 4), to obtain A large number of statistical results of the detected feature vectors on all connections related to the target of interest. 3. The abnormal detection method of network theft according to claim 1 or 2, characterized in that, in said step 2), said correlation judgment includes: Matching the source IP address or the destination IP address of the network data packet with the IP address of the target of interest, if one of the source IP address and the destination IP address matches the IP address of the target of concern, Then there is a correlation between the network data packet and the target of interest; otherwise, there is no correlation. 4. The abnormal detection method for network theft according to claim 1 or 2, wherein said step 3) comprises: Step 3-1), when the network data packet related to the concerned target is a TCP synchronization packet, if the network data packet does not belong to an existing connection, then a new TCP connection is established for the network data packet, if the The network packet belongs to an existing connection, and a new access to the existing connection is started; Step 3-2), when the network data packet related to the concerned target is a TCP asynchronous packet, if the network data packet does not belong to an existing connection, then discard the network data packet, if the network data packet belongs to a If there is already a connection, keep the network data packet; Step 3-3), when the network data packet related to the concerned target is a UDP packet, if the network data packet does not belong to an existing connection, a new UDP connection is established for it, if the network data packet belongs to An existing connection, when the existing connection meets the division rules of UDP connections, start a new access to the current existing connection, and then do subsequent statistical processing; when the existing connection does not meet the division rules of UDP connections, Do the follow-up statistical processing directly. 5. The abnormal detection method for network theft according to claim 4, wherein the rules for dividing UDP connections include: (1), for the UDP packet whose destination port is 53, each UDP packet is a UDP connection; (2) For all other UDP packets, set the UDP connection timer and division threshold, set the timer to 0 when the connection is established or each access starts, and determine whether the timer exceeds the preset value when processing each UDP packet If the division threshold is exceeded, the connection division will be performed, a new access of the current connection will be started, and its timer will be reset to 0. If it does not exceed, no division is required. 6. The abnormal detection method for network theft according to claim 1 or 2, characterized in that, in step 4), The statistics of the latest access interval feature last_interval include: use the variable last_time to record the time when the current connection is established or the time when the last visit occurred; wherein, when the connection is established, let last_time be 0, and when a new visit occurs, let last_time is the current time; The statistics of the traffic characteristic traffic include: using the variable sum to count the sum of all data packet lengths currently connected in the detection interval T ₀ ; The statistics of the periodic feature period of the behavior include: using a circular queue with a length of n to record the time interval of the latest n accesses of the current connection; The statistics of the successive access feature sip_count to the same sip connection include: adopt the CountingBloom Filter method to count the sip_count feature; or, Let start_time be the time when the current connection is established or access occurs, and flag is the flag indicating whether the current connection has been counted. The statistical elements of the Counting Bloom Filter method include the counter count and the end time end_time, both of which are initialized to 0; when the current connection access ends, If flag is equal to 1, then return, otherwise find the statistical element corresponding to the currently connected sip through the Counting Bloom Filter method, if start_time is greater than end_time, then set flag=1, count+1, end_time=current access end time, otherwise if If start_time is less than or equal to end_time, return; The statistics of the connection feature dip_count of the same dip includes: using the Counting BloomFilter method to count the current connected dips. 7. The abnormal detection method of network theft according to claim 6, characterized in that, in said step 5), the calculation of the detection feature value includes: The value of the latest access interval feature last_interval is the difference between the current time and the variable last_time; The value of the traffic characteristic traffic is the product of the variable sum and 8 and the quotient of the detection interval _T0 ; The calculation formula of the value of the periodic characteristic period of the behavior is: $\mathrm{<span\; class="notranslate">\; period</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ~</span>\frac{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; ;</span>$ Wherein, n is the length of the circular queue; Max represents the maximum value in the set; m represents the packet length; p (m) represents the uniformity that the average number of each packet element presents when the packet length is m, its size Calculated using the following formula:

Among them, Min means to take the minimum value in the set,

is the number of groups, $\stackrel{\&OverBar;}{q}=\underset{j=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{q}_j$ is the average number of queue elements, $p_i=\underset{j=m\&times;(i-1)+1}{\overset{\mathrm{Min}(m\&times;i,\mathrm{no})}{\mathrm{\&Sigma;}}}{q}_j/m$ is the average number of i grouping elements; The value of the successive access feature sip_count of the same sip connection is the count value of the statistical element in the Counting Bloom Filter method; The value of the connection feature dip_countsip_count of the same dip is the count value of the statistical element in the Counting BloomFilter method. 8. The abnormal detection method for network theft according to claim 1 or 2, wherein said step 6) includes: Step 6-1), judging whether there is an abnormality in the detection feature; In step 6-2), it is judged whether there is an abnormality in the corresponding connection according to the abnormality judgment result of the detection feature. 9. The abnormal detection method for network theft according to claim 8, characterized in that, in the step 6-1), the method of comparing the detection feature with the corresponding detection threshold or model judgment is adopted The method judges whether there is an abnormality in the detection feature. 10. The abnormal detection method for network theft according to claim 8, characterized in that, in said step 6-2), the method of setting judgment criteria or the method of model judgment is used to check whether the connection exists Exceptions are judged. 11. The abnormality detection method for network theft according to claim 9, characterized in that the judgment criteria include: if the latest access interval feature last_interval and traffic feature traffic are abnormal, or if the latest access interval feature last_interval and behavior If the periodic characteristic period is abnormal, or the consecutive access characteristic sip_count of the same sip connection is abnormal, or the latest access interval characteristic last_interval and the connection characteristic of the same dip are abnormal at the same time, then the connection will behave abnormally. 12. The abnormal detection method for network theft according to claim 8, characterized in that, in said step 6-2), further comprising: Extract the secret information and the secret information; wherein, sip is a secret stealer, dip is a secret stealer, if the direction of the current connection is in, then the IP address of the concerned target is suspected of being a secret stealer, if the direction of the current connection is out, then the The IP address of the target of concern is suspected to be a leaker. 13. An anomaly detection system for network theft, including a data packet recovery module, a correlation judgment module, a connection judgment module, a detection feature vector statistics module, a detection feature value calculation module, and an abnormality judgment module; wherein, The data packet restoration module receives the network data packet, and performs protocol restoration on the obtained network data packet; The correlation judgment module makes a correlation judgment between the network data packet and the target of interest, discards the network data packets without correlation, and sends the relevant network data packets to the connection judgment module; The connection determination module determines the connection to which the network data packet related to the concerned target belongs; wherein, the connection includes a TCP connection or a UDP connection; The detection feature vector statistics module performs statistics on the detection feature vectors on all connections related to the target of interest; wherein, The detection feature vector includes the latest access interval feature last_interval used to represent the time interval from the latest access to the current connection to the time of detection, the traffic feature traffic used to represent the average flow of the current connection within the detection time interval, and used to represent The periodic behavior of the current connection in the time interval of the most recent visits. period, which is used to indicate the same sip connection with the same sip as the current connection and the number of connections that have successively accessed within the detection time interval. The successive access feature sip_count, the connection feature dip_count of the same dip used to indicate the number of connections with the same dip as the current connection within the detection time interval; The detection feature value calculation module calculates the value of each detection feature in the detection feature vector corresponding to the connection related to the target of interest according to the statistical results; The abnormality judging module judges whether the corresponding connection is abnormal according to the value of the detection feature.

Images