CN115550031A - Network defense method, control device and storage medium - Google Patents

Abstract

This application provides a network defense method, a control device and a storage medium. After the control device establishes an attack-defense game model, the control device repeatedly executes to obtain the attack strategy information of the attacking terminal attacking the target working terminal observed by the control device and the a priori type of the identity type of the attacking terminal. , after this operation, according to the attack strategy information and prior probability, determine the optimal defense strategy and the optimal attack strategy for the target working terminal, and execute the optimal defense strategy, and control the device to calculate according to the optimal defense strategy and attack strategy information Markov learning function value, until according to the Markov learning function value, stop the cycle when it is determined that the target working terminal is not in a safe state, the control device determines the attacking state of the target working terminal according to the persistent attack strategy information obtained by the target working terminal, It helps it to determine concealed attacks based on continuously obtained information, so that it can better protect the target work terminal.

Description

Network defense method, control device and storage medium

technical field

The embodiments of the present application relate to the technical field of network security, and in particular to a network defense method, a control device, and a storage medium.

Background technique

While entering the era of the Internet of Everything, the security issues of various terminal devices are becoming increasingly serious. The existing network defense architecture based on passive scanning intrusion detection is not enough for the current network attacks. The existing method extracts the effective relationship path from the interactive relationship between the items to be detected (such as: IP address, interface call, program execution, etc.) in the data to be detected by the security network device, and performs anomaly detection on the effective relationship path to obtain the target host Threat detection results, wherein the effective relationship path includes at least two detection items and the interaction relationship between each detection item. However, it is difficult for this network security device to identify unknown types of network attacks with high concealment, large scale and intelligence.

Contents of the invention

The present application provides a network defense method, control equipment and storage media, which are used to solve the technical problem that the hidden and persistent attack operations of security network equipment are not easy to be detected, and timely defense cannot be realized.

In a first aspect, the present application provides a network defense method, the method is applied to a control device, the control device is located in a target system, the target system includes at least one working terminal and a honeynet cluster, and the method includes:

An attack-defense game model is established; wherein, the attack-defense game model includes a plurality of game participants, the strategy space of each of the game participants, the signal space, the prior probability, the posterior probability and the income of each of the game participants;

Obtaining the attack strategy information of the attacking terminal attacking the target working terminal and the prior probability of the identity type of the attacking terminal observed by the control device repeatedly;

determining an optimal defense strategy and an optimal attack strategy of the attacking terminal for the optimal defense strategy according to the attack strategy information and the prior probability, and executing the optimal defense strategy;

calculating a Markov learning function value according to the optimal defense strategy and the attack strategy information;

Based on the Markov learning function value, it is determined whether the target working terminal is in a safe state and a loop is terminated when the target working terminal is not in a safe state.

In the above technical solution, after the control device establishes the attack-defense game model, it obtains the prior probability and the attack strategy information observed by the control device in at least one cycle, and determines the control device according to the prior probability and attack strategy information in each cycle. The optimal defense strategy for the attack terminal and the optimal attack strategy for the attack terminal against the optimal defense strategy, and use the Markov learning function to determine whether the target working terminal is dangerous under the influence of this attack strategy, so that the control device can determine When the target working terminal is not in a safe state, the cycle is terminated, so that the control device can monitor the multiple attacks performed by the attack terminal, and count the damage caused by multiple attacks to the target system, and detect the concealed and persistent attack operations, thereby When the control device determines that the target working terminal is not in a safe state, it can protect the target working terminal in time, effectively ensuring the safety of the target working terminal.

Optionally, determining an optimal defense strategy and an optimal attack strategy for the target working terminal according to the attack strategy information and the prior probability, specifically including:

calculating the posterior probability of the identity type of the attacking terminal according to the attack strategy information and the prior probability;

Determine the identity type of the attacking terminal according to the posterior probability;

determining system cost and type cost according to the attack strategy information and the identity type;

calculating the attacker payoff of the attacking terminal and the defender payoff of the target system according to the system cost, the type cost, the attack strategy information, and the posterior probability;

An optimal defense strategy and an optimal attack strategy for the target working terminal are determined according to the posterior probability and the defender's income.

Optionally, according to the attack policy information and the identity type, determine system cost and type cost, specifically including:

Querying the attack strategy information in a system cost mapping table to obtain the system cost corresponding to the attack strategy information;

The attack policy information and the identity type are queried in a type cost mapping table to obtain the type cost corresponding to the attack policy information.

Optionally, the strategy space of the game participants includes a defense strategy space, and the defense strategy space includes at least one defense strategy; according to the optimal defense strategy and the attack strategy information, the Markov learning function value is calculated , including:

calculating the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy and the attack strategy information;

Obtaining the cumulative return value of the target working terminal from the initial state to the current state and the Markov learning function value of the target working terminal in the current state;

According to the optimal attack strategy of the attacking terminal for the optimal defense strategy, all the defense strategies, the cumulative reward value and the attack-defense reward value, calculate the maximum cumulative reward value of the target working terminal in the next state;

According to the attack and defense return value in the current state, the maximum cumulative return value in the next state and the Markov learning function value in the current state, calculate the next state of the target working terminal in the current state The Markov learning function value of .

Optionally, the strategy space of the game participants further includes an attack strategy space, and the attack strategy space contains at least one attack strategy; according to the optimal defense strategy and the attack strategy information, calculate the target working terminal The attack and defense return value in the current state, including:

Calculate the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy, the attack strategy information and the attack and defense return value calculation formula:

Wherein, the calculation formula of the offensive and defensive return value specifically includes:

Wherein, sp represents the _pth state after the current state of the target working terminal, Re(f _x (sp )) represents the attack and defense return value of the target working terminal x in state _p , and i represents the The defense identifier in the defense strategy space, j represents the attack identifier in the attack strategy space, i=0 means that the target working terminal x is not defended in state p, i≠0 means that the target working terminal x is in state p In state p, it is defended by the i-th defense strategy, j=0 means that the target working terminal x is not attacked in state p, and j≠0 means that the target working terminal x is in state p by the j-th attack strategy , P represents the protection value, D _i represents the reward value when the target working terminal x is protected by the i-th defense strategy, δ _i represents the reward coefficient, D represents the target working terminal x is protected by the defense strategy The attack return value of the attack, Pk _ij represents the return value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

Optionally, calculating the maximum cumulative reward value of the target working terminal in the next state according to the optimal attack strategy of the target working terminal, all the defense strategies, the cumulative reward value, and the offensive and defensive reward value, Specifically include:

According to the optimal attack strategy of the attack terminal for the optimal defense strategy, each of the defense strategies and the attack and defense return value formula, calculate the attack and defense return value in the next state of the target working terminal adjusted based on each of the defense strategies ;

adding each of the offensive and defensive return values to the cumulative return value, and calculating the cumulative return value in the next state of the target working terminal adjusted based on each of the defense strategies;

Determining the maximum value among the cumulative return values in the next state corresponding to each defense strategy as the maximum cumulative return value.

Optionally, according to the attack and defense return value in the current state, the maximum cumulative return value in the next state, and the Markov learning function value in the current state, calculate the target working terminal in the current state Markov learning function value in the next state, including:

According to the attack and defense return value in the current state, the maximum cumulative return value in the next state, the Markov learning function value in the current state and the learning function update formula, calculate the current state of the target working terminal The Markov learning function value in the next state of ; wherein, the learning function update formula specifically includes:

表示所述目标工作终端在当前状态的下一状态下的马尔科夫学习函数值，

表示所述目标工作终端在当前状态下的马尔科夫学习函数值，α表示学习率参数，γ表示折现率参数，

表示所述下一状态下的最大累计回报值；s₀表示所述目标工作终端的初始状态，Re(f_x(s₀))表示初始状态对应的攻防回报值；s₁表示所述目标工作终端的状态s₀的下一状态，Re(f_x(s₁))表示状态s₁对应的攻防回报值；s_p表示所述终端的当前状态，Re(f_x(s_p))表示当前状态对应的攻防回报值；s_p+1表示所述目标工作终端当前状态的下一状态，Re(f_x(s_p+1))表示下一状态对应的攻防回报值。in,

Indicates the Markov learning function value of the target working terminal in the next state of the current state,

Represents the Markov learning function value of the target working terminal in the current state, α represents the learning rate parameter, γ represents the discount rate parameter,

Represents the maximum cumulative return value under the next state; s ₀ represents the initial state of the target work terminal, Re(f _x (s ₀ )) represents the attack and defense return value corresponding to the initial state; s ₁ represents the target work terminal The next state of the terminal state s ₀ , Re(f _x (s ₁ )) represents the attack and defense return value corresponding to state s ₁ ; sp _p represents the current state of the terminal, Re(f _x (s _p )) represents the current The attack and defense return value corresponding to the state; _sp+1 represents the next state of the current state of the target working terminal, and Re(f _x (s _p+1 )) represents the attack and defense return value corresponding to the next state.

Optionally, determining whether the target working terminal is at a safe level according to the Markov learning function value specifically includes:

determining that the target working terminal is in the safe state when the Markov learning function value is less than or equal to a safe threshold;

When the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in the safety state.

In a second aspect, the present application provides a control device, including: a processor and a memory communicatively connected to the processor;

the memory stores computer instructions;

The processor is used to implement the network defense method involved in the first aspect when executing computer instructions.

In a third aspect, the present application provides a computer-readable storage medium, wherein computer instructions are stored in the computer-readable storage medium, and the computer instructions are used to implement the network defense method involved in the first aspect when executed by a processor.

This application provides a network defense method, a control device and a storage medium. The control device establishes an attack-defense game model, and repeatedly executes to obtain the attack strategy information of the attacking terminal attacking the target working terminal observed by the control device and the a priori type of the identity type of the attacking terminal. After this operation, according to the attack strategy information and prior probability, determine the optimal defense strategy and the optimal attack strategy for the target working terminal, and execute the optimal defense strategy, and control the device to calculate the Mar The Kove learning function value, until according to the Markov learning function value, stop the cycle when it is determined that the target working terminal is not in a safe state, so that the control device determines the attack state of the target working terminal according to the persistent attack strategy information obtained by the target working terminal , which helps it determine concealed attacks based on continuously obtained information, and improves its ability to better protect target working terminals.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

FIG. 1 is an application scenario diagram of a network defense method provided by the present application according to an exemplary embodiment;

FIG. 2 is a schematic flowchart of a network defense method provided by the present application according to an exemplary embodiment;

Fig. 3 is a schematic flow diagram of the calculation process of the Markov learning function value provided by the present application according to an exemplary embodiment;

FIG. 4 is a schematic structural diagram of a network defense device provided by the present application according to an embodiment;

Fig. 5 is a schematic structural diagram of a control device provided by the present application according to an embodiment.

By means of the above drawings, specific embodiments of the present application have been shown, which will be described in more detail hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the application in any way, but to illustrate the concept of the application for those skilled in the art by referring to specific embodiments.

detailed description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

While entering the era of the Internet of Everything, the security issues of various terminal devices are becoming increasingly serious. The existing network defense architecture based on passive scanning intrusion detection is not enough for the current network attacks. The existing method extracts the effective relationship path from the interactive relationship between the items to be detected (such as: IP address, interface call, program execution, etc.) in the data to be detected by the security network device, and performs anomaly detection on the effective relationship path to obtain the target host Threat detection results, wherein the effective relationship path includes at least two detection items and the interaction relationship between each detection item. However, it is difficult for this network security device to identify unknown types of network attacks with high concealment, large scale and intelligence.

This application provides a network defense method, control equipment and storage media, aiming to solve the technical problem that the hidden and persistent attack operations of security network equipment are difficult to be detected and timely defense cannot be realized. The technical concept of this application is: the control device establishes an attack-defense game model, and repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the a priori type of the identity type of the attack terminal. After this operation, according to the attack strategy information and the prior probability, determine the optimal defense strategy and the optimal attack strategy for the target working terminal, and implement the optimal defense strategy, and control the device to calculate the Markov learning function value according to the optimal defense strategy and attack strategy information until the Markov learning function value, stop the cycle when it is determined that the target working terminal is not in a safe state, and forward the request of the working terminal to protect the target working terminal from being attacked again, so that the control device can obtain the continuous The attack strategy information determines the attack status of the target working terminal, which helps it determine concealed attacks based on continuously obtained information, so that it can better protect the target working terminal.

FIG. 1 is an application scenario diagram of a network defense method provided by the present application according to an exemplary embodiment. As shown in FIG. 1 , the application scenario diagram includes: attack terminal 106, routing device 105, first OpenFlow switch 102, second OpenFlow switch 103 , third OpenFlow switch 104 , control device 101 , intrusion detection device 108 , SDN controller 107 , work area 109 and honeynet cluster 111 . Wherein, the working area 109 includes a plurality of working terminals, and the honeynet cluster 111 includes a plurality of honeynet virtual terminals, and a virtual mapping system for the working terminals in the working area 109 is installed in the honeynet virtual terminals. The attack terminal 106 is connected to the routing device 105, the routing device 105 is connected to the first OpenFlow switch 102, and the first OpenFlow switch 102 is respectively connected to the second OpenFlow switch 103, the third OpenFlow switch 104, the control device 101 and the SDN controller 107, and the intrusion The detection device 108 is connected to the SDN controller 107 , the working area 109 is connected to the second OpenFlow switch 103 , and the honeynet cluster 111 is connected to the third OpenFlow switch 104 .

When the attack terminal 106 sends an access request to the target working terminal 110 in the work area 109, the attack terminal 106 sends the access request to the routing device 105, and the routing device 105 forwards the access request to the first OpenFlow switch 102, and the first OpenFlow switch 102 The access request is forwarded to the target work terminal 110 by the second OpenFlow switch 103, and at the same time, the first OpenFlow switch 102 is sent to the intrusion detection device 108 by the SDN controller 107, and an intrusion detection system (intrusion detection system) is installed in the intrusion detection device 108 , referred to as: IDS), the IDS identifies the above access request, and detects whether there is a risk of malicious attack on the target working terminal 110 .

If there is a risk, the detection result is fed back to the control device 101, and the control device 101 controls the first OpenFlow switch 102 to forward the access request sent again by the attack terminal 106 to the target honeycomb that loads the image of the target working terminal 110 in the honeynet cluster 111. network device 112, the target honeynet device 112 is hidden behind the firewall, and all incoming and outgoing data are monitored, captured and controlled; if there is no risk, and the target working terminal 110 determines that the access request belongs to an abnormal operation type, then the target working terminal 110 feeds back the determined result to the control device 101, and the control device 101 determines the corresponding defense strategy through the attack-defense game model, and determines whether the target working terminal 110 is safe when receiving the next access request according to the Markov learning function. More specifically, the control device 101 establishes an attack-defense game model, and obtains the attack strategy information in the access request observed by the control device 101, the prior probability of the identity type of the attacking terminal 106, the system cost and the type cost, wherein the attacking terminal 106's Identity types include User or Attacker. The control device 101 determines the optimal defense strategy and the optimal attack strategy of the attack terminal 106 for the optimal defense strategy according to the attack strategy information, prior probability, system cost, and type cost, and executes the optimal defense strategy. strategy and attack strategy information, and calculate the Markov learning function value of the target working terminal 110 in the current state, so as to determine whether the target working terminal 110 is in a safe state. When the target work terminal 110 is in a safe state, the control device 101 will still determine the defense strategy to protect the target work terminal 110 through the attack-defense game model when the target work terminal 110 obtains the access request again; when the target work terminal 110 is not in the safe state , then the control device 101 directly forwards the access request to the honeynet cluster 111 when the target working terminal 110 obtains the access request again, so as to protect the target working terminal 110 from being attacked by the attacking terminal 106 .

Based on the above application scenarios, some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. The following embodiments and features in the embodiments may be combined with each other under the condition that there is no conflict between the embodiments. In addition, the sequence of steps in the following method embodiments is only an example, rather than a strict limitation.

Fig. 2 is a schematic flowchart of a network defense method provided by the present application according to an exemplary embodiment. As shown in Fig. 2, the network defense method includes:

S201. The control device establishes an attack and defense game model.

The offensive and defensive game model is a model based on game theory. Game theory is based on two or more absolutely rational decision makers making decisions, by guessing the strategies of other decision makers, optimizing their own strategies to maximize their own interests.

The offensive and defensive game model includes multiple game participants, the strategy space of each game participant, signal space, prior probability, posterior probability and the income of each game participant. This model can be expressed as a six-tuple ADGMA={P, A, S, p, p _post , Reward}, where P={Actor, Defender} represents the two parties involved in the game, and refers to those who can make independent decisions and behaviors in the game The subject, in a network attack, usually refers to both the attacker and the defender. In defense against unknown threats, the defender cannot determine the identity of the user, and the identity information of the defender is shared information. In the model, participants include Actor and Defender, and Actor has two types Type={Attacker, User}. When there are multiple Actors in the game participant, the control device assigns a corresponding and unique identifier to each Actor. For example: Actor1, Actor2.

A＝{A _actor ＝(m ¹ ,m ² ,...,m ^x ),A _defender ＝(a ¹ ,a ² ,...,a ^y )} is the strategy space of each participant, which means that each participant can The set of specific actions and operations taken can be observed by all participants, and its type information is implied. When there are multiple Actors in a participant, each Actor corresponds to a strategy space.

S={s ₁ ,s ₂ ,...,s _n } represents the signal space, and the information about the true type conveyed by the Actor's behavior is called "signal", which can be used to guide the judgment and correction of its type.

p represents the prior probability, which refers to the initial judgment of the defender on the Actor type of the attacker as Attacker, that is, p(Type=Attacker)=θ, p(Type=User)=1-θ.

p _post represents the posterior probability, which refers to the posterior probability obtained by the defender after correcting the prior probability of Actor type judgment by using Bayesian rule according to the acquired signal.

Reward={R _A , R _D } represents the net income that each participant's behavior can bring, usually related to the behavior value and behavior cost, and is the fundamental basis for the participant to choose behavior.

After the control terminal establishes the offensive and defensive game model, it initializes the model. The prior probability is initialized to 0.5, the posterior probability is 0, and the net income of each participant is also 0.

S202. The control device repeatedly executes to obtain the attack strategy information of the attacking terminal attacking the target working terminal observed by the control device and the a priori probability of the identity type of the attacking terminal.

Wherein, the control device is the defender Defender in the attack-defense game model established in step S201, and the attack terminal is the attacker Actor in the attack-defense game model.

The prior probability of the identity type of the attack terminal is the prior probability of the identity type of the attack terminal determined by the control device before the target working terminal receives the attack operation of the attack terminal. If the target working terminal is attacked for the first time, the prior probability is the initial value, that is, 0.5; otherwise, the prior probability is the posterior probability of the identity type of the attacking terminal determined by the control device when the target working terminal was attacked last time .

After determining that the access request received by the target working terminal contains an abnormal operation, the target working terminal feeds back the abnormal status information of the access request to the control device. For network attacks, the attack terminal divides the access request with attack behavior into multiple steps to complete, and there is also a certain causal relationship between each step. Each step of behavior can become an atomic behavior e, and the atomic behavior is composed of time sequence and causal sequence Effective behavior m, then m=(e ₁ , e ₂ ,...,e _k ), where k is the number of atomic behaviors contained in the effective behavior m. When at least one atomic behavior in the effective behaviors received by the target working terminal is an abnormal behavior in the atomic abnormal behavior table, it is determined that the effective behavior is abnormal. In an embodiment, the atomic abnormal behavior table is shown in Table 1.

Table 1

After the control device determines that the target working terminal has received the attack operation, it updates the signal space of the attack-defense game model according to the attack strategy information it observes.

S203. The control device determines an optimal defense strategy and an optimal attack strategy for the target working terminal according to the attack strategy information and the prior probability, and executes the optimal defense strategy.

More specifically, the control device calculates the posterior probability of the identity type of the attacking terminal according to the attack strategy information and the prior probability, and then determines the identity type of the attacking terminal according to the posterior probability. The control device also calculates the attacker's income of the attacking terminal and the defender's income of the target system according to the attack strategy information and identity type, and determines the optimal defense strategy and the optimal attack on the target working terminal according to the posterior probability and the defender's income Strategy.

More specifically, the control device calculates the posterior probability of the attacking party being Attacker or User according to the Bayesian rule, and compares the above two posterior probabilities, and determines the type with the larger posterior probability value as the attacking party. identity type, and determine the posterior probability corresponding to this type as the posterior probability of the attacker. At the same time, the prior probability is updated in the attack-defense game model to make it the prior probability when the target working terminal receives an attack again.

The control device first determines the system cost and type cost according to the identity type of the attacker and the attack strategy information, wherein the control device queries the attack strategy information obtained by the target working terminal in the system cost mapping table to obtain the system cost corresponding to the attack strategy information, The system cost represents the impact on the income of other participants caused by changes in the state of the system after the attacker or the defender implements the behavior.

The control device also queries the attack strategy information and the identity type determined according to the posterior probability in the type cost mapping table to obtain the type cost corresponding to the attack strategy information, which represents the cost required for different types of participants to choose behaviors . Among them, the type cost of the participant Attacker is TC _Attacker ∈ [100, 300], TC _user ∈ [500, 2000].

More specifically, both the system cost mapping table and the type cost mapping table can be found in the MIT Lincoln Offensive and Defense Laboratory offensive and defensive behavior classification database, and will not be repeated here.

The control device then calculates the attacker's revenue of the attack terminal and the defender's revenue of the target system according to the system cost, type cost, attack strategy information and posterior probability.

More specifically, the attacker's policy information contains multiple atomic behaviors, and the control device queries each atomic behavior in the atomic abnormal behavior table involved in step S202, determines its own value and potential value corresponding to each atomic behavior, and uses the total The value calculation formula, own value, potential value and weight calculate the total value corresponding to each atomic behavior. Among them, the total value calculation formula is specifically expressed as:

r ₀ (e)=SV(e)+PV(e)×weight,

Among them, r ₀ (e) represents the total value of atomic behavior e, SV(e) represents the self-value of atomic behavior e, PV(e) represents the potential value of atomic behavior e, and weight represents the weight of potential value. Among them, SV(e) is related to the implementation difficulty, resource capability, and environmental requirements of the atomic behavior e. The lower the execution difficulty, the higher the resource capability, and the lower the environmental requirements, the higher its own value; the relationship between PV(e) and e The degree of publicity, defense maturity, generality and other factors are related. The lower the degree of publicity, the lower the defense maturity, and the higher the generality, the higher the potential value. For example, atomic behavior implemented by using zero-day vulnerabilities in commonly used office software , its potential attack value is very high; weight ∈ [0, 1] represents the degree to which the potential value of atomic behavior e plays a role, and is usually related to the level of atomic behavior e. The level of the atomic behavior represents the execution difficulty and performance level of the atomic attack (defense) behavior, which is determined according to the attack and defense classification data of Lincoln Laboratory.

After obtaining the total value of each atomic behavior, the control device adds the total values corresponding to all atomic behaviors included in the attack strategy information to obtain the effective behavior value corresponding to the attack strategy information. The control device subtracts the system cost and type cost from the effective behavior value, and calculates the attacker's benefit of attacking the terminal.

The control device obtains all defensive atomic behaviors that it can execute from the defensive behavior table, and constructs a defensive strategy information table for the attacker's strategy information according to all defensive atomic behaviors, wherein the number of defensive strategy information tables is more than one. The control device calculates the effective behavior value corresponding to each defense strategy information table, and determines the difference between the effective behavior value and the system cost as the defender's benefit corresponding to the defense strategy information table. In one embodiment, the defense behavior table is as shown in Table 2:

Table 2

The control device calculates the optimal defense strategy according to the defender value and the posterior probability corresponding to each defense strategy information table, and estimates the optimal attack strategy for the attacking terminal according to the optimal defense strategy. Among them, the optimal defense strategy and the optimal attack strategy are obtained by solving the game theory, and the solving process is an existing technology, and will not be repeated here.

S204. The control device calculates a Markov learning function value according to the optimal defense strategy and attack strategy information.

Wherein, the Markov learning function value is the cumulative learning function value calculated by using the Markov decision process from the initial state when the target working terminal is attacked for the first time to the current state.

S205. The control device determines whether the target working terminal is in a safe state according to the value of the Markov learning function.

When the Markov learning function value is less than or equal to the safety threshold, it is determined that the target working terminal is in a safe state, and when the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in a safe state. In one embodiment, the security threshold is 1000.

Wherein, the security threshold represents the threshold at which the target working terminal will not pose a serious threat to the target working terminal no matter what kind of attack policy information is received when the target working terminal is attacked next time.

When the target working terminal is in a safe state, go to step S202 for the next round of attack and defense game, otherwise go to step S206.

S206. The control device executes corresponding security operations.

In an embodiment, when the control device performs security operations, it transfers the access request obtained by the target working terminal to the honeynet cluster, and interrupts the operation of the attacking terminal on the target working device.

In the above technical solution, the control device establishes an attack-defense game model, repeatedly executes to obtain the attack strategy information of the attack terminal attacking the target working terminal observed by the control device and the a priori type of the identity type of the attack terminal. After this operation, according to the attack strategy information and Prior probability, determine the optimal defense strategy and the optimal attack strategy for the target working terminal, and implement the optimal defense strategy, and control the equipment to calculate the value of the Markov learning function according to the optimal defense strategy and attack strategy information until the Markov learning function value is calculated according to the Markov Cove learning function value, stop the cycle when it is determined that the target working terminal is not in a safe state, so that the control device can determine the attack status of the target working terminal according to the persistent attack strategy information obtained by the target working terminal, which helps it to obtain The information identifies stealthy attacks, making it better to protect target work terminals.

Fig. 3 is a schematic diagram of a calculation flow of a Markov learning function value provided by the present application according to an exemplary embodiment. As shown in Figure 3, the method includes:

S301. The control device calculates the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy and attack strategy information.

The strategy space of the game participants includes a defense strategy space, and the defense strategy space includes at least one defense strategy; the strategy space of the game participants also includes an attack strategy space, and the attack strategy space includes at least one attack strategy.

The control device calculates the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy, attack strategy information and the calculation formula of the attack and defense return value. The calculation formula of the attack and defense return value specifically includes:

Among them, _sp represents the current state of the target working terminal, Re(f _x (s _p )) represents the attack and defense return value of the target working terminal x in the current state, i represents the defense identity in the defense strategy space, and j represents the attack strategy space i=0 means that the target working terminal x has not been defended; i≠0 means that the target working terminal x is defended by the i-th defense strategy; j=0 means that the target working terminal x has not been attacked; j≠0 means The target working terminal x is attacked by the jth attack strategy, and P represents the protection value. In one embodiment, P is a constant, D _i represents the return value when the target working terminal x is protected by the i-th defense strategy, and δ _i represents Return coefficient, D represents the attack reward value when the target working terminal is attacked when it is not protected by the defense strategy, Pk _ij represents the return value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

S302. The control device obtains the cumulative return value of the target working terminal from the initial state to the current state and the Markov learning function value of the target working terminal in the current state.

Among them, the cumulative return value of the target working terminal from the initial state to the current state can be obtained through the cumulative return value formula, where the cumulative return value formula is specifically expressed as:

表示目标工作终端x在从当前状态s₀变化到状态s_p时的累计回报值，Re(f_x(s₀))表示当前状态对应的攻防回报值；s₁表示目标工作终端的第一状态，第一状态是处于当前状态的目标终端在攻击策略和防护策略的影响下改变后的下一状态，Re(f_x(s₁))表示第一状态对应的攻防回报值；s_p表示终端的当前状态的未来第p个状态，Re(f_x(s_p))表示状态p对应的攻防回报值；γ表示折现率参数，γ∈[0，1]，折现率为0时，目标系统仅考虑当前回报；折现率为1时，目标系统将得到长期高回报。in,

Indicates the cumulative reward value of the target working terminal x when it changes from the current state s ₀ to the state s _p , Re(f _x (s ₀ )) represents the attack and defense reward value corresponding to the current state; s ₁ represents the first state of the target working terminal , the first state is the next state after the target terminal in the current state changes under the influence of the attack strategy and defense strategy, Re(f _x (s ₁ )) represents the attack and defense return value corresponding to the first state; _sp represents the terminal The p-th future state of the current state, Re(f _x (s _p )) represents the attack and defense return value corresponding to state p; γ represents the discount rate parameter, γ∈[0,1], when the discount rate is 0, The target system only considers the current return; when the discount rate is 1, the target system will get long-term high returns.

The learning rate parameter represents the learning state of the control device during the state change process when the target working terminal is attacked. The learning rate parameter ∈ [0, 1], a learning rate parameter of 0 means that the subsequent behavior has no effect on the current situation; a learning parameter of 1 means that the impact of the subsequent behavior on the current situation must be considered.

The Markov learning function value of the target working terminal in the current state is the cumulative return value obtained by the control device from the initial state when the target working device is attacked until the current state, wherein, if the current state is the first attack In the initial state, the Markov learning function in the current state is a preset initial value. In one embodiment, the preset initial value is 0. More specifically, when the state of the target working terminal is p, the state of the target working device was transformed from p-1 to p when it was attacked last time.

S303. The control device calculates the maximum cumulative return value of the target working terminal in the next state according to the attack terminal's optimal attack strategy for the optimal defense strategy, all defense strategies, cumulative reward value, and attack-defense reward value.

The control device calculates the attack and defense return value of the target working terminal adjusted based on each defense strategy in the next state according to the attack terminal's optimal attack strategy for the optimal defense strategy, each defense strategy and the attack and defense return value formula involved in step S301, That is, based on the attack strategy to be adopted by the attacking terminal predicted by the control device and all the defense strategies that the control device can implement for the attack strategy, the attack and defense return values corresponding to each defense strategy are calculated respectively.

After obtaining the attack and defense return values, the control device adds each attack and defense return value to the cumulative return value obtained in step S302 to calculate the cumulative return value in the next state of the target working terminal adjusted based on each defense strategy. The control device determines the maximum value among the cumulative return values in the next state corresponding to each defense strategy as the maximum cumulative return value.

S304. The control device calculates the Markov value of the target working terminal in the next state of the current state according to the attack and defense return value in the current state, the maximum cumulative return value in the next state, and the Markov learning function value in the current state. Learn function values.

According to the attack and defense return value in the current state, the maximum cumulative return value in the next state, the Markov learning function value in the current state and the learning function update formula, calculate the Markov value of the target working terminal in the next state of the current state value of the learning function; wherein, the updating formula of the learning function specifically includes:

表示目标工作终端在当前状态的下一状态下的马尔科夫学习函数值，

表示目标工作终端在当前状态下的马尔科夫学习函数值，α表示学习率参数，γ表示折现率参数，

表示下一状态下的最大累计回报值；s₀表示目标工作终端的初始状态，Re(f_x(s₀))表示初始状态对应的攻防回报值；s₁表示目标工作终端的第一状态，第一状态是处于初始状态的目标终端在攻击策略和防护策略的影响下改变后的状态，Re(f_x(s₁))表示第一状态对应的攻防回报值；s_p表示终端的当前状态，Re(f_x(s_p))表示当前状态对应的攻防回报值；s_p+1表示目标工作终端的下一状态，Re(f_x(s_p+1))表示下一状态对应的攻防回报值。其中，马尔科夫学习函数值的初始化值为0。in,

Indicates the Markov learning function value of the target working terminal in the next state of the current state,

Indicates the Markov learning function value of the target working terminal in the current state, α indicates the learning rate parameter, γ indicates the discount rate parameter,

Represents the maximum cumulative return value in the next state; s ₀ represents the initial state of the target working terminal, Re(f _x (s ₀ )) represents the attack and defense return value corresponding to the initial state; s ₁ represents the first state of the target working terminal, The first state is the changed state of the target terminal in the initial state under the influence of attack strategy and defense strategy, Re(f _x (s ₁ )) represents the attack and defense return value corresponding to the first state; _sp represents the current state of the terminal , Re(f _x (s _p )) represents the attack and defense return value corresponding to the current state; s _p+1 represents the next state of the target working terminal, Re(f _x (s _p+1 )) represents the attack and defense corresponding to the next state return value. Among them, the initial value of the Markov learning function value is 0.

In the above technical solution, after the control device establishes the attack-defense game model, it obtains the prior probability and the attack strategy information observed by the control device in at least one cycle, and determines the control device according to the prior probability and attack strategy information in each cycle. The optimal defense strategy for the attack terminal and the optimal attack strategy for the attack terminal against the optimal defense strategy, and use the Markov learning function to determine whether the target working terminal is dangerous under the influence of this attack strategy, so that the control device can determine When the target working terminal is not in a safe state, the cycle is terminated, so that the control device can monitor the multiple attacks performed by the attack terminal, and count the damage caused by multiple attacks to the target system, and detect the concealed and persistent attack operations, thereby When the control device determines that the target working terminal is not in a safe state, it can protect the target working terminal in time, effectively ensuring the safety of the target working terminal.

FIG. 4 is a schematic structural diagram of a network defense device provided by the present application according to an embodiment. The network defense device 400 includes an acquisition module 401 and a processing module 402, wherein,

The processing module 402 is used to establish an attack-defense game model; wherein, the attack-defense game model includes multiple game participants, the strategy space of each game participant, signal space, prior probability, posterior probability and the income of each game participant.

The acquisition module 401 is configured to repeatedly perform the acquisition of the attack strategy information of the attacking terminal attacking the target working terminal observed by the control device and the a priori probability of the identity type of the attacking terminal.

The processing module 402 is also used to determine the optimal defense strategy and the optimal attack strategy of the attack terminal for the optimal defense strategy according to the attack strategy information and the prior probability, and execute the optimal defense strategy;

The processing module 402 is also used to calculate the Markov learning function value according to the optimal defense strategy and attack strategy information;

The processing module 402 is further configured to determine whether the target working terminal is in a safe state according to the value of the Markov learning function, and terminate the loop when the target working terminal is not in a safe state.

In an embodiment, the processing module 402 is specifically used to:

According to the attack strategy information and the prior probability, calculate the posterior probability of the identity type of the attacking terminal;

Determine the identity type of the attacking terminal according to the posterior probability;

Determine system cost and type cost based on attack strategy information and identity type;

According to the system cost, type cost, attack strategy information and posterior probability, calculate the attacker's benefit of the attack terminal and the defender's benefit of the target system;

According to the posterior probability and the defender's income, the optimal defense strategy and the optimal attack strategy for the target working terminal are determined.

In an embodiment, the processing module 402 is specifically used to:

Query the attack strategy information in the system cost mapping table to obtain the system cost corresponding to the attack strategy information;

Query the attack policy information and identity type in the type cost mapping table to obtain the type cost corresponding to the attack policy information.

In an embodiment, the processing module 402 is specifically used to:

According to the optimal defense strategy and attack strategy information, calculate the attack and defense return value of the target working terminal in the current state;

Obtain the cumulative return value of the target working terminal from the initial state to the current state and the Markov learning function value of the target working terminal in the current state;

Calculate the maximum cumulative return value of the target working terminal in the next state according to the attack terminal's optimal attack strategy for the optimal defense strategy, all defense strategies, cumulative return value, and attack-defense return value;

Calculate the Markov learning function value of the target working terminal in the next state of the current state according to the attack and defense return value in the current state, the maximum cumulative return value in the next state, and the Markov learning function value in the current state.

In an embodiment, the processing module 402 is specifically used to:

Calculate the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy, attack strategy information and attack and defense return value calculation formula:

Among them, the calculation formula of attack and defense return value includes:

Among them, _sp represents the current state of the target working terminal, Re(f _x (s _p )) represents the attack and defense return value of the target working terminal x in the current state, i represents the defense identity in the defense strategy space, and j represents the attack strategy space i=0 means that the target working terminal x has not been defended; i≠0 means that the target working terminal x is defended by the i-th defense strategy; j=0 means that the target working terminal x has not been attacked; j≠0 means The target working terminal x is attacked by the jth attack strategy, P represents the protection value, D _i represents the return value when the target working terminal x is protected by the i-th defense strategy, δ _i represents the return coefficient, D represents the target working terminal is not attacked by The attack reward value of being attacked when the defense strategy is protected, Pk _ij represents the reward value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy.

In an embodiment, the processing module 402 is specifically used to:

According to the optimal attack strategy of the attack terminal for the optimal defense strategy, each defense strategy and the attack and defense return value formula, calculate the attack and defense return value in the next state of the target working terminal adjusted based on each defense strategy;

Adding each offensive and defensive return value to the cumulative return value to calculate the cumulative return value in the next state of the target working terminal adjusted based on each defense strategy;

The maximum value among the cumulative return values in the next state corresponding to each defense strategy is determined as the maximum cumulative return value.

In an embodiment, the processing module 402 is specifically used to:

According to the attack and defense return value in the current state, the maximum cumulative return value in the next state, the Markov learning function value in the current state and the learning function update formula, calculate the Markov value of the target working terminal in the next state of the current state value of the learning function; wherein, the updating formula of the learning function specifically includes:

表示目标工作终端在当前状态的下一状态下的马尔科夫学习函数值，

表示目标工作终端在当前状态下的马尔科夫学习函数值，α表示学习率参数，γ表示折现率参数，

表示下一状态下的最大累计回报值；s₀表示目标工作终端的初始状态，Re(f_x(s₀))表示初始状态对应的攻防回报值；s₁表示目标工作终端的第一状态，第一状态是处于初始状态的目标终端在攻击策略和防护策略的影响下改变后的状态，Re(f_x(s₁))表示第一状态对应的攻防回报值；s_p表示终端的当前状态，Re(f_x(s_p))表示当前状态对应的攻防回报值；s_p+1表示目标工作终端的下一状态，Re(f_x(s_p+1))表示下一状态对应的攻防回报值。in,

Indicates the Markov learning function value of the target working terminal in the next state of the current state,

Indicates the Markov learning function value of the target working terminal in the current state, α indicates the learning rate parameter, γ indicates the discount rate parameter,

Represents the maximum cumulative return value in the next state; s ₀ represents the initial state of the target working terminal, Re(f _x (s ₀ )) represents the attack and defense return value corresponding to the initial state; s ₁ represents the first state of the target working terminal, The first state is the changed state of the target terminal in the initial state under the influence of attack strategy and defense strategy, Re(f _x (s ₁ )) represents the attack and defense return value corresponding to the first state; _sp represents the current state of the terminal , Re(f _x (s _p )) represents the attack and defense return value corresponding to the current state; s _p+1 represents the next state of the target working terminal, Re(f _x (s _p+1 )) represents the attack and defense corresponding to the next state return value.

In an embodiment, the processing module 402 is specifically used to:

When the Markov learning function value is less than or equal to the safety threshold, it is determined that the target working terminal is in a safe state;

When the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in a safe state.

Fig. 5 is a schematic structural diagram of a control device provided by the present application according to an embodiment. Wherein, the control device 500 includes a memory 501 and a processor 502, and the memory 501 is used for storing computer instructions executable by the processor. The memory 501 may include a high-speed random access memory (Random Access Memory, RAM), and may also include a non-volatile storage (Non-Volatile Memory, NVM), such as at least one disk storage, and may also be a U disk, a mobile hard disk , read-only memory, disk or CD-ROM, etc.

When the processor 502 executes computer instructions, it implements various steps in the network defense method in the above-mentioned embodiments with the control device as the execution subject. For details, refer to the related descriptions in the foregoing method embodiments. The processor 502 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in conjunction with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

Optionally, the above-mentioned memory 501 may be independent or integrated with the processor 502 . When the memory 501 is set independently, the control device 500 also includes a bus for connecting the memory 501 and the processor 502 . The bus may be an Industry Standard Architecture (Industry Standard Architecture, ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, the buses in the drawings of the present application are not limited to only one bus or one type of bus.

The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored, and when the processor executes the computer instructions, each step in the network defense method in the foregoing embodiments is implemented.

The embodiment of the present application also provides a computer program product, including computer instructions, and when the computer instructions are executed by a processor, each step in the network defense method in the foregoing embodiments is implemented.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in the application . The specification and examples are to be considered exemplary only, with a true scope and spirit of the application indicated by the following claims.

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A network defense method, wherein the method is applied to a control device, the control device is located in a target system, and the target system includes at least one working terminal and a honeynet cluster, the method comprising: An attack-defense game model is established; wherein, the attack-defense game model includes a plurality of game participants, the strategy space of each of the game participants, the signal space, the prior probability, the posterior probability and the income of each of the game participants; Obtaining the attack strategy information of the attacking terminal attacking the target working terminal and the prior probability of the identity type of the attacking terminal observed by the control device repeatedly; determining an optimal defense strategy and an optimal attack strategy of the attacking terminal for the optimal defense strategy according to the attack strategy information and the prior probability, and executing the optimal defense strategy; calculating a Markov learning function value according to the optimal defense strategy and the attack strategy information; Based on the Markov learning function value, it is determined whether the target working terminal is in a safe state and a loop is terminated when the target working terminal is not in a safe state. 2. The method according to claim 1, wherein, according to the attack strategy information and the prior probability, determining an optimal defense strategy and an optimal attack strategy for the target working terminal, specifically comprising: calculating the posterior probability of the identity type of the attacking terminal according to the attack strategy information and the prior probability; Determine the identity type of the attacking terminal according to the posterior probability; determining system cost and type cost according to the attack strategy information and the identity type; calculating the attacker payoff of the attacking terminal and the defender payoff of the target system according to the system cost, the type cost, the attack strategy information, and the posterior probability; An optimal defense strategy and an optimal attack strategy for the target working terminal are determined according to the posterior probability and the defender's income. 3. The method according to claim 2, wherein the system cost and type cost are determined according to the attack policy information and the identity type, specifically comprising: Querying the attack strategy information in a system cost mapping table to obtain the system cost corresponding to the attack strategy information; The attack policy information and the identity type are queried in a type cost mapping table to obtain the type cost corresponding to the attack policy information. 4. The method according to claim 1, wherein the strategy space of the game participants comprises a defense strategy space, and at least one defense strategy is included in the defense strategy space; according to the optimal defense strategy and the Attack strategy information, calculate Markov learning function value, including: calculating the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy and the attack strategy information; Obtaining the cumulative return value of the target working terminal from the initial state to the current state and the Markov learning function value of the target working terminal in the current state; According to the optimal attack strategy of the attacking terminal for the optimal defense strategy, all the defense strategies, the cumulative reward value and the offensive and defensive reward value, calculate the maximum cumulative reward of the target working terminal in the next state value; According to the attack and defense return value in the current state, the maximum cumulative return value in the next state and the Markov learning function value in the current state, calculate the next state of the target working terminal in the current state The Markov learning function value of . 5. method according to claim 4, is characterized in that, the strategy space of described game participant also comprises attack strategy space, comprises at least one attack strategy in the described attack strategy space; The above attack strategy information is used to calculate the attack and defense return value of the target working terminal in the current state, specifically including: Calculate the attack and defense return value of the target working terminal in the current state according to the optimal defense strategy, the attack strategy information and the attack and defense return value calculation formula: Wherein, the calculation formula of the offensive and defensive return value specifically includes:

Wherein, sp represents the _pth state after the current state of the target working terminal, Re(f _x (sp )) represents the attack and defense return value of the target working terminal x in state _p , and i represents the The defense identifier in the defense strategy space, j represents the attack identifier in the attack strategy space, i=0 means that the target working terminal x is not defended in state p, i≠0 means that the target working terminal x is in state p In state p, it is defended by the i-th defense strategy, j=0 means that the target working terminal x is not attacked in state p, and j≠0 means that the target working terminal x is in state p by the j-th attack strategy , P represents the protection value, D _i represents the reward value when the target working terminal x is protected by the i-th defense strategy, δ _i represents the reward coefficient, D represents the target working terminal x is protected by the defense strategy The attack return value of the attack, Pk _ij represents the return value when the game theory model adopts the i-th protection strategy to resist the j-th attack strategy. 6. The method according to claim 5, characterized in that, according to the optimal attack strategy of the attacking terminal for the optimal defense strategy, all the defense strategies, the cumulative return value and the attack-defense return value, calculate The maximum cumulative return value of the target working terminal in the next state specifically includes: According to the optimal attack strategy of the attack terminal for the optimal defense strategy, each of the defense strategies and the attack and defense return value formula, calculate the attack and defense return value in the next state of the target working terminal adjusted based on each of the defense strategies ; adding each of the offensive and defensive return values to the cumulative return value, and calculating the cumulative return value in the next state of the target working terminal adjusted based on each of the defense strategies; Determining the maximum value among the cumulative return values in the next state corresponding to each defense strategy as the maximum cumulative return value. 7. The method according to claim 6, characterized in that, according to the attack and defense return value under the current state, the maximum cumulative return value under the next state and the Markov learning function value under the current state , calculating the Markov learning function value of the target working terminal in the next state of the current state, specifically including: According to the attack and defense return value in the current state, the maximum cumulative return value in the next state, the Markov learning function value in the current state and the learning function update formula, calculate the current state of the target working terminal The Markov learning function value in the next state of ; wherein, the learning function update formula specifically includes:

in,

Indicates the Markov learning function value of the target working terminal in the next state of the current state,

Represents the Markov learning function value of the target working terminal in the current state, α represents the learning rate parameter, γ represents the discount rate parameter,

Represents the maximum cumulative return value under the next state; s ₀ represents the initial state of the target work terminal, Re(f _x (s ₀ )) represents the attack and defense return value corresponding to the initial state; s ₁ represents the target work terminal The next state of the terminal state s ₀ , Re(f _x (s ₁ )) represents the attack and defense return value corresponding to state s ₁ ; sp _p represents the current state of the terminal, Re(f _x (s _p )) represents the current The attack and defense return value corresponding to the state; _sp+1 represents the next state of the current state of the target working terminal, and Re(f _x (s _p+1 )) represents the attack and defense return value corresponding to the next state. 8. The method according to claim 1, wherein, according to the Markov learning function value, determining whether the target working terminal is at a safe level specifically comprises: determining that the target working terminal is in the safe state when the Markov learning function value is less than or equal to a safe threshold; When the Markov learning function value is greater than the safety threshold, it is determined that the target working terminal is not in the safety state. 9. A control device, comprising: a processor and a memory connected to the processor in communication; the memory stores computer instructions; The processor is used to implement the network defense method according to any one of claims 1 to 8 when executing the computer instructions. 10. A computer-readable storage medium, wherein computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed by a processor, they are used to implement the network defense methods described above.

Images