CN110224990A - A kind of intruding detection system based on software definition security architecture - Google Patents

Abstract

The invention discloses an intrusion detection system based on a software-defined security framework, which belongs to the field of network information security. The intrusion detection system includes a client module and a cloud module; the cloud module includes a cloud agent, an intrusion detection engine, an expert rule base, and a machine learning module. library and log database; the intrusion detection engine uses Snort-based feature detection technology and machine learning-based anomaly detection technology. The software-defined security architecture provides programmable control and global status monitoring of the network, provides a unified and transparent access mode for the abstracted underlying security devices, expands northbound security applications upward, and leverages cloud computing technology for elastic computing, With the advantages of distributed computing, load balancing, and big data processing capabilities, the expert rule base, intrusion detection engine, and related artificial intelligence detection algorithms are deployed on the cloud, which improves the system's intelligent detection efficiency, enhances the system's dynamic expansion capabilities and new Rapid response capabilities to security threats.

Description

An intrusion detection system based on software-defined security architecture

technical field

The present application belongs to the field of network information security, and in particular, relates to an intrusion detection system and a detection method under a software-defined security framework.

Background technique

In recent years, with the rapid development of Internet technology, the continuous expansion of network scale, the continuous increase of network traffic, and the increasing complexity of network architecture, traditional network architecture is facing more and more severe challenges and tests. At the same time, the complex network environment has also brought many network security problems, such as malicious software attacks, spoofing attacks and distributed denial of service attacks. The increasingly prominent network security issues have gradually posed more serious challenges to the traditional security system architecture, service models and technical means. On the one hand, with the continuous development of related technologies such as cloud computing and virtualization, network application requirements have become more and more complex, and traditional network architectures have problems such as difficult expansion and high configuration complexity. Not only that, various virtualization technologies have realized the rapid arrangement and flexible deployment of network resources, and the traditional network security architecture is gradually stretched and it is difficult to meet people's needs. On the other hand, some existing security defense technologies (such as firewalls, intrusion detection systems, etc.) are mostly deployed in the local area network in the form of hardware devices, with relatively single functions and poor flexibility. Segmented and independent, unable to perform system combination configuration, has the disadvantages of weak real-time defense capabilities and poor scalability, and is difficult to adapt to dynamic business needs and perform online upgrades of security functions.

As a dynamic, manageable, cost-effective and adaptable emerging architecture, software-defined network (SDN) is reshaping the way of thinking of the network, and provides an important supporting solution for the evolution of network information security. The security protection function module is globally optimized in a defined way, so as to achieve the purpose of unified management and dynamic configuration, that is, software-defined security (SDS). SDN decouples network control and forwarding functions, makes network control directly programmable, and abstracts the underlying infrastructure for applications and network services. The isolation of the control plane and the data plane makes network management easier. The controller is used to provide centralized control of the network, realize global status monitoring, and flexibly obtain and collect network activity information. Through the controller, network administrators can quickly and easily make and roll out decisions about how the underlying systems in the data plane (switches, routers) handle traffic, enable integration of disparate architectures, and facilitate the creation of network applications and services to better adapt to user needs. Through centralized management and control, SDN realizes dynamic resource allocation and scheduling, and optimizes network configuration, monitoring, management, scheduling, and optimization. Therefore, introducing SDN research results into existing network security protection technologies is a development trend of network information security. Software-defined security (SDS) enhances network management capabilities, collaboration levels, and service quality, and provides feasible solutions for network security issues. s solution.

As one of the key technologies to protect the network from malicious attacks, the Intrusion Detection System (IDS) aims to detect malicious activities (including viruses, worms and DDoS attacks, etc.) by collecting traffic information of key nodes in the network and analyzing all network activities. , and take timely alarm measures. IDS can identify and detect intrusion attacks in real time, and is widely used in traditional networks. Existing IDS systems and security devices are generally deployed in local area networks, which makes the coordination and linkage of the system poor. The above problems can be well solved by software-defined security IDS. Generally, the traditional IDS detects attack patterns based on expert rule bases, which brings the risk of high false positive rate and high false positive rate. At the same time, it is difficult to detect new types of network attacks in real time to adapt to complex and changeable network environments.

At present, most intrusion detection systems are based on feature detection. Once the attacker slightly modifies the known attack and changes the signature of the known attack, the method of feature detection will not be able to detect any abnormal content. In view of the fact that traditional IDS is facing the challenges and threats of a large number of increasingly complex and heterogeneous network attacks, in order to overcome the disadvantages and limitations of traditional IDS, machine learning technology is introduced into IDS. Due to the good adaptive characteristics and mathematical robustness of machine learning, various algorithms such as Neural Network, Support Vector Machine, Naive Bayes, Decision Tree, Random Forest (Random Forest) and other algorithms have also been added to the intrusion detection technology. In recent years, more and more researches have been done combining machine learning with hot issues and application fields. By learning from existing intrusion data, machine learning algorithms can detect new unknown attacks. However, the currently commonly used feature-based detection methods need to accurately describe the features and pre-define the rules to match the intrusion target, and cannot realize the identification and detection of unknown attacks. Detect inevitable trends and needs for development.

Contents of the invention

Aiming at the problems existing in the prior art, this application proposes an intrusion detection system based on a software-defined security architecture, which utilizes cloud computing's ability to support virtualization, large-scale data processing, distributed computing, and load balancing to combine expert rule base, The intrusion detection engine and related artificial intelligence detection algorithms are deployed on the cloud, providing a solution for deploying an intrusion detection system on the cloud. It not only reduces the computing and processing burden of the client, effectively improves the system's intelligent detection efficiency and rapid response to new security threats, but also enhances the system's dynamic expansion capability and resource self-adaptive configuration capability by using the cloud environment deployment method.

This solution is realized through the following technical solutions: an intrusion detection system based on a software-defined security architecture, the intrusion detection system includes an SDN controller, a client module, and a cloud module; the client module includes a client agent, a communication transmission module and data packet sniffing module; the cloud module includes a cloud agent, a communication transmission module, and an intrusion detection engine; the data packet sniffing module collects network data and delivers it to the client agent, and the client agent encapsulates the data And according to the communication transmission module to realize the communication with the cloud agent, the communication transmission module will adopt the communication protocol defined by the cloud agent and the client. The cloud agent receives the flow data sent from the client module, and sends the flow data to the intrusion detection engine for detection, and then returns the result of the intrusion detection to the client agent, and realizes rapid response and proactive through the SDN controller. Defense; the intrusion detection engine adopts Snort-based feature detection technology and machine learning-based anomaly detection technology.

Further, the cloud module also includes an expert rule base, a machine learning base, and a log database, and the intrusion detection engine distinguishes known attacks from unknown attacks through the expert rule base. Snort-based feature detection technology is used for real-time detection of known attacks; unknown attacks are sent to the machine learning library for training and learning, new rules are constructed to supplement the expert rule library in real time, and data packets and detection results are recorded in the log database.

Further, the anomaly detection technology uses an incremental learning method to perform real-time incremental training for real-time detection according to sequentially arriving data traffic, and save it to the classifier model, and new traffic samples that arrive subsequently can be automatically identified by existing models Classification, distinguishing whether it is malicious traffic, and fully improving the detection performance of the classifier through continuous learning.

Further, the incremental learning method includes an offline part and an online part, the main steps are respectively offline training model, offline verification model, online incremental learning, the offline part uses the historical data of the cloud log database, and the online part is Based on live new data samples.

Compared with the existing technology, this scheme has the following beneficial effects: (1) Through the intrusion detection engine Snort-IDS, the relevant machine learning algorithm is introduced, a machine learning library is designed and constructed, and combined with the features of Snort based on the expert rule base intrusion detection. The implementation of the machine learning library is as a third-party plug-in. By loading the machine learning plug-in into the Snort software, the cloud hybrid network intrusion detection system architecture is realized; (2) A complete intrusion detection system with complete functions and global coordination is proposed, namely A complete IDS mechanism that unifies and combines multiple links such as client capture of network traffic, cloud intrusion detection and data storage, result feedback and response. The application proposes a scalable and programmable intrusion detection system based on software-defined security architecture. By separating security data from control, it realizes automatic collaboration and global view management between security applications-SDN controllers-security bottom equipment, and can Acquire and collect network information flexibly, and discover and identify abnormal events and behaviors in a timely manner. Based on the software-defined security architecture SDS, with the goal of realizing intrusion detection, taking advantage of big data and cloud computing technology, deploying expert rule bases, intrusion detection engines and related artificial intelligence detection algorithms in the cloud not only effectively improves the intelligent detection of the system Efficiency and security protection capabilities also enhance the system's dynamic expansion capabilities and resource allocation capabilities.

Description of drawings

Fig. 1 is the intrusion detection system architecture based on software-defined security of the present invention;

Fig. 2 frame diagram of the present invention;

Fig. 3 flow chart of the present invention;

Fig. 4 cloud intrusion detection engine of the present invention;

Fig. 5 is the attack scene diagram of the present invention;

Fig. 6 is a line graph of algorithm simulation of the present invention;

Fig. 7 is a simulation test time delay diagram of the present invention.

Detailed ways

An intrusion detection system based on a software-defined security architecture, the intrusion system includes an SDN controller, a client module and a cloud module. The client module includes a client agent, a communication transmission module and a data packet sniffing module; the cloud module includes a cloud agent, a communication transmission module, an intrusion detection engine, an expert rule library, a machine learning library, and a log database.

The intrusion detection system consists of client network traffic capture, cloud intrusion detection and data storage, result feedback and response. First, the packet sniffing module is responsible for collecting network data and delivering it to the client agent, and the client agent communicates and interacts with the cloud agent through the encapsulated data according to the communication transmission module. The cloud agent is responsible for receiving the data collected from the client module, sending the traffic data to the intrusion detection engine for detection, and finally returning the intrusion detection result to the client agent, and realizing rapid response and active defense through the SDN controller.

In order to improve the real-time detection efficiency of the system and detect unknown attacks, the intrusion detection engine uses two intrusion detection technologies at the same time, the feature detection technology based on Snort and the anomaly detection technology based on machine learning, and is deployed on the virtual machine instance running on the computing node Realize distributed joint detection. The intrusion detection engine distinguishes normal and abnormal traffic based on the existing expert rule base, and sends new types of attacks or derived attacks from existing attacks to the machine learning library for training and identification, thereby supplementing the expert rule base in real time and sending the data packets and test results are recorded in the cloud log database module.

The intrusion detection (IDS) system of the present invention: the cloud agent receives the data packets sent from the client, and the intrusion detection engine deployed on the computing node performs joint intrusion detection on these network data packets, and according to the existing expert rule base To distinguish whether these traffic are normal data packets or abnormal data packets. The expert rule base is a method based on predefined rules. Known intrusion behavior characteristics or attack codes are compiled into a rule set. If the data traffic matches a certain feature of the expert rule base, the data is judged as malicious traffic. . Send the captured network data flow to the abnormal detection-based machine learning algorithm library for training analysis and knowledge learning. If it is found to be abnormal traffic, summarize the rules and compare them with the expert rule base. If the expert rule base does not contain the rule, then add the new rule to the expert rule base, thereby supplementing the expert rule base in real time, and record the alarm log in the log database for backup. Among them, the machine learning-based anomaly detection technology uses an incremental learning method, trains the classifier model through the existing log database, and realizes incremental training when new samples arrive, dynamically updates the model configuration, and realizes real-time streaming attack detection.

Example

The design and implementation of the OpenStack cloud platform, the design and implementation of the client, the design and implementation of the cloud, and the design and implementation of the communication pipeline between the client and the cloud will be described in detail with specific embodiments.

1. Design and implementation of OpenStack cloud platform

This solution adopts the OpenStack multi-node construction mode of 1 master node, 1 controller node, and 2 compute nodes. Both the controller node and the compute node provide block storage and network services, and are deployed centrally on physical servers. For cloud platform network planning and design, there are three network cards, namely:

eth0 (OpenStack cluster management network, CIDR is 10.20.0.1/24), used to manage cluster nodes;

eth1 (external network/floating IP network, CIDR is 172.16.0.1/24), used for cluster public network and providing virtual machine floating IP address;

eth2 (management/storage/internal network, CIDR is 192.168.1.0/24), used for the internal communication network of the virtual machine.

In order to ensure the stability and effectiveness of the cloud environment, first deploy the three nodes in the same LAN, and ensure that they can ping each other. At the same time, set the time zone of the three hosts to ensure that the NTP time is synchronized. Then deploy different component services on each node.

After the service is correctly configured on each node, the cloud platform can be accessed through the interface provided by Dashboard.

As shown in Figure 1. Logically, the cloud space is divided into:

1) Big data center: The center has designed two types of databases, one is log database for storing historical data and detection results, and the other is rule-oriented expert rule base. The databases are implemented based on MySQL. Under the guidance and deployment of the intelligent center algorithm, the expert rule base can update the rules in real time.

2) Intelligent center: stores relevant machine learning algorithms used by the intrusion detection engine and new rule generation algorithms used to formulate new rules, and the unified interface is used for external access. Machine learning algorithms include CNN, SVM, SOM, K-NN, GDBT, AdaBoost, etc. The new rule generation algorithm is mainly based on the machine learning algorithm to detect unknown attack patterns, then extract the main features such as source IP address, destination IP address, source port, destination port, protocol type, etc., and compile them into new rules in a certain format.

3) Control Center; the Center is designed to achieve the following purposes:

a) Deploy the cloud agent to receive the data files uploaded by the client and save them in the log database for backup. The cloud agent is also responsible for the communication transmission service of the external network as a window.

b) Provide centralized view and cluster management, global control of cloud virtual resources (such as computing resources and network resources, etc.), which can be used to realize distributed joint computing of intrusion detection;

c) Pass the message queue between the cluster nodes, and transmit the detection result in JSON format to the SDN controller through the HTTPRESTful API interface service.

d) Flexibly guide the data flow to the intrusion detection engine deployed on the function center to realize intrusion behavior detection.

4) Function center: the place where the intrusion detection engine snort-IDS runs, that is, the virtual IDS. Multiple snort-IDS can be flexibly arranged according to the network status and specific needs, and corresponding algorithms can be called from the intelligence center for training and knowledge update .

2. Design and implementation of client module

The client module includes a client agent and a packet sniffing module. The client module captures data by installing Tcpdump. Tcpdump can completely intercept and analyze the data packets transmitted in the network, and it supports filtering for network layer, protocol, host, network or port.

In order to capture data packets, the network card of the client module needs to be set to promiscuous mode to monitor all network devices on the network. At this time, use Tcpdump to sniff and capture the current network data. The client proxy upload data packet program needs to encapsulate the collected traffic files, send them to the cloud proxy after data compression and Hex character stream conversion, and wait for the cloud proxy to return for detection. result.

The client module can use the Linux cron command to realize timing packet capture and save, and specify to generate a tcpdump file every 1000 network connections, set the start time STIME for packet capture and the end time ETIME for packet capture, and save the data captured each time The packages are named $STIME-$ETIME and are temporarily stored locally in .pcap.gz compressed format.

3. Design and implementation of cloud modules

The cloud module includes cloud agent, expert rule library, log database, machine learning library and intrusion detection engine. The specific cloud intrusion detection process is shown in Figure 3.

1) Design of cloud proxy

The cloud agent is deployed on the master node of OpenStack. First, according to the communication module, the cloud agent receives the flow data transmitted from the client; secondly, the cloud agent transmits the received flow file to the cloud computing node, and the intrusion detection engine deployed on the computing node realizes the detection of intrusion behavior , and finally the detection result is returned by the cloud proxy to the client proxy and the SDN controller.

2) Design of cloud related database

This solution mainly designs two types of databases in the cloud, namely feature-oriented expert rule base and log-oriented log database. Both databases are implemented based on MySQL. Among them, the expert rule base is used to store records such as rule tables and event tables; the log database backs up the captured network data packets and records the alarm logs generated by the intrusion detection engine.

3) Cloud intrusion detection engine and machine learning library design

The cloud intrusion detection engine uses the open source intrusion detection software Snort to implement signature intrusion detection. Snort is a lightweight network intrusion detection system and a rule-based network intrusion detection system. It uses a rule-based search mechanism, and its specific implementation is to use content-based pattern matching on data packets to find intrusion behavior.

As shown in Figure 4, Snort is mainly composed of five basic modules: sniffer, decoder, preprocessor, detection engine, and alarm output, and records the output results in the log database. The detection engine implements intrusion according to the expert rule base. Behavioral detection. In the process of intrusion detection, in order to facilitate the detection of intrusion data flow, Snort adopts a modular design method. Users can expand Snort and design third-party plug-ins according to their needs. This solution uses a machine learning plug-in on the Snort platform, which can integrate and develop multiple machine learning algorithms to build a machine learning library to achieve anomaly detection and generate new rules.

This solution is based on various machine learning algorithms introduced in the existing IDS, designs and builds a machine learning library, adds popular algorithms such as SVM, CNN, Random Forest, SOM, K-NN, etc., and combines the features of Snort based on expert rule bases For intrusion detection, the machine learning library is implemented as a third-party plug-in. By loading the machine learning plug-in into the Snort software, the cloud hybrid network intrusion detection system architecture is realized.

4. Design and implementation of communication pipeline between client and cloud

The client agent needs to send the data packets collected by the client to the cloud agent, and also needs to receive the results returned from the cloud agent. At the same time, the cloud agent not only needs to receive the data sent from the client, but also needs to send the feedback result of the intrusion detection engine back to the client agent. So this is a full-duplex working mode, both parties are both client and server.

As shown in Figure 2, the communication protocol between the client agent and the cloud is selected as the HTTP protocol. The communication method mainly uses two HTTP methods, namely GET and POST. The system requests access to data resources through the GET method, and requests the server to transmit the main body of the information entity through the POST method. For example, the client agent sends an HTTP request to obtain resources from the cloud agent. First, set the time interval T, so that the system will automatically run the script regularly to capture packets and upload real-time data packets. Specifically, the client agent establishes a socket connection with the cloud agent, and the client agent polls the local folder. Once a new Tcpdump file is generated, the client agent sends an HTTP POST request, and the cloud agent immediately responds to the HTTP request. The client agent compresses and encapsulates the captured data packet resource, and transmits it to the cloud agent in JSON format. The cloud proxy receives the data and continues to parse the JSON data, thereby restoring the original data package and storing it in the log database for backup and preservation.

The software environment and development platform of the experiment of the present invention: for client, hardware platform is 2.8 GHz IntelCore i5, and software platform is Mac OS X10.14.4; Cloud hardware platform is Intel (R) Core (TM) i7-7700 CPU@3.60GHz , the OpenStack software platform is Centos release6.5, Ubuntu release14.04, and the OpenStack version number is liberty. The development and compilation environments involved include PyCharm Community Edition 2017.2.3, Python 3.6, Xcode version 10.2.1, and Snort version 2.9.11.1.

Algorithms such as SOM, BP, CNN, and SOM&KNN were developed through the machine learning plug-in of Snort-IDS, and corresponding algorithm simulation tests in different attack scenarios were carried out based on these algorithms. The four attack scenarios were normal traffic mixed Probe attack, normal traffic Mixed DoS attack, normal traffic mixed U2R and R2L attack, mixed attack. In the experiment, the KDDCUP99 dataset was used to train the model, and the corresponding numerical one-hot processing was performed on the KDD99 dataset, and the 144-dimensional features of each data sample were finally obtained by padding zeros. In the experiment, the corresponding training set, verification set and test set are obtained by dividing the data set. The test set is used to randomly select a new sample for each incremental learning. First, when training the SOM neural network, use 10×10 as the weight matrix of the competition layer. After that, take the weight map of SOM as the input of K-NN, then recalculate the Euclidean distance from each sample to every 100 neurons, and finally select the top 3 winning neuron results. Secondly, when training the CNN neural network, a 2-layer convolutional layer, a 2-layer pooling layer, and a 2-layer fully connected layer are implemented. For each input sample, a 23-dimensional label vector is finally obtained, and the subscript index corresponding to the maximum value is The value corresponds to the attack type. Finally, the SOM and BP algorithms are implemented as control groups. Figure 6 presents the results of four experiments showing the binary classification accuracy of different algorithms. The results show that the algorithm has high detection efficiency for attack types with a large number of samples in the data set, such as DOS and Probe attacks, but the effect of the classifier is not obvious for U2R and R2L attacks with a small number of samples. In general, the detection efficiency of the hybrid algorithm of SOM and KNN is relatively good, while the effect of CNN neural network on real-time stream detection is poor. A feasible solution is to increase the number of samples input for each incremental learning.

Next, the intrusion detection system test is carried out, and the denial of service attack (DoS) is mainly applied. Denial of service attack, also known as flood attack, is a major network attack method that prevents computers or servers from processing legal requests by occupying system resources or network resources. The attack scenario diagram is shown in Figure 5. By simulating the attack flow, the corresponding system simulation tests were carried out for three attack modes, namely TCP SYN flood attack, UDP flood attack and ICMP Ping flood attack, and the JSON format The detection result is returned to the SDN controller, and the SDN controller issues flow table policies for active defense, thereby realizing rapid response and security protection of the system. The delay diagram of the experiment is shown in Figure 7. It is not difficult to find that the response time from external attack to defense is controlled within 10 to 20 seconds, which verifies the strong linkage coordination ability and real-time detection level of the system.

Claims (4)

1. a kind of intruding detection system based on software definition security architecture, which is characterized in that the intruding detection system includes SDN controller, client modules and cloud module；The client modules include Client Agent, communication transmission module with And ingress-only packet sniffing module；The cloud module includes cloud agency, communication transmission module, intrusion detection engine；The data Packet sniff module acquisition network data simultaneously consigns to Client Agent, and the Client Agent is encapsulated data and passed according to communication Defeated module is realized and cloud agent communication, and the communication transmission module will be using cloud agency and the customized communication protocols of client View.The cloud agency receives the data on flows sent from client modules, and data on flows is sent into the intrusion detection Engine detection, then the result of intrusion detection is returned into Client Agent, quick response and master are realized by the SDN controller Dynamic defence；The intrusion detection engine uses the feature detection techniques based on Snort and the abnormality detection skill based on machine learning Art.

2. a kind of intruding detection system based on software definition security architecture according to claim 1, which is characterized in that institute Stating cloud module further includes Expert Rules library, machine learning library and log database, and the intrusion detection engine is by special Family divides known attack and unknown attack in regular reservoir area.Feature detection techniques based on Snort are used for the real-time detection of known attack； Unknown attack is admitted in machine learning library training study, constructs new rule supplement Expert Rules library in real time, and by data packet and Testing result is recorded in log database.

3. a kind of intruding detection system based on software definition security architecture according to claim 1, which is characterized in that institute Abnormality detection technology is stated using incremental learning method, according to the data traffic that sequence reaches, the increment type being measured in real time Training is saved in sorter model, and the new flow sample of subsequent arrival can be by having model automatic recognition classification, and whether differentiation For malicious traffic stream.

4. the abnormality detection technology according to claim 3 based on machine learning, which is characterized in that the incremental learning Method includes offline part and online part, and key step is respectively off-line training model, off-line verification model, online increment It practises, the offline part uses cloud log database historical data, and the online part is based on real time new data sample.