CN101699788A - Modularized network intrusion detection system - Google Patents

Abstract

The invention discloses a modular network intrusion detection system. The system includes a data source module, which is used for packet acquisition and encoding analysis, and a distribution module, which is used to complete the scheduling of network packets from the encoding analysis module, and distributes the network packets to each detection sub-module in the detection module; the detection module, It is used to detect whether there is an attack in the network packet, which uses multiple threads, including one or more detection sub-modules; wherein, the data source module and the distribution module use one thread. The technical solution of the present invention can independently design the data acquisition and distribution/detection modular architecture, which is convenient to use hardware to realize the functions of specific modules; it can make full use of the processing capabilities of multi-core processors, and the performance can increase linearly with the number of processor cores, and can Dynamic loading of multiple detection modules is convenient for configuration according to different hardware conditions; at the same time, memory consumption is greatly reduced, and the versatility of the system is enhanced.

Description

Modular Network Intrusion Detection System

technical field

The invention relates to the technical field of network filtering and monitoring, in particular to a modularized network intrusion detection system.

Background technique

Network Intrusion Detection System (NIDS) is a network security system that monitors malicious or harmful behaviors in network or system activities and issues alarms. It intercepts data packets in the network (hereinafter referred to as network packets) through bypass or online, and analyzes its content, and gives an alarm or intercepts the network packets containing attacks, so as to realize the function of intrusion detection/defense. Due to the complex functions of NIDS, it is generally realized by software and runs inside the firewall or security gateway system.

In recent years, the industry has increasingly higher requirements on the throughput of network devices. As the performance bottleneck of the security gateway, NIDS requires a large amount of computing power to achieve high-throughput processing. Under the general-purpose processor architecture, to meet the improvement of NIDS computing requirements, only a single CPU is limited. The multi-core processors developed in recent years have solved the problem of insufficient growth in the computing power of single-core processors. However, there is a condition for multi-core applications, that is, one thread of the program can only run on one processor core. To fully utilize the performance advantages of multiple cores, multiple threads must be used for parallel processing. Therefore, the NIDS parallel processing architecture is an important technology to meet the multi-core of today's processors.

As shown in FIG. 1 , the NIDS commonly used in the prior art includes the following parts: a packet acquisition module, a code analysis module, an intrusion detection module and an information feedback module. Among them, the intrusion detection module is the core function part of NIDS. This module detects whether there is an attack in the network packet through several steps such as preprocessing and rule matching. However, the intrusion detection module is also the most time-consuming module. Generally, the parallelized NIDS will select the detection module for parallel processing to achieve the purpose of improving the utilization of multi-core CPUs.

There are not many parallel NIDSs under the general-purpose processor architecture in the prior art, and the typical ones include Supra-Snort of Intel Corporation and Multi-Snort of Purdue University in the United States. However, the former does not realize the memory sharing between multi-threads, which makes the memory consumption of multi-threads run too large, which limits the scope of application of the system; the latter does not implement a modular framework, so it is difficult to add new applications to the system.

Contents of the invention

In view of the defects and deficiencies in the prior art, the purpose of the present invention is to provide a modular network intrusion detection system with a parallel processing architecture, which can greatly reduce memory consumption and add new applications conveniently.

In order to achieve the above object, the present invention proposes a modular network intrusion detection system, including a data source module, which is used for packet acquisition and encoding analysis, and the system also includes:

A distribution module, configured to complete the scheduling of network packets from the encoding and parsing module, and distribute the network packets to each detection sub-module in the detection module through the buffer queue; and

The detection module is used to detect whether there is an attack in the network packet. It uses multiple threads and includes one or more detection sub-modules. The detection sub-modules of the same type share read-only data. Wherein, the data source module and the distribution module use one thread.

Wherein, the distribution module may also include a quintuple information distribution sub-module, which is used to distribute the network packet to the detection module according to the quintuple information of the network packet, wherein the quintuple information includes source/ Destination network layer address, source/destination transport layer port and transport layer protocol flag.

Wherein, the distribution module may further include a hash encoding submodule, configured to perform hash encoding on the five-tuple information to determine which detection submodule in the detection module to distribute the network packet to.

Wherein, the detection module may at least include one or more detection sub-modules of an intrusion detection module, an anti-virus module, and a website filtering module.

Wherein, the system may also include information feedback and network packet blocking modules.

Wherein, the system further includes a detection submodule loading module, which is used to dynamically load the detection submodule by reading configuration files.

Wherein, the detection sub-modules of the same type can be set to the same priority, and the distribution module automatically distributes the flow evenly to each detection sub-module of the same priority.

Wherein, according to the type of the detection sub-module included in the detection module, correspondingly, the read-only data may at least include one or more of an intrusion detection rule set, a virus database, and a website database.

The above technical solution has the following advantages: the present invention realizes parallel processing of the system through the shunt module, thereby separating data collection and detection, forming an independently designed data collection and distribution/detection modular architecture, which is convenient for different types of data sources It is designed separately from the detection module, and uses hardware to realize the function of a specific module, and new applications can be added conveniently; the detection module uses multi-thread design, which can make full use of the processing power of multi-core processors, and its performance can vary with the processor. The number of cores increases linearly, and multiple detection modules can be dynamically loaded, which is convenient for configuration according to different hardware conditions; at the same time, memory sharing between the same modules is realized, which greatly reduces memory consumption and enhances the versatility of the system.

Description of drawings

Fig. 1 is the functional block diagram of the NIDS of prior art;

Fig. 2 is the functional block diagram of the network intrusion detection system proposed by the embodiment of the present invention;

Fig. 3 is a block diagram of a use example of the network intrusion detection system proposed by the embodiment of the present invention.

Detailed ways

The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

As shown in Figure 2, the system according to the embodiment of the present invention includes:

The data source module is used for packet acquisition and encoding analysis (respectively equivalent to the packet acquisition module and encoding analysis module in the existing NIDS system), specifically, it integrates data acquisition, data analysis, flow management, and IP fragment reassembly , TCP flow reassembly and other functions;

Distributing module, for completing the dispatching of the network packet from described coding analysis module, manages the dispatching of network packet buffer queue, and distributes network packet to detection module by network packet buffer queue (using buffer queue so that the modules do not need to wait for each other) Each detection sub-module in ;

The detection module is used to detect whether there is an attack in the network packet. It adopts a multi-thread design and includes one or more detection sub-modules. The detection sub-modules of the same type share read-only data. The data source module and the distribution module use a thread (data collection and distribution thread); and

An information feedback and network packet blocking module (equivalent to the information feedback module in the NIDS system of the prior art).

Wherein, the splitting module can finish distributing the network packet to the detection module according to the quintuple information of the network packet, wherein the quintuple information includes source/destination network layer address (each 32bit), source/destination transport layer port (each 16bit) and transport layer protocol flag (8bit). The distribution module can perform hash coding on the five-tuple information to determine which detection sub-module in the detection module to distribute the network packet to. The detection module may at least include one or more detection sub-modules of an intrusion detection module, an anti-virus module and a network address filtering module. For example, as shown in FIG. 3 , it may include an intrusion detection module and an antivirus module. In Figures 2-3, each dotted box represents a thread.

In this embodiment, the detection sub-module may be dynamically loaded by the system by reading a configuration file. The detection sub-modules of the same type can be set to the same priority, and the distribution module will automatically distribute the traffic evenly to each detection sub-module of the same priority at each priority level, and the same network packet can be sent by multiple detection sub-modules in sequence processing to achieve flexible configuration functions.

In this embodiment, according to the type of the detection sub-module included in the detection module, correspondingly, the read-only data may at least include one or more of an intrusion detection rule set, a virus database, and a website database.

In this embodiment, the detection submodules of the same type can share read-only data through TLS (ThreadLocal Storage, thread local storage) technology. The threads of each detection sub-module are originally unrelated, but the data between the detection sub-modules is redundant, and some memory that could have been shared will be repeatedly requested. This type of memory is mainly read-only at runtime. For example, the compiled rule set is as much as 2.3GB. If each thread of the detection submodule has a copy, each thread needs to consume 2.3G of memory. It is easy to cause insufficient memory after threading. Therefore, the embodiment of the present invention realizes the memory sharing between the detection sub-modules, for example, sharing the intrusion detection rule set and the virus database, so that the memory consumption is greatly reduced, and the generality of the system is enhanced.

The foregoing is only an embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, some improvements and modifications can be made without departing from the technical principle of the present invention. It should be regarded as the protection scope of the present invention.

Claims (8)

1. a modular network intrusion detection system, comprising a data source module, which is used for packet acquisition and encoding analysis, characterized in that said system also includes: A shunting module, used to complete the scheduling of the net packets from the encoding and parsing module, and distribute the net packets to each detection sub-module in the detection module through the buffer queue; A detection module is used to detect whether an attack is contained in the network packet, and it includes one or more detection sub-modules; Wherein, the data source module and the distribution module use one thread. 2. The modularized network intrusion detection system according to claim 1, wherein the distribution module also includes a quintuple information distribution submodule, which is used to complete the network packet according to the quintuple information of the network packet Distributed to the detection module, wherein the five-tuple information includes source/destination network layer address, source/destination transport layer port and transport layer protocol flag. 3. The modularized network intrusion detection system according to claim 2, wherein the shunting module further comprises a hash coding submodule, which is used to carry out hash coding of the five-tuple information to determine the Which detection sub-module in the detection module to distribute the network packet to. 4. The modularized network intrusion detection system according to claim 1, wherein the detection module at least includes one or more detection sub-modules of an intrusion detection module, an anti-virus module and a website filtering module. 5. The modularized network intrusion detection system according to any one of claims 1-4, characterized in that the system also includes information feedback and network packet blocking modules. 6. The modularized network intrusion detection system according to any one of claims 1-4, further comprising a detection sub-module loading module for dynamically loading the detection sub-module by reading configuration files. 7. The modularized network intrusion detection system according to any one of claims 1 to 4, wherein the detection sub-modules of the same type are set to the same priority, and the distribution module automatically distributes the traffic evenly Give each detection submodule the same priority. 8. The modularized network intrusion detection system according to claim 4, wherein, according to the type of the detection sub-module included in the detection module, correspondingly, the read-only data at least includes an intrusion detection rule set One or more of , virus database and URL database.

Images