CN101119373B - A gateway-level streaming virus scanning method and system thereof - Google Patents

Abstract

The invention discloses a gateway-level streaming virus scanning method and system thereof. The method comprises firstly organizing session data packets into data unit queues, and then analyzing whether the data types in the data units support streaming scanning, and scanning the data according to the data types. Carry out file scanning or flow scanning; Its system comprises data flow obtaining device, preprocessing device, virus scanning device, the table device of storing context information; Gateway level flow type virus scanning method and system that the present invention proposes, in guaranteeing While performing effective and accurate virus detection on the data, the response speed of the gateway to the client is improved, and the storage resources of the gateway are saved.

Description

A gateway-level streaming virus scanning method and system thereof

technical field

The invention relates to a gateway-level streaming virus scanning method and system thereof, belonging to the technical fields of computer network and data communication.

Background technique

As the virus spreads increasingly networked, gateway-level virus filtering is getting more and more attention. The gateway-level virus filtering tool enables viruses to be detected and killed immediately at the entrance of the LAN, limiting the spread of viruses as early as possible. However, due to the limitations of traditional virus detection methods, most gateway-level virus filtering tools use file-based killing methods, that is, first reintegrate multiple network data packets into a complete file in external memory or memory, and then perform virus scan. This method needs to cache multiple data packets at the gateway first, integrate them into files to scan for viruses, and then forward them again, which will cause delays in data packets and reduce network performance. Especially for applications with high real-time requirements, such as Web services, the impact is relatively large.

Scanning and killing viruses through files at the gateway will not only affect real-time performance, but also occupy a large amount of memory. In a LAN with heavy network traffic, if all sessions are cached and released after virus detection, it will increase the gateway’s memory overhead, and even affect the stability of the gateway’s operation in severe cases. In addition, some viruses appear at the beginning of the session, but the file-type killing method needs to cache all the contents of the session before scanning the virus, which not only wastes system resources, but also affects the timeliness of virus killing.

In addition, the supported protocols for scanning and killing viruses in the form of files at the gateway are relatively limited. Since all data is cached at the gateway, the gateway must simulate the communication between the actual client and the server, and after the anti-virus is completed, simulate the connection between the server and the actual client again and complete the communication. For some complex application protocols, it is difficult for the gateway to simulate the entire communication process, so it is also difficult to obtain complete communication data to form a file.

Contents of the invention

The purpose of the present invention is to provide a novel gateway-level streaming virus scanning method and system thereof.

Streaming virus scanning means that you don’t have to wait for all the session data from the network to arrive and restore them to files for virus scanning. Instead, with the transmission of session data, you can divide the data stream into data units for continuous scanning. The receiving, scanning, and sending of data units The process runs in parallel.

The present invention adopts following technical scheme to realize the purpose of the present invention:

A novel gateway-level streaming virus scanning method, the steps of which are:

1) Organizing the session data packet into a data unit queue;

2) Determine whether the data type in the data unit supports stream scanning;

If stream scanning is not supported, file scanning is performed;

If stream scanning is supported, perform the following steps:

a) grouping data units for virus scanning;

b) Judging whether the scanned data unit is safe, if it is not safe, stop scanning and close the connection with the client, and exit the instance process; if it is safe, send the group of data;

c) judge whether the session ends, if not, then retain the boundary information of the group of data, and use the table device of context information to carry out boundary processing; the method is: A) copy the session data packets supporting streaming scanning to the table of context information In the buffer buffer of the device, the starting address of copying is: buffer+offset+cur_buff_len; if there is boundary data from the previous scan in the buffer, the newly read data follows the boundary data, and the next step The scanning operation can calculate these two parts of data as a whole; B) copy the group of data to the data unit queue bb maintained by the table device of the context information at the same time; C) pass the data in the buffer of the table device of the context information through The stream scanning interface of the virus scanning device is passed to the virus scanning device;

Wherein, the key field of the data structure of the table device of the context information includes: buffer: its function is to store the data to be scanned based on the streaming scanning mode, and the data that needs to be stored when dealing with boundary effects, and the data type is a character array ;cur_buff_len: Its function is to store the value of the length of the data stored in the current buffer, and the data type is an integer; offset: Its function is to store the offset between the starting position of the data stored in the current buffer and the first address of the buffer, and the data type is an integer ;tmpFile: Its function is based on the file scanning mode, the temporary file that stores the queue data, and the data type is a file pointer; partcnt: Its function is to record during the virus scanning process, the multi-mode virus signature has been matched Sub-mode information, and their positional relationship, the data type is a structure linked list;

d) Repeat steps 1) and 2), and exit the instance process if it ends.

The boundary information is the end data of the maximum virus signature length of the virus database included in the group of data units.

The boundary information includes recording the sequence numbers of the sub-patterns detected for the multi-pattern virus signature in the group of data, and the positional relationship between the sub-patterns.

In the stream scanning method, the method for dealing with the boundary problem when using the table device of context information to match the multi-mode virus signature is: record the serial number of the multi-mode virus signature in partcnt, and the matched sub-pattern information for subsequent When the data arrives, continue to match; if the scanned data matches all the subpatterns defined in the virus signature of the multi-pattern type and satisfies the specified positional relationship, the current data is considered to be infected.

The session data is composed of multiple data unit queues, and the last data unit of the last queue stores an end flag EOS.

The data packet payload data types that do not support stream scanning include, but are not limited to, one or more of the following types: gzip, zip, bz, rar, and base64.

A gateway-level streaming virus scanning system, comprising:

A data flow acquiring device, configured to acquire incoming data units, and organize them into data unit queues for sessions;

A preprocessing device to analyze whether the data type supports streaming scanning;

A virus scanning device, which performs virus scanning processing on the data of the two interfaces of streaming virus scanning and file virus scanning respectively;

A data stream sending device, configured to forward the processed data;

The table device for storing context information is used to deal with the virus segmentation problem caused by segmenting data streams during stream scanning; the key domains of its data structure include:

buffer: Its function is to store the data to be scanned and the data to be stored when dealing with boundary effects based on the streaming scanning mode, and the data type is a character array;

cur_buff_len: Its function is to store the value of the length of data stored in the current buffer, and the data type is integer;

offset: Its function is to store the offset from the starting position of the data stored in the current buffer to the first address of the buffer, and the data type is an integer;

tmpFile: Its function is based on the temporary file for storing queue data in the file scanning mode, and the data type is a file pointer;

partcnt: Its function is to record the matched sub-pattern information and their positional relationship during the multi-pattern virus scanning process during stream scanning, and the data type is a structure linked list.

Positive effect and advantage of the present invention

The streaming virus detection system and method proposed by the present invention, the method of the present invention is to scan the network data flow passing through according to the order of the flow, and the receiving, scanning and sending processes of the data units are carried out in parallel, without the need to All data packets are cached and integrated into files before scanning, which reduces the delay of data transmission and improves scanning efficiency; while ensuring effective and accurate virus detection of data, it improves the response speed of the gateway to the client and saves storage resources of the gateway.

Description of drawings

Fig. 1 is the flowchart of the inventive method;

Fig. 2 is a network topology diagram of an embodiment of the present invention;

Fig. 3 is a system structure diagram of the present invention.

Detailed ways

The gateway-level streaming virus scanning system of the present invention will be described in detail below with reference to the accompanying drawings.

The flow process of the inventive method is as shown in Figure 1, and its work process is as follows:

(1) System initialization. This step includes two aspects of work, one is to initialize the data structure of the table device for storing the context information to assist in completing streaming virus scanning, and the other is to initialize the virus scanning device. Initializing the virus scanning device includes importing the virus database. During the import process, the length maxpatlen of the largest virus code in the current virus database is calculated according to the length of each virus signature code: the length of the single-mode virus signature code is the length of the signature string. The length of the pattern virus signature is calculated according to the length of the included subpattern and its offset relationship. The running period of the virus scanning device starts from the arrival of the first data packet of the protocol layer session and ends with the last data packet. Therefore, the virus scanning device should perform context protection within the scope of a complete protocol layer session. When the gateway receives data of a data packet, the data acquisition device will create a virus scanning device instance dedicated to this request for processing the current queue. When the next data belonging to the same session arrives, the data stream acquisition device will automatically invoke the filtering instance belonging to the same request. When all the data processing of this session is finished, the virus scanning instance will exit and end its operation. A table device that stores context information throughout the processing of a session.

(2) Data flow acquisition, organizing session data packets into data unit queues. Data packets belonging to the same session have the same basic information such as source address, destination address, source port, destination port, protocol number, etc., and the high-level protocols to which they belong have corresponding status information, time stamps, and other identifications. The data acquisition module can distinguish the session to which the data packet belongs according to the above information.

When the gateway receives a data packet, the data acquisition module will determine the session to which the data packet belongs, and store a certain number of data packets into the data unit according to the inflow sequence, and then insert the data unit into the data unit queue of the session . If there is no data unit queue for the session currently, create a dedicated data unit queue. If the data packet contains an end-of-session flag, the device for obtaining the data stream inserts an "EOS" flag into the data unit at the end of the queue to which it belongs. A complete session will consist of multiple data unit queues. The data flow obtaining device transmits the organized data unit queue to the virus scanning device.

(3) Data stream preprocessing, judging whether the data type of the data unit supports stream scanning. The purpose of this step is to prepare for the next scanning process. The mechanism is divided into two situations: preprocessing based on streaming scanning; preprocessing based on file scanning.

Some data types that do not support stream scanning are pre-declared in the system, such as: gzip, zip, bz, rar, base64, etc. At the same time, in order to assist the virus scanning device to deal with the virus segmentation problem caused by streaming virus scanning and killing, a table device for storing context information is established, because when the data unit queue transmits data, there will be formation between data units and between queues. border effect. The table device for storing context information is shown in Table 1.

Table 1. streamav_ctx table

Data information type of data meaning buffer character array Used in streaming scanning mode, storing data to be scanned, and data that needs to be stored when dealing with boundary effects cur_buff_len integer For streaming scan mode, the length of data stored in the current buffer offset integer For streaming scan mode, the offset between the starting position of the data stored in the current buffer and the first address of the buffer tmpFile file pointer Temporary file for storing queue data for file scan mode bb character array linked list Used in streaming scanning mode, the data unit queue maintained by the virus scanning device instance itself partcnt Structure linked list It is used in flow scanning mode to record the matched sub-pattern information and their positional relationship during the multi-mode virus scanning process

During data stream preprocessing, if the result of the data type analysis is a special type, it is necessary to use a file-based scanning mode to preprocess the data, that is, directly cache the data unit data obtained by the data stream acquisition device into a temporary file (temporary file pointer Stored in the tmpFile field of the streamav_ctx table structure). After the session ends, the data content in the temporary file will be properly processed (such as decompressing the file), and then the file-type scanning interface of the virus scanning device will be called.

If the data type is not a special type, the virus scanning device uses the data preprocessing mechanism based on the streaming scanning mode to call the streaming scanning interface of the virus scanning device to perform streaming virus scanning. During the streaming virus scanning process, the virus scanning The device divides every three data units in the queue into a group, and processes the remaining data units after each complete processing of the current three data units, including preprocessing, scanning, and sending. The following is a detailed process of a preprocessing.

A) Copy the data of the first three data units in the data unit queue to the buffer buffer of the table streamav_ctx, and the starting address of the copy is: buffer+offset+cur_buff_len. If there is boundary data from the previous scan in the buffer, the newly read data follows the boundary data, and the next scanning operation can calculate these two parts of data as a whole.

B) Copy the data in these three data units to the data unit queue bb maintained by streamav_ctx at the same time.

C) Transfer the data in the buffer of the table streamav_ctx to the virus scanning device.

(4) Virus scanning. The virus scanning device provides a file scanning interface and a stream scanning interface. In the process of initialization and preprocessing, the scanning data of the corresponding interface has been prepared for virus scanning, and this step only needs to directly call the corresponding interface of the virus scanning device. If it is file scanning, the virus scanning device scans the temporary file pointed to by tmpFile of streamav_ctx; if it is stream scanning, the virus scanning device scans the data unit data in the streamav_ctx table bb queue. If the scanning result shows that the data is safe, and there is still data to be processed in this session, that is, no session end sign EOS appears, then the virus scanning device performs boundary data protection processing. The virus scanning device has a virus database, and the virus database contains a collection of virus signatures. If the scanned content contains a character string sequence matching the virus signature in the virus database, it can be determined that the scanned content is infected by the virus. The virus scanning device records the maximum length of the virus signature code in the virus database during the initialization process, so that the virus scanning system can handle the boundary effect caused by data segmentation during the virus scanning process. When the virus scanning device performs boundary processing, it saves the data of the length maxpatlen at the end of the data unit stored in the buffer. If the scanned data matches a virus signature of a single pattern type, the current data is considered infected. If the scanned data matches all the sub-patterns defined in the virus signature of a certain multi-pattern type and satisfies the specified positional relationship, the current data is considered to be infected. If the scanned data matches part of the subpatterns defined in a virus signature of a certain multi-pattern type, record the serial number of the multi-pattern virus signature and the matched subpattern information in the partcnt of the streamav_ctx table for subsequent data arrival continue to match. The detected infected data will be directly discarded, and subsequent data will not be processed. The virus scanning device submits the scanning result to the data stream sending device.

(5) Data stream output. If the detected data is safe, the data stream sending device immediately sends this part of data to the client. For streaming scanning mode, if there are data units waiting to be processed in the currently processed data unit queue, or there is no data in the currently processed data unit queue, but there is no EOS data unit indicating the end of the session, the virus scanning device instance will continue to be reserved In memory, process or wait to process the remaining data unit data.

If the measured data contains a virus, the data stream sending device sends a prompt message to the client, closes the connection with the client, and exits the instance process. If the virus scanning device has processed all data for this session, it will also exit the instance process. Before exiting the instance process, the memory space allocated during the scanning process will be released.

A gateway-level streaming virus scanning system is shown in FIG. 3 . Accompanying drawing 2 is that the system of the present invention runs at the gateway and is located between the local area network and the external network. The system proposed by the present invention includes: a data flow acquisition device for obtaining incoming data packets, and organizes data flow queues for sessions; a preprocessing device for analyzing data flow queues and judging data types; a virus scanning device that supports streaming Two interfaces, virus scanning and file virus scanning, are used for virus scanning and processing for different types of data respectively; the data stream sending device is used to forward the processed data and release the memory space occupied during the scanning process; the device for storing context information A table device for dealing with virus segmentation problems caused by segmenting data streams during stream scanning. Among them, streaming virus scanning refers to virus scanning of the flowing network data streams according to the order of the flow, and the receiving, scanning, and sending processes of data units are carried out in parallel, instead of caching and integrating all data units into files before scanning , thereby reducing the delay of data transmission and improving the scanning efficiency.

Among them, the data flow acquisition device can distinguish the incoming network data packets according to the session, and store a certain number of data packets together according to the inflow sequence to form a "data unit" that is conducive to virus scanning, and then arrange the "data units" belonging to the same session into a "data unit queue". Among them, "data unit" and "data unit queue" both correspond to specific data structures defined by the data acquisition module for efficient memory operation and management. In the data structure corresponding to the data unit, each field mainly includes the content and size of the included network data packet, the function pointer of the basic operation, and the queue pointer of the data unit where it is located. The data structure corresponding to the data unit queue is a circular linked list containing a series of related data units, which is used to provide flexible and efficient memory operations, such as allocation and recovery of data units, for use by other modules. A complete session will consist of multiple data unit queues. The data stream acquisition device can judge whether the session is over, and insert an "EOS" flag into the data unit at the end of the queue to mark the end of the session. The data flow obtaining device transmits the organized data unit queue to the virus scanning device.

The preprocessing device analyzes the data type in the data unit queue. If the data is a data type that does not support streaming scanning (such as compression types such as gzip and rar, or encoding types such as base64), all data in the current session need to be It is cached as a complete file, and after being properly processed (such as decompressing the file), the file-based scanning interface of the virus scanning device is called. If the data type in the data unit queue is not a special type, the streaming scanning interface of the virus scanning device is invoked to perform streaming virus scanning.

The virus scanning device operates independently in a process mode, and scans specified data after receiving a scanning request from the preprocessing device. The virus scanning device is equipped with a virus database, which contains a set of virus signature codes, and the virus signature codes are divided into two forms: single mode and multimode. If the scanned content contains a character string sequence matching a single-mode virus signature, it can be determined that the scanned content is infected by the virus. If the character string sequence contained in the scanned content can match multiple patterns defined in a multi-mode virus signature and meet certain specified conditions, it can be determined that the scanned content is infected by the virus.

The table device for storing context information, as shown in Table 1, is used to assist the virus scanning device in dealing with the problem of virus segmentation caused by streaming virus scanning and killing.

Claims (7)

1. A gateway-level streaming virus scanning method, the steps of which are: 1) Organizing the session data packet into a data unit queue; 2) Determine whether the data type in the data unit supports stream scanning; If stream scanning is not supported, file scanning is performed; If stream scanning is supported, perform the following steps: a) grouping data units for virus scanning; b) Judging whether the scanned data unit is safe, if it is not safe, stop scanning and close the connection with the client, and exit the instance process; if it is safe, send the group of data; c) judge whether the session ends, if not, then retain the boundary information of the group of data, and use the table device of context information to carry out boundary processing; the method is: A) copy the session data packets supporting streaming scanning to the table of context information In the buffer buffer of the device, the starting address of copying is: buffer+offset+cur_buff_len; if there is boundary data from the previous scan in the buffer, the newly read data follows the boundary data, and the next step The scanning operation can calculate these two parts of data as a whole; B) copy the group of data to the data unit queue bb maintained by the table device of the context information at the same time; C) pass the data in the buffer of the table device of the context information through The stream scanning interface of the virus scanning device is passed to the virus scanning device; Wherein, the key field of the data structure of the table device of the context information includes: buffer: its function is to store the data to be scanned based on the streaming scanning mode, and the data that needs to be stored when dealing with boundary effects, and the data type is a character array ;cur_buff_len: Its function is to store the value of the length of the data stored in the current buffer, and the data type is an integer; offset: Its function is to store the offset between the starting position of the data stored in the current buffer and the first address of the buffer, and the data type is an integer ;tmpFile: Its function is based on the file scanning mode, the temporary file that stores the queue data, and the data type is a file pointer; partcnt: Its function is to record during the virus scanning process, the multi-mode virus signature has been matched Sub-mode information, and their positional relationship, the data type is a structure linked list; d) Repeat steps 1) and 2), and exit the instance process if it ends. 2. The method according to claim 1, wherein the boundary information is the end data of the maximum virus signature length of the virus database contained in the group of data units. 3. The method according to claim 1, wherein the boundary information is to record the sequence numbers of sub-patterns detected for the multi-pattern virus signature in the group of data, and the positional relationship between each sub-pattern. 4. method as claimed in claim 1 is characterized in that the method for processing border problem when utilizing the table device of context information to carry out multi-mode virus signature code matching in stream scanning is: Record the serial number of the multi-mode virus signature and the matched sub-pattern information in partcnt for continued matching when subsequent data arrives; if the scanned data matches all the sub-patterns defined in the multi-mode virus signature, and If the specified positional relationship is met, the current data is considered to be infected. 5. The method according to claim 1, wherein the session data is composed of multiple data unit queues, and the last data unit of the last queue contains an end flag EOS. 6. The method according to claim 1, wherein the data packet payload data types that do not support stream scanning include but are not limited to one or more of the following types: gzip, zip, bz, rar, base64. 7. A gateway-level streaming virus scanning system, comprising: A data flow acquiring device, configured to acquire incoming data units, and organize them into data unit queues for sessions; A preprocessing device to analyze whether the data type supports streaming scanning; A virus scanning device, which performs virus scanning processing on the data of the two interfaces of streaming virus scanning and file virus scanning respectively; A data stream sending device, configured to forward the processed data; The table device for storing context information is used to deal with the virus segmentation problem caused by segmenting data streams during stream scanning; the key domains of its data structure include: buffer: Its function is to store the data to be scanned and the data to be stored when dealing with boundary effects based on the streaming scanning mode, and the data type is a character array; cur_buff_len: Its function is to store the value of the length of data stored in the current buffer, and the data type is integer; offset: Its function is to store the offset from the starting position of the data stored in the current buffer to the first address of the buffer, and the data type is an integer; tmpFile: Its function is based on the temporary file for storing queue data in the file scanning mode, and the data type is a file pointer; partcnt: Its function is to record the matched sub-pattern information and their positional relationship during the multi-pattern virus scanning process during stream scanning, and the data type is a structure linked list.

Images