CN106713462A - Network data packet processing method and device - Google Patents

Abstract

The present application discloses a method and device for processing network data packets. The method includes: in the driver of the kernel state, establishing a mapping relationship between the memory space of the user state and the buffer area of the network device; data; the acquired data is directly converted from the kernel state to the user state; the CPU is used to process the data, and the device includes: a mapping relationship establishment unit, which is used in the driver of the kernel state to provide user state memory space and network equipment The buffer to establish a mapping relationship; the data acquisition unit is used to obtain data from the buffer of the network device; the data conversion unit is used to directly convert the acquired data from the kernel state to the user state; the data processing unit is used for The data is processed by the CPU. The above method and device can reduce system resources and time occupied by each data packet processing, and improve network data packet processing performance.

Description

Method and device for processing network data packets

technical field

The invention belongs to the technical field of data processing, in particular to a method and device for processing network data packets.

Background technique

With the development of network communication technology, network bandwidth increases rapidly, 1Gbit and 10Gbit networks are gradually popularized, and 40Gbit networks are widely used in telecom operators and large-scale enterprise-level backbone networks. The exponential growth of network traffic poses new challenges to the performance of network data packet analysis and processing network products such as firewalls, IDS/IPS, and application gateways. Traditional network data packet processing methods face problems such as high packet loss rate, large delay, and high system load in high-speed network environments, and cannot meet the requirements of real-time processing of network data packets in high-speed network environments.

Contents of the invention

In order to solve the above problems, the present invention provides a network data packet processing method and device, which can reduce system resources and time occupied by each data packet processing, and improve network data packet processing performance.

A kind of network packet processing method provided by the present invention comprises:

In the kernel mode driver, establish a mapping relationship between the user mode memory space and the buffer of the network device;

Obtain data from the buffer of the network device;

Convert the acquired data directly from the kernel state to the user state;

The data is processed by the CPU.

Preferably, in the above method for processing network data packets, the acquisition of data from the buffer of the network device is:

Using the polling thread to acquire data from the buffer of the network device.

Preferably, in the above method for processing network data packets, before using the polling thread to obtain data from the buffer of the network device, it further includes:

Bind each polling thread to a corresponding CPU.

Preferably, in the above-mentioned method for processing network data packets, in the driver of the kernel state, before establishing a mapping relationship between the user state memory space and the buffer of the network device, it also includes:

The device driver is split into the driver of the kernel state and the memory space of the user state.

A network data packet processing device provided by the present invention includes:

A mapping relationship establishment unit is used to establish a mapping relationship between the user mode memory space and the buffer of the network device in the driver of the kernel mode;

a data acquisition unit, configured to acquire data from the buffer of the network device;

The data conversion unit is used to directly convert the obtained data from the kernel state to the user state;

The data processing unit is configured to process the data by using the CPU.

Preferably, in the above-mentioned device for processing network data packets, the data acquisition unit is specifically used for:

Using the polling thread to acquire data from the buffer of the network device.

Preferably, in the above-mentioned network data packet processing device, it also includes:

The binding unit is used to bind each polling thread to a corresponding CPU.

Preferably, in the above-mentioned network data packet processing device, it also includes:

The splitting unit is configured to split the device driver into the kernel-mode driver and the user-mode memory space.

It can be seen from the above description that the above-mentioned network data packet processing method and device provided by the present invention include: in the driver of the kernel state, establishing a mapping relationship between the memory space of the user state and the buffer area of the network device; Obtain data in the area; directly convert the acquired data from the kernel state to the user state; use the CPU to process the data. The method and device avoid the copying process from the core state to the user state, can simplify the protocol layer processing process, and shield the protocol processing of the socket layer, so can reduce system resources and time occupied by each data packet processing, and improve network data packet processing performance .

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.

FIG. 1 is a schematic diagram of a first network packet processing method provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a first network data packet processing device provided by an embodiment of the present application.

detailed description

The core idea of the present invention is to provide a network data packet processing method and device, which can reduce system resources and time occupied by each data packet processing, and improve network data packet processing performance.

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The first network data packet processing method provided in the embodiment of the present application is shown in FIG. 1 , and FIG. 1 is a schematic diagram of the first network data packet processing method provided in the embodiment of the present application. The method comprises the steps of:

S1: In the kernel mode driver, establish a mapping relationship between the user mode memory space and the buffer of the network device;

It should be noted that when the network protocol stack processes network data, it no longer performs multiple data copies, but directly copies the data in the hardware to the user state, avoiding the copy process from the core state to the user state, which can simplify The protocol layer processing process shields the protocol processing of the socket layer.

S2: Obtain data from the buffer of the network device;

In this step, methods such as interrupt or polling may be used to acquire, which is not limited here.

S3: Convert the acquired data directly from the kernel state to the user state;

It should be noted that, in the implementation of the kernel protocol stack in the prior art, data needs to go through multiple processes of Firmware-Device Driver-Ethernet-IP-TCP/UDP-Sockets-SystemCall Interface starting from the physical device arriving at the network card, These steps will involve direct movement of data in memory or index movement, as well as conversion from kernel mode to user mode, which will bring unnecessary overhead. Using this step, the data in the hardware will be directly copied to the user mode. , this problem can be solved.

S4: Using the CPU to process the data.

It should be noted that the above method optimizes the processing process of network data packets by rewriting the Linux kernel network protocol stack to achieve the purpose of improving the performance of network data packet processing. When implementing processing threads, avoid using system calls in the processing process , to reduce the overhead of entering system calls.

It can be seen from the above description that the above-mentioned first network data packet processing method provided by the embodiment of the present application includes: in the driver of the kernel state, establishing a mapping relationship between the memory space of the user state and the buffer zone of the network device; Obtain data from the buffer of the device; directly convert the obtained data from the kernel state to the user state; use the CPU to process the data. The method avoids the copying process from the core state to the user state, can simplify the protocol layer processing process, and shield the socket layer protocol processing, so it can reduce the system resources and time occupied by each data packet processing, and improve the network data packet processing performance.

The second network data packet processing method provided in the embodiment of the present application is based on the above-mentioned first network data packet processing method, and also includes the following technical features:

The data obtained from the buffer of the network device is:

Using the polling thread to acquire data from the buffer of the network device.

It should be noted that in the prior art driver implementation methods, a large number of interrupts are used to notify the system of data changes. When there are system call statements in the program, when the program executes to the system call, first use soft interrupts and other similar instructions. Save the scene, make a system call, execute it in the kernel mode, and then restore the scene. Each process will have two stacks, a kernel mode stack and a user mode stack. When the interrupt is executed, it will switch from the user mode stack to the kernel mode stack. The stack needs to be switched when the system is called. Moreover, the kernel code does not trust the user and requires additional checks. The return process of the system call has a lot of extra work, such as checking whether scheduling is required. However, with the continuous development of computer technology, interrupt resources have increasingly become the bottleneck of system performance. Instead, CPU performance has become higher and higher, and polling threads have become a better way. Therefore, in the optimization of the driver in the embodiment of the present application , using the polling method instead of the interrupt method, which has three advantages: reducing the occupation of system interrupt resources; reducing the impact of system scheduling; because the polling method is used to access external devices, the impact of external device abnormalities on the system can be reduced. In addition, in the process of writing the polling function for processing network data packets, avoid using the system call interface. You can use direct memory operations, such as memory address operations, memory linked list operations, and memory data operations, such as modifying user applications and using new Interfaces, such as redefined interfaces such as init, start, stop, reset, read, and write, implement access to network devices, which can improve processing performance.

The third network data packet processing method provided in the embodiment of the present application is based on the above-mentioned second network data packet processing method, and also includes the following technical features:

Before using the polling thread to obtain data from the buffer of the network device, it also includes:

Bind each polling thread to a corresponding CPU.

In this case, according to the number of cores of the current CPU in the system, set no more than polling threads for processing network data, and bind them to the CPU to ensure that each thread always runs on one core, which can prevent threads from running on the CPU core. The cost of switching between.

The fourth network data packet processing method provided in the embodiment of the present application is based on the above-mentioned first to third network data packet processing methods, and also includes the following technical features:

In the driver in the kernel state, before establishing a mapping relationship between the user state memory space and the buffer of the network device, it also includes:

The device driver is split into the driver of the kernel state and the memory space of the user state.

Specifically, it is to rewrite the network device driver and split the original Device Driver into two parts: the driver in the kernel space and the driver in the user space, so as to provide a basis for converting data from the kernel state to the user state.

The first network data packet processing device provided in the embodiment of the present application is shown in Figure 2, and Figure 2 is a schematic diagram of the first network data packet processing device provided in the embodiment of the present application, and the device includes:

The mapping relationship establishment unit 201 is used to establish a mapping relationship between the user mode memory space and the buffer zone of the network device in the driver of the kernel mode. Data replication, but directly copy the data in the hardware to the user state, avoiding the copy process from the core state to the user state, which can simplify the protocol layer processing process and shield the protocol processing of the socket layer;

The data obtaining unit 202 is configured to obtain data from the buffer of the network device, which may be obtained by means of interruption or polling, which is not limited here;

The data conversion unit 203 is used to directly convert the obtained data from the kernel state to the user state, and can directly copy the data in the hardware to the user state, reducing unnecessary memory overhead;

The data processing unit 204 is configured to use the CPU to process the data, avoid using system calls during processing, and reduce the overhead of entering system calls.

It can be seen from the above description that the above-mentioned first network data packet processing device provided by the embodiment of the present application avoids the copy process from the core state to the user state, can simplify the protocol layer processing process, and shield the protocol processing of the socket layer, so it can reduce each The system resources and time occupied by each data packet processing can be improved, and the performance of network data packet processing can be improved.

The second network data packet processing device provided in the embodiment of the present application is based on the above-mentioned first network data packet processing device, and further includes the following technical features:

The data acquisition unit is specifically used for:

Using the polling thread to acquire data from the buffer of the network device.

It should be noted that in the prior art driver implementation methods, a large number of interrupts are used to notify the system of data changes. When there are system call statements in the program, when the program executes to the system call, first use soft interrupts and other similar instructions. Save the scene, make a system call, execute it in the kernel mode, and then restore the scene. Each process will have two stacks, a kernel mode stack and a user mode stack. When the interrupt is executed, it will switch from the user mode stack to the kernel mode stack. The stack needs to be switched when the system is called. Moreover, the kernel code does not trust the user and requires additional checks. The return process of the system call has a lot of extra work, such as checking whether scheduling is required. However, with the continuous development of computer technology, interrupt resources have increasingly become the bottleneck of system performance. Instead, CPU performance is getting higher and higher, and polling threads has become a better way. Therefore, in the optimization of the driver in this embodiment of the application, the polling method is used instead of the interrupt method. This method has three advantages: reducing the occupation of system interrupt resources; reducing the impact of system scheduling; equipment, which can reduce the impact of external equipment abnormalities on the system. In addition, in the process of writing the polling function for processing network data packets, avoid using the system call interface. You can use direct memory operations, such as memory address operations, memory linked list operations, and memory data operations, such as modifying user applications and using new Interfaces, such as redefined interfaces such as init, start, stop, reset, read, and write, implement access to network devices, which can improve processing performance.

The third network data packet processing device provided in the embodiment of the present application is based on the above-mentioned second network data packet processing device, and further includes the following technical features:

The binding unit is used to bind each polling thread to a corresponding CPU.

In this case, according to the number of cores of the current CPU in the system, set no more than polling threads for processing network data, and bind them to the CPU to ensure that each thread always runs on one core, which can prevent threads from running on the CPU core. The cost of switching between.

The fourth network data packet processing device provided in the embodiment of the present application is based on any one of the above-mentioned first to third network data packet processing devices, and further includes the following technical features:

The splitting unit is configured to split the device driver into the kernel-mode driver and the user-mode memory space.

Specifically, it is to rewrite the network device driver and split the original Device Driver into two parts: the driver in the kernel space and the driver in the user space, so as to provide a basis for converting data from the kernel state to the user state.

To sum up, the above-mentioned method and device provided by the embodiment of the present application can reduce the processing time of each network data packet, and the advantages are as follows: reduce the number of times data is copied in the memory, prevent repeated copying; reduce the occupation of system resources, Including system hard interrupts, soft interrupts and CPU resources, etc.; reducing the impact of external device exceptions on system stability; improving network packet processing performance.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (8)

1. a kind of processing method of network data packets, it is characterised in that including：

It is that User space memory headroom sets up mapping relations with the buffering area of the network equipment in the driving of kernel state；

Data are obtained from the buffering area of the network equipment；

The data of acquisition are directly changed into User space from kernel state；

The data are processed using CPU.

2. processing method of network data packets according to claim 1, it is characterised in that described from the slow of the network equipment Rushing acquisition data in area is：

Using the poll thread data are obtained from the buffering area of the network equipment.

3. processing method of network data packets according to claim 2, it is characterised in that utilize the poll thread described Before obtaining data from the buffering area of the network equipment, also include：

Each described poll thread is tied on a corresponding CPU.

4. the processing method of network data packets according to claim any one of 1-3, it is characterised in that described in kernel state It is before User space memory headroom sets up mapping relations with the buffering area of the network equipment, also to include in driving：

Device drives are split as the driving of the kernel state and the User space memory headroom.

5. a kind of network packet processing unit, it is characterised in that including：

Mapping relations set up unit, the buffering area in the driving of kernel state, being User space memory headroom and the network equipment Set up mapping relations；

Data capture unit, for obtaining data from the buffering area of the network equipment；

Date Conversion Unit, for the data of acquisition to be directly changed into User space from kernel state；

Data processing unit, for being processed the data using CPU.

6. network packet processing unit according to claim 5, it is characterised in that the data capture unit is specifically used In：

Using the poll thread data are obtained from the buffering area of the network equipment.

7. network packet processing unit according to claim 6, it is characterised in that also include：

Binding unit, for each described poll thread to be tied into a corresponding CPU.

8. the network packet processing unit according to claim any one of 5-7, it is characterised in that also include：

Split cells, for the driving that device drives are split as the kernel state and the User space memory headroom.