[go: up one dir, main page]

CN100459574C - Network flow classifying, state tracking and message processing device and method - Google Patents

Network flow classifying, state tracking and message processing device and method Download PDF

Info

Publication number
CN100459574C
CN100459574C CNB2005100864404A CN200510086440A CN100459574C CN 100459574 C CN100459574 C CN 100459574C CN B2005100864404 A CNB2005100864404 A CN B2005100864404A CN 200510086440 A CN200510086440 A CN 200510086440A CN 100459574 C CN100459574 C CN 100459574C
Authority
CN
China
Prior art keywords
flow
message
record
flow record
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB2005100864404A
Other languages
Chinese (zh)
Other versions
CN1937574A (en
Inventor
张建宇
韦韬
邹维
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to CNB2005100864404A priority Critical patent/CN100459574C/en
Publication of CN1937574A publication Critical patent/CN1937574A/en
Application granted granted Critical
Publication of CN100459574C publication Critical patent/CN100459574C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to a classifying, status tracking and message processing device and method for network, implementing finely granular flow control by dynamic flow classification method based on IP quinary group or other specific tags, implementing high speed parallel flow processing by plural parallel devices and inside-device multiple process/thread mechanisms, avoiding large number of exclusive and synchronous operations caused by parallel processing by the measures of slitting network flow table, setting unique writer of flow recording list, strictly stipulating write instruction sequence in list insert and delete operating processes to keep list integrity, etc, and thus further raising processing efficiency. And it is applied to various parallel processing environments, such as network processor, ASIC, FPGA, multikernel processor, symmetrical multiprocessor (SMP), and software process/ thread, having good inter-platform compatibility, extendibility and practicality.

Description

To the apparatus and method that network flow is classified, status tracking and message are handled
Technical field
The invention belongs to computer network and data communication technology field, be specifically related to a kind of to network flow is classified, status tracking and message are handled apparatus and method, can be used among the network equipment such as router, security gateway, flow monitoring and audit, network billing, load balancing and the software, realize to flow fine granularity control, improve the efficient that message is handled.
Background technology
Continuous increase along with VoIP (Voice over IP), mobile data services, P2P emerging application demands such as (Peer-to-Peer), network application presents development trend with rapid changepl. never-ending changes and improvements, also rapid growth of the network bandwidth meanwhile, the key business function of the network facilitiess such as the detection of QoS (Quality of Service), attack/invasion and defence, flow monitoring and audit, network billing, load balancing faces new and bigger challenge.These business functions relate to many Processing tasks at session, user or upper-layer protocol, session and user's load often reaches 100,000 grades even 1,000,000 grades, need take into account the high speed processing of message when flow being carried out fine granularity control, its core is to realize traffic classification efficiently.The classification of rule-based passive flow is by searching the rule that is complementary with message in the rule list of setting up in advance, message is referred in the Business Stream of matched rule appointment to handle.The method of passive flow classification does not write down the context status information of flow, need search at each message, and therefore often more complicated and load are bigger.Article one, Business Stream comprises back and forth the flow of both direction usually, and the passive flow sorting technique can't be set up the incidence relation between this both direction flow, can not satisfy the needs of some application (as intrusion detection).In addition, also there are scalability problems such as regular number restriction (maximum several ten thousand usually), regular incremental update in the passive flow sorting technique, has certain limitation.And have advantages such as fine size, extensibility are good based on the dynamic flow sorting technique of IP five-tuple (source address, destination address, source port/ICMP id, destination interface/ICMP type and code, protocol number) or other specific identifiers, therefore obtained using widely.
The dynamic flow sorting technique is based on a network flow table that dynamically updates, its basic operation is: when message arrives, the IP five-tuple information or other specific identifier requester network stream tables that comprise according to message, determine the network flow that message is affiliated, and message is done respective handling according to the processing policy information in the correspondence stream record.In addition, dynamic flow classification also will relate to the important process of two aspects: be the management of network flow table on the one hand, comprise the stream record newly-built, replace, aging and reclaim; Be the renewal that stream mode is followed the tracks of and stream writes down on the other hand.Because the scale of network flow table often reaches up to a million records, message number to be processed is also very many, so performance becomes the problem of primary solution.In addition, practicality and cross-platform compatibility also are to consider.
In sum, the apparatus and method of needing outstanding, practical, the cross-platform compatibility of a kind of performance, can classify to network flow, status tracking and message being handled are controlled and the high speed processing of message the fine granularity of flow realizing.
Summary of the invention
The purpose of this invention is to provide a kind of to network flow is classified, status tracking and message are handled apparatus and method.
According to an aspect of the present invention, provide a kind of to network flow is classified, status tracking and message are handled device, this device comprises: based on the dynamic flow sorter of IP five-tuple (source address, destination address, source port/ICMP id, destination interface/ICMP type and code, protocol number) or other specific identifiers, carry out the dynamic flow sort operation, the IP five-tuple information or other specific identifier requester network stream tables that comprise according to message, determine the network flow that message is affiliated, for the message that can not find corresponding stream record, give stream table management devices and handle; The stream mode tracking means is carried out stream mode and is followed the tracks of operation, according to the information such as stream mode, timestamp and ageing time in the message field (MFLD) content update stream record; Message process device is carried out message and is handled operation, according to the processing policy information in the stream record message is correspondingly processed; Stream table management devices, be used for carrying out network flow table stream record newly-built, replace, aging and reclaimer operation; The network flow table is used to write down network flow information, and the stream record adopts the hash table mode to organize, and adopts the chained list mode to solve Hash collision; Idle stream record list item buffering area is used to store idle stream record list item, and employing (FILO) mode first-in last-out distributes idle list item.
Wherein, a key character of dynamic flow sorter, stream mode tracking means, stream table management devices and message process device is all to comprise a plurality of processes or thread, can carry out high-speed parallel and handle.Another key character of stream table management devices is: each process or thread are responsible for the different piece of network flow table respectively, thereby the stream record chained list of each hash table entry in the assurance stream table and sensing thereof has only the person of writing, the insertion and the deletion that are chained list are responsible for by unique process or thread, and in the insertion of chained list and deletion action process by strict regulations write command order keeping the integrality of chained list, thereby avoided a large amount of mutual exclusions and the simultaneous operation that cause owing to parallel processing.
According to another aspect of the present invention, provide a kind of to network flow is classified, status tracking and message are handled method, this method comprises: carry out the dynamic flow sort operation according to IP five-tuple information or other specific identifiers that message comprises, determine the network flow that message is affiliated; Carry out stream mode according to the message field (MFLD) content and follow the tracks of operation, upgrade the stream record; Handle operation according to the information and executing message in the stream record, realize the corresponding business function; For the message of inquiry in the network flow table less than correspondence stream record, give stream table management devices with it, after stream table management devices confirms not exist corresponding stream record, carry out newly-built operation of stream record (idle stream record list item buffering area is not empty) or stream record replacement operation (idle stream record list item buffering area is empty) according to the situation of idle stream record list item buffering area again; Regularly carry out the aging operation of stream record, the stream record that meets or exceeds ageing time is deleted from the network flow table; The stream record list item of having deleted from the network flow table is carried out reclaimer operation, be recovered in the idle stream record list item buffering area.
The step of described dynamic flow sort operation is:
1) the IP five-tuple information that comprises with message (source address, destination address, source port/ICMP id, destination interface/ICMP type and code, protocol number) or other specific identifiers are key assignments substitution hash function, are that index finds hash table entry corresponding in the network flow table with the operation result.The all corresponding stream record chained list of each hash table entry is to solve the Hash collision problem;
2) with the key assignments of message successively with described hash table entry in the key assignments of each stream record compare.If find corresponding stream record (key assignments equates), then message is given the stream mode tracking means and carried out stream mode tracking operation; Otherwise, index according to hash table entry is given process or thread corresponding in the stream table management devices with message, after stream table management devices confirms not exist corresponding stream record, carry out newly-built operation of stream record (idle stream record list item buffering area is not empty) or stream record replacement operation (idle stream record list item buffering area is empty) according to the situation of idle stream record list item buffering area again.If stream table management devices finds to have existed corresponding stream record, then directly message is given the stream mode tracking means, carry out stream mode and follow the tracks of operation.
Described stream mode is followed the tracks of operation, and the step of upgrading the stream record is:
1) stream of message correspondence record is added writes lock, perhaps adopt mutual exclusion to write atomic instructions flow record content is made amendment;
2) upgrade flow state information in the stream record according to field contents (as the flags field in TCP packet header) relevant in agreement under the message (TCP, UDP, ICMP, or the like), the message and message transmissions direction with protocol status;
3) upgrade the timestamp information that flows in the record with the current time;
4) upgrade the ageing time information (the ageing time intervals that different stream modes is corresponding different) that flows in the record according to current stream mode;
5) remove the lock of writing that flows record;
6) give message process device with message, carry out message and handle operation.
The step that described message is handled operation is:
1) reads processing policy information in the stream record of message correspondence;
2) according to processing policy message is handled, realized the corresponding business function.Typical business function comprises the detection of QoS, packet filtering, attack/invasion and defence, network address translation, message forwarding, load balancing, traffic statistics, or the like.
The step that described stream writes down newly-built operation is:
1) (one network flow comprises back and forth the flow of both direction as the forward key assignments of stream with the key assignments of message, divide into positive direction flow and opposite direction flow according to first direction that arrives message, the key assignments difference of both direction), handle by the process or the thread of forward key assignments correspondence earlier;
2) fill in the information such as forward key assignments, time started, timestamp, ageing time and stream mode that flow in the record.Further, according to required business function, fill in the processing policy information in the stream record, typical business function comprises QoS, network security, network address translation, route, two layers of conversion, load balancing, traffic statistics, or the like.Then determine the reverse key assignments of stream and be filled up to flow in the record;
3) will flow the record list item inserts in the stream record chained list of forward key assignments correspondence in the network flow table;
4) message is given stream reverse key assignments correspondence process or thread process.Stream record list item is inserted in the stream record chained list of reverse key assignments correspondence in the network flow table, then message is given the stream mode tracking means, carry out stream mode and follow the tracks of operation.
The step of described stream record replacement operation is:
1) checks timestamp and the ageing time information that each stream writes down in the current stream record chained list successively, select to have reached or surpassed the stream record of ageing term.If there is not such stream record, then adopt recent minimum use replacement policy (LRU), select the oldest stream record of timestamp in the chained list, perhaps adopt first in first out replacement policy (FIFO), select to be in the stream record of linked list head;
2) the list item reclaimer operation carried out in the stream record of choosing.
The step of the aging operation of described stream record is:
1) its that part of network flow table of being responsible for of each process of stream table management devices or thread periodic scanning reclaims the stream record that meets or exceeds ageing term;
2) be the expense of the aging operation of control, the threshold value of maximum scanning list items in the once-through operation need be set.Each aging operation is all proceeded scanning since the place of finishing last time.
The step of described stream record list item reclaimer operation is:
1) handles by the process or the thread of the forward key assignments correspondence that flows record earlier.Delete in the stream record chained list with stream record list item forward key assignments correspondence from the network flow table;
2) process or the thread of then giving the reverse key assignments correspondence of stream record handled.Stream record list item is oppositely deleted in the stream of the key assignments correspondence record chained list from the network flow table, and be recovered in the idle stream record list item buffering area.Before reclaiming, using the message of this list item if also there are some, in order not influence it in removal process and reclaim later normal use, avoid since reclaim cause synchronization overhead, when reclaiming, do not empty contents in table, but by the time empty again during sub-distribution under this list item, simultaneously idle stream being write down the distributed list item threshold value of list item buffering area is arranged to less than the idle list item number of maximum---because idle stream record list item buffering area takes mode first-in last-out to distribute idle list item, therefore the list item that is recovered can not redistributed away at once, makes the message of current this list item of use successfully to dispose.
The described operating procedure of inserting stream record list item in the network flow table is:
1) supposes and between stream record list item A in the stream record chained list and C, to insert a new list item B.At first, read the value (being the position of list item C) of next list item field of list item A;
2) value that will read is write in next list item field of list item B;
3) address of list item B is inserted in next list item field of list item A.
Described operating procedure of deleting stream record list item in the network flow table is:
1) supposes and in stream record chained list, deletion to flow the list item B that writes down between list item A and the C.At first, read the value (being the position of list item C) of next list item field of list item B;
2) value that will read is write in next list item field of list item A;
3) content (next the list item field that comprises list item B) of reservation list item B does not empty.Even current like this have along this chained list carry out the reader of query manipulation and just in time arrive list item B, also can because B by from chained list the deletion and not influence its visit back list item.
The present invention relates to a kind of network flow be classified, the apparatus and method that status tracking and message are handled, employing has realized the fine granularity of flow is controlled based on the dynamic flow sorting technique of IP five-tuple or other specific identifiers, adopt a plurality of parallel devices and the inner multi-process of device or threading mechanism to realize the high-speed parallel of flow is handled, by cutting network flow table, it is unique that the stream record chained list person of writing is set, the write command order is to keep the integrality of chained list in insertion of strict regulations chained list and the deletion action process, the a large amount of mutual exclusions and the simultaneous operation that cause owing to parallel processing has been avoided in the measures such as distributed list item threshold value that idle stream record list item buffering area is set, and makes treatment effeciency be further enhanced.The present invention is applicable to various parallel processing environments such as network processing unit, ASIC, FPGA, multi-core processor, symmetric multi processor (smp), software process or thread, has good cross-platform compatibility, extensibility and practicality.
Description of drawings
Below in conjunction with accompanying drawing the present invention is illustrated in further detail:
Fig. 1 is a network flow hoist pennants of the present invention;
Fig. 2 is an idle stream record list item buffering area schematic diagram of the present invention;
Fig. 3 for according to embodiments of the invention to network flow is classified, status tracking and message are handled device schematic diagram;
Fig. 4 for according to embodiments of the invention to network flow is classified, status tracking and message are handled method flow diagram;
Embodiment
Below with reference to accompanying drawing of the present invention, describe most preferred embodiment of the present invention in more detail and describe in detail.
The present invention is a kind of to network flow is classified, status tracking and message are handled apparatus and method.
Referring to Fig. 1, network flow table of the present invention is used to write down network flow information, adopts the hash table mode to organize, and hash table length is L.Adopt the chained list mode to solve Hash collision, each hash table entry all comprises the head pointer of a stream record chained list.Because network flow comprises the flow of positive and negative both direction, so each stream record list item all belongs to two stream record chained lists of network flow table respectively, forward key assignments of the corresponding respectively stream record of these two chained lists and reverse key assignments.
The structure of network flow table hash table entry is as shown in the table:
Data message (arranging) according to storage order Length (position) Implication
Stream record chain meter pointer (flowlist) 32 The chained list that sensing is made up of the stream record list item of corresponding same Hash value
Direction signs (dir) 8 The forward key assignments of stream record is pointed in 0 expression, and 1 represents to point to the reverse key assignments of stream record, down together
The structure of stream record list item is as shown in the table:
Figure C20051008644000111
Reverse output equipment (reoutdev) 16 Reverse output equipment/forward input equipment
Reverse next list item pointer (renext) 32 Point to the next list item of the chained list of reverse key assignments retuple correspondence
Time started (starttime) 32 Timestamp constantly set up in the stream record
Timestamp (timestamp) 32 Arrive the timestamp of message recently
Ageing time is (agetime) at interval 32 Ageing time at interval, and is different and different according to stream mode
Stream mode (flowstate) 8 Stream mode
Write lock (wlock) 8 Be used for the mutual exclusion of writing of this stream record field
Processing policy information (action) Indefinite Needed information when preserving each business function processing message
Statistical information (stats) Indefinite Statistical informations such as the flow of process
Referring to Fig. 2, idle stream of the present invention record list item buffering area is used to store idle stream record list item, adopts the chained list mode to organize, and adopts first-in last-out (FILO) mode to distribute idle list item.Indicating by buffering area head pointer Ph and buffering area tail pointer Pt respectively end to end of buffering area.For fear of because the synchronization overhead that causes of reclaimer operation guarantees that the list item that is recovered can not redistributed and empty at once, setting can distribute list item threshold value Tr and maximum idle list item to count S (0<Tr<S).
Referring to Fig. 3, of the present invention the device that network flow is classified, status tracking and message are handled is comprised: dynamic flow sorter 1, stream mode tracking means 2, message process device 3, and stream table management devices 4.In addition, also comprise network flow table shown in Fig. 1-2 and idle stream record list item buffering area.Dynamic flow sorter 1 is carried out the dynamic flow sort operation, according to the IP five-tuple information inquiry network flow table that message comprises, determines the network flow that message is affiliated, for the message that can not find corresponding stream record, gives stream table management devices 4 and handles.Stream mode tracking means 2 is carried out stream mode and is followed the tracks of operation, according to the information such as stream mode, timestamp and ageing time in the message field (MFLD) content update stream record.Message process device 3 is carried out message and is handled operation, according to the processing policy information (action) in the stream record message is done corresponding processing, realizes the related service function.Stream table management devices 4 be used for carrying out network flow table stream record newly-built, replace, aging and reclaimer operation.All comprise N process or thread in each device, handle to realize high-speed parallel.In order to eliminate the mutual exclusion and the synchronization overhead of the network flow table access that causes owing to parallel processing, need the network flow table to be carried out cutting according to process in the stream table management devices or Thread Count, each process in the stream table management devices or thread independently are responsible for the part of network flow table, to guarantee that having only unique person of writing to carry out to every stream record chained list inserts and deletion action.In addition, for controlling the expense of each aging operation, threshold value Ta need be set to allow the number of the list item of scanning in the once aging operation of control.
Referring to Fig. 4, of the present invention the method that network flow is classified, status tracking and message are handled is comprised the steps:
1) network message at first enters the dynamic flow sorter, carries out dynamic flow sort operation S1.The IP five-tuple information that comprises with message is key assignments substitution hash function H, calculates index value i.The typical computing formula of function H is:
(source address+destination address+source port+destination interface+protocol number) %L
Find hash table entry E corresponding in the network flow table according to i.With the key assignments of message successively with the flowlist field indication chained list of hash table entry in the key assignments of each stream record compare.If the stream mode tracking means then given message in the stream record that finds key assignments to equate, change step 2); Otherwise, give (i%N) individual process or thread in the stream table management devices with message, change step 4);
2) carry out stream mode and follow the tracks of operation S2.To flow to write down to add and write lock (wlock), then according to agreement under the message (TCP, UDP, ICMP, or the like), the field contents (as flags field in TCP head) relevant with protocol status and message transmissions direction are upgraded the stream record in the message flowstate field, according to the timestamp field that current time renewal stream writes down, upgrade the agetime field (the ageing time intervals that different stream modes is corresponding different) that stream writes down according to the value of current flowstate field.Remove the lock of writing of stream record, give message process device with message then, change step 3);
3) carry out message and handle operation S3.Action field in the reading flow record is handled message according to processing policy information wherein, realizes the corresponding business function.Typical business function comprises the detection of QoS, packet filtering, attack/invasion and defence, network address translation, message forwarding, load balancing, traffic statistics, or the like;
4) with the key assignments of message forward key assignments, handle by the process or the thread of forward key assignments tuple correspondence earlier as stream.At first requester network stream table confirms whether there has been corresponding stream record.If exist, then directly message is given the stream mode tracking means, change step 2); Otherwise, from idle stream record list item buffering area, distribute an idle list item and list item carried out zero clearing.If idle stream record list item buffering area is empty (the allocation table item number reaches threshold value Tr),, change step 6) then with packet loss;
5) carry out the newly-built operation S4 of stream record.Fill in fields such as tuple, starttime in the stream record, timestamp, agetime, flowstate.Further, according to required business function, fill in the action field in the stream record.Then determine the reverse key assignments retuple of stream and be filled into to flow in the record---generally, the computational methods of retuple are:
(resip,redip,resport,redport,proto)=(dip,sip,dport,sport,proto)
Stream record list item is inserted in the stream record chained list of tuple correspondence in the network flow table, then message is given the process or the thread process of retuple correspondence.The process of retuple correspondence or thread are responsible for stream record list item is inserted in the stream record chained list of retuple correspondence in the network flow table, then message are given the stream mode tracking means, change step 3);
6) carry out stream record replacement operation S5.Check the timestamp and the agetime field of each stream record in the current stream record chained list successively, select to have reached or surpassed the stream record of ageing term.If there is not such stream record, then adopt recent minimum use replacement policy (LRU), select the oldest stream record of timestamp in the chained list, perhaps adopt first in first out replacement policy (FIFO), select to be in the stream record of linked list head, change step 7);
7) carry out stream record reclaimer operation S6.Handle by the process or the thread of the forward key assignments tuple correspondence that flows record earlier, delete in the stream record chained list with stream record list item tuple correspondence from the network flow table.Then give the process or the thread of the reverse key assignments retuple correspondence of stream record and handle, delete in the stream record chained list with stream record list item retuple correspondence from the network flow table, be recovered in the idle stream record list item buffering area.Before reclaiming, may also exist some using the message of this list item, in order not influence it in removal process and reclaim later normal use, avoid since reclaim cause synchronization overhead, when reclaiming, do not empty contents in table, but by the time empty again during sub-distribution under this list item, simultaneously idle stream being write down the distributed list item threshold value of list item buffering area is arranged to less than the idle list item number of maximum---because idle stream record list item buffering area takes mode first-in last-out to distribute idle list item, therefore the list item that is recovered can not redistributed away at once, makes the message of current this list item of use successfully to dispose.
In addition, each process of stream table management devices or thread also need regularly to carry out the aging operation of stream record S7, promptly scan that part of network flow table that it is responsible for, and the stream record that meets or exceeds ageing term is reclaimed.Once aging operation is Ta list item of scanning at most, and each aging operation is all proceeded scanning since the place of finishing last time.
The operating procedure of inserting stream record list item in the network flow table of the present invention is:
1) supposes and between stream record list item A in the stream record chained list and C, to insert a new list item B.At first, read the value (being the position of list item C) of the next field of list item A;
2) value that will read is write in the next field of list item B;
3) address of list item B is inserted in the next field of list item A.
Operating procedure of deleting stream record list item in the network flow table of the present invention is:
1) supposes and in stream record chained list, deletion to flow the list item B that writes down between list item A and the C.At first, read the value (being the position of list item C) of the next field of list item B;
2) value that will read is write in the next field of list item A;
3) content (the next field that comprises list item B) of reservation list item B does not empty.Even current like this have along this chained list carry out the reader of query manipulation and just in time arrive list item B, also can because B by from chained list the deletion and not influence its visit back list item.
So, the present invention adopts the dynamic flow sorting technique based on IP five-tuple or other specific identifiers to realize the fine granularity of flow is controlled, adopt a plurality of parallel devices and the inner multi-process of device or threading mechanism to realize the high-speed parallel of flow is handled, by cutting network flow table, it is unique that the stream record chained list person of writing is set, the write command order is to keep the integrality of chained list in insertion of strict regulations chained list and the deletion action process, the a large amount of mutual exclusions and the simultaneous operation that cause owing to parallel processing has been avoided in the measures such as distributed list item threshold value that idle stream record list item buffering area is set, and makes treatment effeciency be further enhanced.The present invention is applicable to various parallel processing environments such as network processing unit, ASIC, FPGA, multi-core processor, symmetric multi processor (smp), software process or thread, has good cross-platform compatibility, extensibility and practicality.
The present invention using, has obtained good effect on the network security processing platform of the processor Network Based of applicant development and gigabit level security gateway, the performance index excellence has realized purpose of the present invention.The present invention has good practicability and popularizing application prospect.
Although disclose specific embodiments of the invention and accompanying drawing for the purpose of illustration, its purpose is to help to understand content of the present invention and implement according to this, but it will be appreciated by those skilled in the art that: without departing from the spirit and scope of the invention and the appended claims, various replacements, variation and modification all are possible.Therefore, the present invention should not be limited to most preferred embodiment and the disclosed content of accompanying drawing, and the scope of protection of present invention is as the criterion with the scope that claims define.

Claims (10)

1.一种对网络流进行分类、状态跟踪和报文处理的装置,该装置包括:1. A device for classifying, state tracking and message processing of network flows, the device comprising: 基于IP五元组信息或者特定标识的动态流分类装置,用于执行动态流分类操作,根据报文包含的IP五元组信息或者特定标识查询网络流表,确定报文所属的网络流,对于找不到对应流记录的报文,交给流表管理装置处理;A dynamic flow classification device based on IP quintuple information or a specific identifier is used to perform dynamic flow classification operations, query the network flow table according to the IP quintuple information or specific identifier contained in the message, and determine the network flow to which the message belongs. If the message corresponding to the flow record cannot be found, it will be handed over to the flow table management device for processing; 流状态跟踪装置,用于执行流状态跟踪操作,根据报文字段内容更新流记录中的流状态、时间戳和老化时间信息;A flow state tracking device, configured to perform a flow state tracking operation, and update the flow state, timestamp and aging time information in the flow record according to the content of the message field; 报文处理装置,用于执行报文处理操作,根据流记录中的处理策略信息对报文作相应的处理;A message processing device, configured to perform a message processing operation, and process the message accordingly according to the processing policy information in the flow record; 流表管理装置,用于执行网络流表中流记录的新建、替换、老化和回收操作;A flow table management device, configured to perform operations of creating, replacing, aging and reclaiming flow records in the network flow table; 网络流表,用于记录网络流信息,流记录采用散列表方式进行组织,采用链表方式解决散列碰撞;以及The network flow table is used to record network flow information, the flow records are organized in a hash table, and the hash collision is solved in a linked list; and 空闲流记录表项缓冲区,用于存储空闲的流记录表项,采用先进后出方式分配空闲表项。The idle flow record entry buffer is used to store idle flow record entries, and the idle entries are allocated in a first-in-last-out manner. 2.根据权利要求1所述的对网络流进行分类、状态跟踪和报文处理的装置,其特征在于:动态流分类装置、流状态跟踪装置、流表管理装置和报文处理装置均包含多个进程或线程,能够进行高速并行处理;流表管理装置的各个进程或线程分别负责网络流表的不同部分。2. The device for classifying, state tracking and message processing of network flows according to claim 1, characterized in that: the dynamic flow classification device, the flow state tracking device, the flow table management device and the message processing device all include multiple Each process or thread can perform high-speed parallel processing; each process or thread of the flow table management device is responsible for different parts of the network flow table. 3.一种对网络流进行分类、状态跟踪和报文处理的方法,具体包括以下步骤:3. A method for classifying, state tracking and message processing of network flows, specifically comprising the following steps: 根据报文包含的IP五元组信息或者特定标识执行动态流分类操作,确定报文所属的网络流;Perform dynamic flow classification operations according to the IP quintuple information contained in the message or a specific identifier to determine the network flow to which the message belongs; 根据报文字段内容执行流状态跟踪操作,更新流记录;Perform flow state tracking operations according to the message field content, and update flow records; 根据流记录中的信息执行报文处理操作,实现相应的业务功能;Execute message processing operations according to the information in the flow record to realize corresponding business functions; 对于在网络流表中查询不到对应流记录的报文,将其交给流表管理装置,经流表管理装置确认不存在对应流记录后,再根据空闲流记录表项缓冲区的情况执行流记录新建操作或者流记录替换操作;For the message whose corresponding flow record cannot be found in the network flow table, it will be handed over to the flow table management device. After the flow table management device confirms that there is no corresponding flow record, it will be executed according to the condition of the idle flow record entry buffer Flow record creation operation or flow record replacement operation; 定时执行流记录老化操作,将已经达到或超过老化时间的流记录从网络流表中删除;Execute the flow record aging operation regularly, and delete the flow records that have reached or exceeded the aging time from the network flow table; 对已经从网络流表中删除的流记录表项执行回收操作,回收到空闲流记录表项缓冲区中。Execute the recovery operation on the flow record entries that have been deleted from the network flow table, and recycle them into the buffer of idle flow record entries. 4.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述动态流分类操作的步骤为:4. The method for classifying, state tracking and message processing of network flows according to claim 3, wherein the steps of the dynamic flow classification operation are: 1)以报文包含的IP五元组信息或者特定标识为键值代入散列函数,以运算结果为索引找到网络流表中对应的散列表项;1) Substituting the IP quintuple information contained in the message or a specific identifier into the hash function as a key value, and using the operation result as an index to find the corresponding hash table item in the network flow table; 2)将报文的键值依次与所述散列表项中的流记录链表头指针字段所指链表中各流记录的键值进行比较:如果找到对应的流记录,则将报文交给流状态跟踪装置执行流状态跟踪操作;否则,根据散列表项的索引将报文交给流表管理装置中对应的进程或线程,流表管理装置确认不存在对应流记录后,再根据空闲流记录表项缓冲区的情况执行流记录新建操作或者流记录替换操作。2) The key value of the message is compared with the key value of each flow record in the linked list pointed to by the flow record linked list head pointer field in the hash table item: if the corresponding flow record is found, the message is given to the flow record The state tracking device executes the flow state tracking operation; otherwise, according to the index of the hash table entry, the message is delivered to the corresponding process or thread in the flow table management device, and the flow table management device confirms that there is no corresponding flow record, and then records the message according to the idle flow record In the case of the entry buffer, execute the operation of creating a new flow record or replacing the operation of a flow record. 5.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述流状态跟踪操作,更新流记录的步骤为:5. The method for classifying, state tracking and message processing of network flows according to claim 3, characterized in that, the flow state tracking operation, the steps of updating flow records are: 1)将报文对应的流记录加写锁,或者采用互斥写原子指令对流记录内容进行修改;1) Add a write lock to the flow record corresponding to the message, or modify the content of the flow record by using mutually exclusive write atomic instructions; 2)根据报文所属协议、报文中与协议状态相关的字段内容以及报文传输方向更新流记录中的流状态信息;2) Update the flow state information in the flow record according to the protocol to which the message belongs, the field content related to the protocol state in the message, and the transmission direction of the message; 3)用当前时间更新流记录中的时间戳信息;3) Update the timestamp information in the flow record with the current time; 4)根据当前流状态更新流记录中的老化时间信息;4) Update the aging time information in the flow record according to the current flow state; 5)解除流记录的写锁;6)将报文交给报文处理装置,执行报文处理操作。5) release the write lock of the flow record; 6) deliver the message to the message processing device, and execute the message processing operation. 6.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述报文处理操作的步骤为:6. The method for classifying, state tracking and message processing of network flows according to claim 3, wherein the steps of the message processing operation are: 1)读取报文对应的流记录中的处理策略信息;1) Read the processing policy information in the flow record corresponding to the message; 2)根据处理策略对报文进行处理,实现相应的业务功能。2) Process the message according to the processing policy to realize the corresponding service function. 7.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述流记录新建操作的步骤为:7. The method for classifying, state tracking and message processing of network flows according to claim 3, wherein the step of creating a new operation of the flow record is: 1)以报文的键值作为流的正向键值,先由正向键值对应的进程或线程进行处理;1) The key value of the message is used as the forward key value of the flow, and the process or thread corresponding to the forward key value is first processed; 2)填写流记录中的正向键值、开始时间、时间戳、老化时间和流状态信息;2) Fill in the forward key value, start time, timestamp, aging time and flow status information in the flow record; 进一步的,根据所需业务功能,填写流记录中的处理策略信息;接着确定流的反向键值并填写到流记录中;Further, according to the required business function, fill in the processing policy information in the flow record; then determine the reverse key value of the flow and fill it in the flow record; 3)将流记录表项插入网络流表中正向键值对应的流记录链表中;3) Insert the flow record entry into the flow record linked list corresponding to the forward key value in the network flow table; 4)将流记录表项插入网络流表中反向键值对应的流记录链表中,然后将报文交给流状态跟踪装置,执行流状态跟踪操作。4) Insert the flow record entry into the flow record linked list corresponding to the reverse key value in the network flow table, and then deliver the message to the flow state tracking device to perform the flow state tracking operation. 8.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述流记录替换操作的步骤为:8. The method for classifying, state tracking and message processing of network flows according to claim 3, wherein the steps of the flow record replacement operation are: 1)依次检查当前流记录链表中各流记录的时间戳和老化时间信息,选择已经达到或者超过老化期限的流记录;如果没有这样的流记录,则采用近期最少使用替换策略,选择链表中时间戳最老的流记录,或者采用先进先出替换策略,选择处于链表头的流记录;1) Check the timestamp and aging time information of each flow record in the current flow record linked list in turn, and select the flow record that has reached or exceeded the aging period; if there is no such flow record, use the least recently used replacement strategy and select the time in the linked list Stamp the oldest flow record, or use the first-in-first-out replacement strategy to select the flow record at the head of the linked list; 2)对选中的流记录执行表项回收操作。2) Execute table item recovery operation on the selected flow records. 9.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述流记录老化操作的步骤为:9. The method for classifying, state tracking and message processing of network flows according to claim 3, wherein the steps of the flow record aging operation are: 1)流表管理装置的每个进程或线程定期扫描其负责的那部分网络流表,将达到或超过老化期限的流记录进行回收;1) Each process or thread of the flow table management device regularly scans the part of the network flow table it is responsible for, and recycles the flow records that have reached or exceeded the aging period; 2)为控制老化操作的开销,需设置一次操作中最多扫描表项的阈值,每次老化操作都从上次结束的地方开始继续进行扫描。2) In order to control the overhead of the aging operation, it is necessary to set the threshold of the most scanned table items in one operation, and each aging operation will continue to scan from the place where it ended last time. 10.根据权利要求3所述的对网络流进行分类、状态跟踪和报文处理的方法,其特征在于,所述流记录表项回收操作的步骤为:10. The method for classifying, state tracking and message processing of network flows according to claim 3, characterized in that, the step of reclaiming the flow record entry is: 1)先由流记录正向键值对应的进程或线程进行处理,将流记录表项从网络流表中正向键值对应的流记录链表中删除;1) The process or thread corresponding to the forward key value of the flow record is first processed, and the flow record entry is deleted from the flow record linked list corresponding to the forward key value in the network flow table; 2)接着交给流记录反向键值对应的进程或线程进行处理,将流记录表项从网络流表中反向键值对应的流记录链表中删除,并回收到空闲流记录表项缓冲区中;在进行回收前,如果还存在一些正在使用该表项的报文,为了不影响其在回收过程中以及回收以后的正常使用,避免由于回收引起的的同步开销,在回收时不清空表项内容,而是等到该表项下次分配时再清空,同时将空闲流记录表项缓冲区的可分配表项阈值设置成小于最大空闲表项数。2) Then hand it over to the process or thread corresponding to the reverse key value of the flow record for processing, delete the flow record entry from the flow record linked list corresponding to the reverse key value in the network flow table, and recycle it to the idle flow record entry buffer In the zone; before recycling, if there are still some messages using this entry, in order not to affect its normal use during and after recycling, and to avoid synchronization overhead caused by recycling, it is not cleared during recycling Instead, wait until the next allocation of the entry before clearing it, and at the same time set the allocatable entry threshold of the idle flow record entry buffer to be less than the maximum number of idle entries.
CNB2005100864404A 2005-09-19 2005-09-19 Network flow classifying, state tracking and message processing device and method Expired - Fee Related CN100459574C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2005100864404A CN100459574C (en) 2005-09-19 2005-09-19 Network flow classifying, state tracking and message processing device and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2005100864404A CN100459574C (en) 2005-09-19 2005-09-19 Network flow classifying, state tracking and message processing device and method

Publications (2)

Publication Number Publication Date
CN1937574A CN1937574A (en) 2007-03-28
CN100459574C true CN100459574C (en) 2009-02-04

Family

ID=37954848

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2005100864404A Expired - Fee Related CN100459574C (en) 2005-09-19 2005-09-19 Network flow classifying, state tracking and message processing device and method

Country Status (1)

Country Link
CN (1) CN100459574C (en)

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101170563A (en) * 2007-11-30 2008-04-30 杭州华三通信技术有限公司 A method and device for matching message rule
CN101252541B (en) * 2008-04-09 2011-05-04 中国科学院计算技术研究所 Method for establishing network flow classified model and corresponding system thereof
CN101599894B (en) * 2008-06-04 2013-01-30 华为技术有限公司 Processing method, device and system for message with clock information
CN101420371B (en) * 2008-07-03 2010-12-01 江苏华丽网络工程有限公司 A dynamic function support method and system for ASIC converged network equipment
CN101610209B (en) * 2008-11-28 2011-08-03 北京网康科技有限公司 Method and device for multi-core parallel concurrent processing of network traffic flows
CN101753332B (en) * 2008-12-03 2012-08-22 财团法人资讯工业策进会 Event relation analyzing method and system
CN101572670B (en) * 2009-05-07 2011-08-10 成都市华为赛门铁克科技有限公司 Data packet processing method based on flow table, device and network system
CN101635676B (en) * 2009-08-31 2011-07-27 杭州华三通信技术有限公司 Message processing method and network equipment
CN101729240B (en) * 2009-11-13 2012-10-10 北京中创信测科技股份有限公司 Method and device for realizing time synchronization
CN101827021B (en) * 2010-03-16 2012-11-28 杭州华三通信技术有限公司 Method, device and system for classifying and marking QoS
CN102385588B (en) * 2010-08-31 2014-08-06 国际商业机器公司 Method and system for improving performance of data parallel insertion
CN102004673A (en) * 2010-11-29 2011-04-06 中兴通讯股份有限公司 Processing method and system of multi-core processor load balancing
CN103380600B (en) * 2011-02-17 2016-11-09 日本电气株式会社 Network system and network flow tracking method
CN102761517B (en) * 2011-04-25 2015-06-24 工业和信息化部电信传输研究所 Content reduction method for high-speed network
US8625422B1 (en) * 2012-12-20 2014-01-07 Unbound Networks Parallel processing using multi-core processor
CN103023728B (en) * 2013-01-15 2016-03-02 中国人民解放军信息工程大学 flow monitoring method
CN103748842B (en) * 2013-06-26 2017-04-12 华为技术有限公司 Method, device and route device for forwarding data packages
CN104348716B (en) * 2013-07-23 2018-03-23 新华三技术有限公司 A kind of message processing method and equipment
US9979613B2 (en) 2014-01-30 2018-05-22 Hewlett Packard Enterprise Development Lp Analyzing network traffic in a computer network
CN104009924B (en) * 2014-05-19 2017-04-12 北京东土科技股份有限公司 Message processing method and device based on TCAM and FPGA
CN106330582B (en) * 2015-06-18 2020-11-20 中兴通讯股份有限公司 Method and device for detecting number of shared internet access mobile terminals
CN106330694A (en) * 2015-06-26 2017-01-11 中兴通讯股份有限公司 Method and device for realizing flow table traversal business
CN108092914B (en) * 2016-11-21 2022-03-04 华为技术有限公司 Network traffic load balancing scheduling method and device
CN107317759A (en) * 2017-06-13 2017-11-03 国家计算机网络与信息安全管理中心 A kind of thread-level dynamic equalization dispatching method of network interface card
CN107508757B (en) * 2017-08-15 2021-11-19 网宿科技股份有限公司 Multi-process load balancing method and device
CN107608773B (en) * 2017-08-24 2020-08-04 阿里巴巴集团控股有限公司 Task concurrent processing method and device and computing equipment
CN109831394B (en) * 2017-11-23 2021-07-09 华为技术有限公司 Data processing method, terminal and computer storage medium
CN108243107B (en) * 2018-01-30 2020-11-20 盛科网络(苏州)有限公司 Method and device for dynamically adjusting hardware table entry aging period
CN110471944B (en) * 2018-05-11 2024-09-20 北京京东尚科信息技术有限公司 Index statistics method, system, equipment and storage medium
CN111107042B (en) * 2018-10-26 2021-03-09 广州汽车集团股份有限公司 Message parsing method, device, computer equipment and storage medium
CN109410445A (en) * 2018-10-31 2019-03-01 湖南金码智能设备制造有限公司 A kind of method and self-help shopping system of multiple unit cabinets of selling goods
CN110851334B (en) * 2019-11-19 2024-06-14 深圳市网心科技有限公司 Flow statistics method, electronic equipment, system and medium
CN113347090B (en) * 2020-02-18 2023-06-20 华为技术有限公司 Message processing method, forwarding equipment and message processing system
CN112311895B (en) * 2020-11-12 2022-10-11 中国电子科技集团公司第五十四研究所 Transparent mode TCP flow load balancing method and device based on SDN
CN112667375B (en) * 2020-12-22 2024-07-26 新讯数字科技(杭州)有限公司 Task scheduling method and system based on big data service
CN112749028B (en) * 2021-01-11 2024-06-07 科大讯飞股份有限公司 Network traffic processing method, related equipment and readable storage medium
CN113518130B (en) * 2021-08-19 2023-03-24 北京航空航天大学 Packet burst load balancing method and system based on multi-core processor
CN115150331B (en) * 2022-09-02 2022-11-25 无锡沐创集成电路设计有限公司 Information processing method, information processing device, electronic device, and medium
CN117478767B (en) * 2023-10-24 2025-03-21 奥特酷智能科技(南京)有限公司 A method for accurately tracking specified RTPS messages in DDS communication

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5379297A (en) * 1992-04-09 1995-01-03 Network Equipment Technologies, Inc. Concurrent multi-channel segmentation and reassembly processors for asynchronous transfer mode
JP2003298638A (en) * 2002-04-05 2003-10-17 Matsushita Electric Ind Co Ltd Apparatus and method for transmitting packet
US20040085964A1 (en) * 2002-10-29 2004-05-06 Janne Vaananen Method and apparatus for scheduling available link bandwidth between packet-switched data flows
CN1612527A (en) * 2003-10-28 2005-05-04 华为技术有限公司 Data service information collecting device and charging method using same
CN1633111A (en) * 2005-01-14 2005-06-29 中国科学院计算技术研究所 High-speed Network Traffic Classification Method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5379297A (en) * 1992-04-09 1995-01-03 Network Equipment Technologies, Inc. Concurrent multi-channel segmentation and reassembly processors for asynchronous transfer mode
JP2003298638A (en) * 2002-04-05 2003-10-17 Matsushita Electric Ind Co Ltd Apparatus and method for transmitting packet
US20040085964A1 (en) * 2002-10-29 2004-05-06 Janne Vaananen Method and apparatus for scheduling available link bandwidth between packet-switched data flows
CN1612527A (en) * 2003-10-28 2005-05-04 华为技术有限公司 Data service information collecting device and charging method using same
CN1633111A (en) * 2005-01-14 2005-06-29 中国科学院计算技术研究所 High-speed Network Traffic Classification Method

Also Published As

Publication number Publication date
CN1937574A (en) 2007-03-28

Similar Documents

Publication Publication Date Title
CN100459574C (en) Network flow classifying, state tracking and message processing device and method
US20210367887A1 (en) Flow classification apparatus, methods, and systems
US7525958B2 (en) Apparatus and method for two-stage packet classification using most specific filter matching and transport level sharing
US20130304926A1 (en) Concurrent linked-list traversal for real-time hash processing in multi-core, multi-thread network processors
Luo et al. Hop: Heterogeneity-aware decentralized training
Ramabhadran et al. Efficient implementation of a statistics counter architecture
US7177276B1 (en) Pipelined packet switching and queuing architecture
US7372857B1 (en) Methods and apparatus for scheduling tasks
CN101350771A (en) Three-state content addressable memory entry sort-free storage method and system thereof
US20050083935A1 (en) Method and apparatus for two-stage packet classification using most specific filter matching and transport level sharing
CN102487374B (en) Access control list realization method and apparatus thereof
CN107566206A (en) A kind of flow-measuring method, equipment and system
CN101650730B (en) Method and system for mining frequent items with weights in data stream
EP2676189B1 (en) Sorting
CN111988231B (en) Mask quintuple rule matching method and device
CN100448225C (en) A device and method for realizing dynamic flow classification without IP fragment reassembly
CN102263701A (en) Queue adjustment method and device
Daly et al. Tuplemerge: Building online packet classifiers by omitting bits
Canini et al. Experience with high-speed automated application-identification for network-management
US20050262294A1 (en) Method for policy matching using a hybrid TCAM and memory-based scheme
CN105553695A (en) IP data flow management method based on two-level bidirectional Hash list
CN111881165B (en) Data aggregation method and device and computer readable storage medium
CN118175113A (en) TCP (transmission control protocol) reorganization ordering method and device based on fast and slow paths and intrusion detection and defense system
CN114640641B (en) Flow-aware OpenFlow flow table elastic energy-saving searching method
Xie et al. Towards capacity-adjustable and scalable quotient filter design for packet classification in software-defined networks

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090204

Termination date: 20140919

EXPY Termination of patent right or utility model