JP7535951B2 - COLLECTOR DEVICE, NETWORK MONITORING METHOD, PROGRAM, AND NETWORK MONITORING SYSTEM - Google Patents

Description

The present invention relates to a collector device for monitoring a network, a network monitoring method, a program, and a network monitoring system.

To operate and manage increasingly complex networks, it is important to constantly monitor the status of traffic flowing through the network and respond immediately if a problem occurs. Technologies such as NetFlow (see Non-Patent Document 1) are used to check the status and normality of network traffic.

NetFlow is a communication protocol for acquiring flow statistics of network traffic by network relay devices such as routers and switches, or dedicated devices that receive mirrored traffic from network relay devices (hereafter referred to as "sensors"), and transmitting the information to a traffic analysis device (hereafter referred to as "collector"). The collector that receives the flow statistics can visualize and analyze traffic for various purposes. Systems for collecting and analyzing such flow statistics are disclosed in Patent Document 1 and Patent Document 2.

JP 2018-38062 A JP 2016-144153 A

RFC3954 “Cisco Systems NetFlow Services Export Version 9”, Cisco Systems, October 2004

In a method in which a collector collects and analyzes flow statistics acquired by a sensor using a protocol such as NetFlow, situations can arise in which the flow statistics collected from the sensor do not represent the actual traffic situation. For example, a sensor may be unable to send statistical information due to a malfunction such as a breakdown, or an increase in network traffic or an increase in the number of statistical information collection points may cause the process of assembling and sending packets that store the flow statistics to be unable to keep up, resulting in a loss of flow statistics. As a result, the collector is faced with the problem of being unable to perform accurate monitoring or analysis.

Specifically, for example, in a network system consisting of multiple sensors installed at several flow statistics acquisition points within a network and one collector that collects flow statistics from those sensors, if one of the sensors stops due to a malfunction or other reason, the flow statistics from the stopped sensor will be missing. However, it is difficult for the collector to distinguish whether the lack of flow statistics means that network traffic has actually stopped flowing or that a problem has occurred with the sensor. Therefore, when the collector analyzes traffic and performs monitoring to issue an alarm based on the traffic volume, there is a problem in that it may erroneously determine that traffic has decreased even though the actual traffic volume has not changed, and issue an unnecessary alarm.

In addition, if the volume of network traffic being monitored increases due to some problem (for example, an external network attack or abnormal traffic caused by a malware infection on a terminal within the network), but the volume exceeds the processing capacity of the sensor that acquires the flow statistics, and correct flow statistics cannot be sent to the collector, there is an issue that the collector, which analyzes traffic and issues alarms based on the traffic volume, does not issue an alarm for an event that should actually be issued.

The object of the present invention is to appropriately generate alarms when monitoring network traffic using NetFlow information. In other words, in a network monitoring system in which traffic flow statistics are acquired by a sensor and collected by a collector for analysis, etc., the object is to provide a network monitoring system that prevents the collector from issuing unnecessary alarms or being unable to issue necessary alarms even if a deviation occurs between the flow statistics collected by the collector and the actual traffic situation due to an abnormality in the sensor or limitations in the flow statistics acquisition capacity.

A representative example of the invention disclosed in the present application is as follows: That is, a collector device that monitors network traffic using NetFlow information includes a NetFlow information receiving unit that acquires NetFlow information from a sensor device that receives network traffic and generates NetFlow information, an information receiving unit that receives alive information of the sensor device from the sensor device as information for determination , a threshold information correction unit that uses the received alive information to determine whether a statistical value of the received NetFlow information is correct, and if it is determined that the statistical value of the received NetFlow information is incorrect, updates threshold information used to monitor network traffic using sensor threshold information determined for each sensor device, and a threshold determination unit that uses the updated threshold information to determine whether network traffic is normal .

According to one aspect of the present invention, an alarm can be generated appropriately even if a sensor malfunctions or excessive traffic occurs that exceeds the sensor's capacity.

FIG. 1 illustrates an example of a configuration of a network monitoring system according to a first embodiment. FIG. 2 is a block diagram showing a configuration example of a sensor according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of a collector according to the first embodiment. FIG. 4 is a diagram illustrating an operation sequence of a collector in the first embodiment. 4 is a diagram illustrating an example of the configuration of a life and death information accumulation unit according to the first embodiment; FIG. 4 is a diagram illustrating an example of the configuration of a sensor/threshold value association information storage unit according to the first embodiment; FIG. 4 is a diagram illustrating an example of the configuration of a threshold information storage unit according to the first embodiment; 11 is a diagram showing an example of a traffic volume of traffic derived based on NetFlow information in a state in which all sensors are operating normally in the first embodiment; FIG. 11 is a diagram illustrating an example of a traffic volume of traffic derived based on NetFlow information in a state where some sensors are stopped in the first embodiment. FIG. FIG. 11 is a block diagram showing an example of the configuration of a sensor according to a second embodiment. FIG. 11 is a block diagram showing an example of the configuration of a collector according to a second embodiment. FIG. 11 is a diagram showing an operation sequence of a collector in the second embodiment. FIG. 13 is a diagram illustrating an example of the configuration of a NetFlow information storage unit in the second embodiment. FIG. 11 illustrates an example of the configuration of a packet statistics accumulation unit according to the second embodiment; FIG. 13 is a diagram illustrating an example of the configuration of a corrected NetFlow information storage unit in the second embodiment.

<Embodiment 1>
FIG. 1 is a diagram illustrating an example of the configuration of a network monitoring system according to the first embodiment.

The network monitoring system is composed of a network 100 to be monitored, network devices 200 included in the network 100 to be monitored, a sensor 300 that acquires statistical information on traffic, a collector 400 that collects and analyzes the statistical information, and a management terminal 500. The network devices 200 are included in the network 100 to be monitored, and transfer data in the network 100 to be monitored. The sensor 300 is connected to each network device 200 to be monitored, and acquires statistical information on traffic flowing through the network 100. The collector 400 is connected to the sensor 300, and collects and analyzes the statistical information generated by the sensor 300. The management terminal 500 is a computer operated by an administrator to refer to the analysis information output from the collector 400, etc.

The first embodiment is an example of a configuration that allows the collector 400 to recognize an abnormality in the sensor 300 (e.g., a stop due to a malfunction) and aims to suppress the generation of unnecessary alarms.

The number of network devices 200, sensors 300, and collectors 400 may be any number greater than one, and any connection means may be used between them. When there are multiple network devices 200, sensors 300, and collectors 400 and it is necessary to distinguish between them in the description, they are distinguished by reference characters with numbers added, such as network device 200-1 for network device 200. The configuration example in Figure 1 shows an example of a system with n network devices 200 and n sensors 300, and one collector 400.

Figure 2 is a block diagram showing an example configuration of the sensor 300 in the first embodiment.

The sensor 300 has a CPU 340, a memory 310, a storage unit 320, and multiple network interfaces 330-1 and 330-2 (network interfaces are abbreviated as NW-IF in the figure). The network interface 330-1 is connected to the network device 200, and the network interface 330-2 is connected to the collector 400.

The CPU 340 is a computing device that executes the programs stored in the memory 310. Note that some of the processing performed by the CPU 340 executing the programs may be executed by another computing device (e.g., a hardware computing device such as an FPGA or ASIC).

Memory 310 includes ROM, which is a non-volatile storage element, and RAM, which is a volatile storage element. ROM stores unchanging programs (e.g., BIOS) and the like. RAM is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by CPU 340 and data used when executing the programs. Specifically, memory 310 stores programs executed by CPU 330, namely, programs for constituting traffic receiving unit 311, NetFlow information generating unit 312, NetFlow information transmitting unit 313, alive/dead information generating unit 314, alive/dead information transmitting unit 315, MIB (Management Information Base) response unit 316, and MIB request receiving/response transmitting unit 317.

The storage unit 320 is, for example, a large-capacity, non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD). The storage unit 320 also stores data used by the CPU 340 when executing a program (for example, a flow information accumulation unit 321 that holds acquired traffic statistical information) and the program executed by the CPU 340. That is, the program is read from the storage unit 320, loaded into the memory 310, and executed by the CPU 340 to realize each function of the sensor 300.

In the configuration of FIG. 2, the process in which the sensor 300 acquires statistical information about traffic on the network 100 and transmits it to the collector 400 will be described.

The network device 200 mirrors the traffic data flowing through the network 100 and transmits a copy to the sensor 300. The sensor 300 receives the data transmitted from the network device 200 via the network interface 330-1.

The traffic receiving unit 311 receives traffic data received by the network interface 330-1, generates flow information, which is a type of statistical information on that traffic, and stores the generated flow information in the flow information storage unit 321. Here, flow information refers to, for example, the number of packets and the amount of data (the sum of the lengths of the individual packets observed) observed for each flow identified by the source address and destination address contained in the header portion of the packets that make up the received traffic.

The NetFlow information generation unit 312 collects the flow information stored in the flow information storage unit 321 at regular intervals and converts it into NetFlow information that conforms to the packet format defined by the NetFlow protocol. The NetFlow information transmission unit 313 receives the converted NetFlow information from the NetFlow information generation unit 312 and transmits it from the network interface 330-2 to the collector 400 in accordance with the provisions of the NetFlow protocol.

Next, we will explain the processing on the sensor 300 side that is necessary for the collector 400 to recognize an abnormality in the sensor 300.

First, the alive information generating unit 314 generates alive information indicating the operating status of the sensor 300. Here, alive information is information indicating whether the sensor 300 is operating normally. For example, when the alive information generating unit 314 recognizes that the sensor 300 is operating normally through its own fault monitoring function, it generates information indicating normal operation, and when it recognizes that a partial fault has caused it to be unable to obtain statistical information or to transmit NetFlow information to the collector 400, it generates information indicating that transmission of NetFlow information has been stopped.

The alive/dead information sending unit 315 stores the alive/dead information generated by the alive/dead information generating unit 314 in a packet of a specified format and sends it to the collector 400 via the network interface 330-2. The alive/dead information should be sent periodically at a specified time interval regardless of changes in the alive/dead state. If the entire sensor 300 stops and it becomes impossible to send alive/dead information, the collector 400 monitors the reception of alive/dead information, and becomes able to recognize an abnormality in the sensor 300 when the alive/dead information packet is not received for a certain period of time or more.

Alternatively, the alive/dead information may not be sent spontaneously from the alive/dead information sending unit 315, but may be sent in response to an MIB request sent by the collector 400 to obtain alive/dead information. In this case, the MIB request receiving/response sending unit 317 receives an MIB request from the collector 400 using a protocol such as SNMP (Simple Network Management Protocol) via the network interface 330-2, and transmits the received MIB request to the MIB response unit 316. The MIB response unit 316 recognizes that the content of the MIB request is to obtain alive/dead information, and obtains alive/dead information from the alive/dead information generating unit 314. The MIB request receiving/response sending unit 317 transmits the alive/dead information obtained by the alive/dead information generating unit 314 from the network interface 330-2 to the collector 400 as a response to the MIB request. Even with the MIB request/response method, the collector 400 can recognize an abnormality in the sensor 300 if there is no response to the request.

The above is the configuration and operation of the sensor 300 in this embodiment.

Next, we will explain the configuration and operation of the collector 400 in this embodiment.

Figure 3 is a block diagram showing an example configuration of the collector 400 in the first embodiment.

The collector 400 has a CPU 440, memory 410, a storage unit 420 such as a hard disk or SSD, and multiple network interfaces 430-1 and 430-2 (network interfaces are abbreviated as NW-IF in the figure). Network interface 430-1 is connected to the sensor 300, and network interface 430-2 is connected to the management terminal 500.

The CPU 440 is a computing device that executes the programs stored in the memory 410. Note that some of the processing performed by the CPU 440 executing the programs may be executed by another computing device (e.g., a hardware computing device such as an FPGA or ASIC).

The memory 410 includes a ROM, which is a non-volatile storage element, and a RAM, which is a volatile storage element. The ROM stores unchanging programs (e.g., BIOS) and the like. The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by the CPU 440 and data used when the programs are executed. Specifically, the memory 410 stores programs executed by the CPU 440, namely, programs for constituting the NetFlow information receiving unit 411, the alive information receiving unit 412, the MIB request transmission/response receiving unit 413, the threshold information correction unit 414, the NetFlow threshold determination unit 415, and the Syslog/Trap transmission unit 416.

The storage unit 420 is, for example, a large-capacity, non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD). The storage unit 420 also stores data used by the CPU 440 when executing a program (for example, the NetFlow information accumulation unit 421, the alive/dead information accumulation unit 422, the sensor/threshold related information holding unit 423, and the threshold information holding unit 424), and the program executed by the CPU 440. That is, the program is read from the storage unit 420, loaded into the memory 410, and executed by the CPU 440 to realize each function of the collector 400.

In the configuration of Figure 3, the process by which the collector 400 analyzes the NetFlow information received from the sensor 300 is described below.

First, the network interface 430-1 receives a packet containing NetFlow information transmitted from the sensor 300. Then, the NetFlow information receiving unit 411 extracts the NetFlow information from the packet received by the network interface 430-1 and stores it in the NetFlow information storage unit 421.

Next, the collector 400 analyzes the received NetFlow information. In this embodiment, an analysis process that monitors the increase or decrease in traffic volume based on a pre-specified threshold will be described as an example.

The NetFlow threshold determination unit 415 derives the traffic volume for each flow from the NetFlow information stored in the NetFlow information storage unit 421, and compares the derived traffic volume for each flow with the threshold information stored in the threshold information storage unit 424. An example of the threshold information set in the threshold information storage unit 424 will be described later.

If the comparison determines that the traffic is abnormal, such as the traffic volume exceeding a predetermined upper threshold or falling below a predetermined lower threshold, the determination result is sent to the Syslog/Trap sending unit 416. The Syslog/Trap sending unit 416 sends information indicating the occurrence of an abnormality to the management terminal 500 via the network interface 430-2. The management terminal 500 makes the administrator aware of the occurrence of an abnormality by displaying the determination result included in the information indicating the occurrence of an abnormality, for example, on a management screen such as a display.

Here, we further explain the threshold information used in the traffic analysis process in this embodiment.

Figure 6 shows an example of the configuration of the sensor/threshold related information storage unit 423.

The sensor/threshold related information storage unit 423 is composed of a table that stores threshold information that is referenced when determining the normality of traffic.

The sensor/threshold related information storage unit 423 has multiple entries, and each entry includes a target flow condition 4231 that specifies the flow in the traffic to be monitored, sensor information 4232 that is the identifier of the sensor 300 that is the source of the statistical information, and information that associates an upper threshold 4233 and a lower threshold 4234 of the traffic volume for the flow of the target flow condition 4231 acquired by the sensor 300 indicated by the sensor information 4232.

The target flow condition 4231 is, for example, a condition in which the traffic to be monitored conforms to a standard TCP/IP protocol, the source IP address in the IP header is a specific value, or the destination port number in the TCP header indicating the type of application service is a specific value. The sensor information 4232 is, for example, the IP address of the sensor 300. The upper threshold 4233 and the lower threshold 4234 are a range of values that are determined to be normal operation of the statistical information to be analyzed, and are expressed, for example, by the amount of traffic per hour in units of bytes/sec.

In the sensor/threshold related information storage unit 423, appropriate values are preset by the administrator through the management terminal 500 based on the purpose of monitoring and information transferred over the network, such as the size of the network to be monitored, the type of application being used, and the placement of the sensors 300. Figure 6 shows an example in which different upper and lower thresholds are set for each of three sensors 300 identified by the sensor information for the same target flow condition "Condition A."

Figure 7 shows an example of the configuration of the threshold information storage unit 424.

As described above, the threshold information storage unit 424 is configured as a table that stores threshold information that the NetFlow threshold determination unit 415 directly references when making its determination.

The threshold information storage unit 424 has multiple entries, and each entry includes a target flow condition 4241 that specifies the traffic to be analyzed, and information that associates an upper threshold 4243 and a lower threshold 4244 of the traffic volume for the flow of the target flow condition 4241.

The threshold information storage unit 424 stores a threshold for the total traffic volume measured by multiple sensors 300 that measure flows that satisfy the target flow condition 4241.

The case where the target flow condition is "Condition A" will be explained as an example using the actual numerical examples shown in Figures 6 and 7. Traffic that satisfies "Condition A" is measured by three sensors 300 as shown in Figure 6, and the sum of the upper threshold 4233 values, 21,000,000, and the sum of the lower threshold 4234 values, 12,000,000, are stored in the upper threshold 4243 and lower threshold 4244 of the entry in the threshold information storage unit 424 in Figure 7 where the target flow condition 4241 is "Condition A". These values are first calculated and stored as initial values when the values are set in the sensor/threshold related information storage unit 423.

Furthermore, when a malfunction such as a stoppage of the sensor 300 occurs, the collector 400 recalculates and corrects the threshold value stored in the threshold information storage unit 424 so as not to erroneously determine a traffic abnormality. The operation of the collector 400 for threshold correction will be described below. Note that the process described here is executed in parallel, independently of the NetFlow information reception process and threshold determination process by the NetFlow receiving unit 441 and the NetFlow threshold determination unit 415 described above.

Figure 4 is a diagram showing the operational sequence in the collector 400 in the first embodiment from receiving alive information from the sensor 300 to storing the corrected threshold information in the threshold information storage unit 424.

First, the sensor 300 transmits alive/dead information to the collector 400. The alive/dead information receiving unit 412 of the collector 400 receives the alive/dead information through the network interface 430-1 (step S601), and updates the alive/dead information storage unit 422 based on the received alive/dead information (step S602). Note that an alive/dead monitoring device that collects the operating status of the sensor 300 may be provided, and the alive/dead information receiving unit 412 may receive the alive/dead information of the sensor 300 from the alive/dead monitoring device, rather than receiving the alive/dead information from the sensor 300.

Figure 5 is a diagram showing an example of the configuration of the alive/dead information accumulation unit 422. The alive/dead information accumulation unit 422 has multiple entries, and each entry includes information in which sensor information 4221, which is an identifier of the sensor 300 that sent the alive/dead information, alive/dead status 4222 indicating the state of the sensor 300 indicated by the sensor information 4221, and latest reception time 4223 indicating the time when the latest alive/dead information was received from the sensor 300 indicated by the sensor information 4221 are associated. In this embodiment, the sensor information 4221 stores the IP address of the sensor 300, and the alive/dead status 4222 stores "1" when the sensor 300 indicated by the sensor information 4221 is operating normally, and "0" when an abnormality has occurred and the sensor 300 cannot transmit NetFlow information.

In step S602, the alive/dead information accumulation unit 422 is updated as follows:

First, the alive/dead information receiving unit 412 searches for an entry in the alive/dead information storage unit 422 where the source IP address of the received alive/dead information is the same as the sensor information 4221, sets the alive/dead status 4222 of that entry to "0" or "1", and stores the reception time of the alive/dead information in the latest reception time 4223. If no entry in the alive/dead information storage unit 422 where the source IP address of the received alive/dead information is the same as the sensor information 4221 is found, a new entry is created.

In addition, in step S602, the alive/dead information receiving unit 412 checks the value of the latest received time 4223 of each entry in the alive/dead information storage unit 422, and if there is an entry whose difference between the value of the latest received time 4223 and the current time is equal to or greater than a predetermined value, sets the alive/dead status 4222 of that entry to "0." This is a process for recognizing an abnormality in the sensor 300 caused by the periodic transmission of alive/dead information being stopped.

Returning to the sequence diagram in FIG. 4, the explanation will be continued. Next, if there is an entry whose value of the alive/dead status 4222 has changed due to the update of the alive/dead information storage unit 422 in step 602, i.e., if there is a sensor 300 whose alive/dead status has changed, the alive/dead information receiving unit 412 notifies the threshold information correcting unit 414 of the change in the alive/dead information (step S603).

When the threshold information correction unit 414 receives the notification in step S603, it acquires alive information from the alive information accumulation unit 422 (step S604), and further acquires the threshold information of the sensor 300 stored in each entry of the sensor/threshold association information holding unit 423 (step S605).

Next, the threshold information correction unit 414 determines whether the received NetFlow information is correct (i.e., whether the NetFlow information is likely to be incorrect because the sensor 300 is not operating) based on the alive/dead information acquired in step S604, and if there is a possibility that the NetFlow information is incorrect, it recalculates appropriate threshold information based on the alive/dead status of the sensor 300 based on the threshold information acquired in step S605 (step S606).

Here, as an example of recalculating threshold information, we will show a specific example of calculation based on the numerical examples of various information shown in Figures 5 and 6.

First, consider the case where the alive status information storage unit 422 is updated by the processing in step S602, and as a result, the alive status 4222 of the sensor 300 (IP address 192.168.0.1) with entry number 1 changes to "0" as shown in FIG. 5, i.e., the NetFlow information cannot be transmitted.

In this state, in step S606, the threshold information correction unit 414 recalculates the upper threshold and the lower threshold using the thresholds of only the sensors 300 whose alive/dead status 4222 in the alive/dead information accumulation unit 422 is set to "1". In the case of the numerical example shown in Figures 5 and 6, only the sensors 300 in entries #2 and #3 in the alive/dead information accumulation unit 422 are operating normally, so the new threshold information for traffic satisfying "Condition A" for these sensors is recalculated using the thresholds of the sensors 300 in entries #2 and #3 in the sensor/threshold related information holding unit 423, with the upper threshold being 15,000,000 and the lower threshold being 9,000,000.

Next, the threshold information correction unit 414 overwrites the threshold information recalculated in step S606 and updates the threshold information storage unit 424 (step S607).

The threshold information correction unit 414 may send a packet including this notification information to the management terminal 500 via the network interface 430-2 to notify the administrator that an abnormality has occurred in the sensor 300 and that the abnormality determination threshold has been recalculated in response to the occurrence of the abnormality. The management terminal 500 displays the received notification information on a management screen such as a display.

As described above, by setting the upper and lower thresholds of the NetFlow information expected from each sensor 300 in advance in the sensor/threshold related information storage unit 423, the upper and lower thresholds for appropriately determining the normality of communication according to the alive status of the sensor 300 can be obtained by processing of the threshold information correction unit 414.

Figures 8 and 9 are diagrams that show the relationship between the change in traffic volume per unit time based on statistical information collected by the collector 400 and the threshold, in order to clearly explain the effect of recalculating the threshold in this embodiment.

Figure 8 shows an example of the traffic volume of traffic that satisfies "Condition A" derived based on the NetFlow information received by the collector 400 of the first embodiment when all sensors 300 are operating normally.

The figure shows the traffic volume 612 that changes over time, and the upper threshold 611 and lower threshold 613 of the traffic volume that satisfy "Condition A" stored in the threshold information storage unit 424 in the initially set state shown in FIG. 7. The traffic volume 612 falls between the upper threshold 611 and the lower threshold 613, and no abnormality has occurred in the traffic volume 612.

Figure 9 shows an example of the traffic volume of traffic that satisfies "Condition A" derived based on the NetFlow information received by the collector 400 of the first embodiment when some of the sensors 300 have stopped operating and some of the NetFlow information cannot be received.

In the figure, the traffic volume 622 that changes with time, the upper threshold 611 and lower threshold 613 of the traffic volume before the threshold information correction unit 414 recalculates the threshold information based on the alive status of the sensor 300, and the upper threshold 621 and lower threshold 623 of the traffic volume after the threshold information correction unit 414 recalculates the threshold information based on the alive status of the sensor 300. The traffic volume 622 is a value less than the actual traffic volume because it does not include information from stopped sensors 300. Similarly, the upper threshold 621 and lower threshold 623 after recalculation are less than the upper threshold 611 and lower threshold 613 before recalculation by the number of stopped sensors 300.

As shown in FIG. 9, if the traffic volume calculated by the collector 400 based on statistical information becomes smaller than the actual traffic volume due to the stoppage of some of the sensors 300, a false positive determination that the traffic volume is abnormal may occur even when the traffic volume is normal, unless the upper and lower thresholds are recalculated. However, it can be seen that such false positives can be suppressed by recalculating the upper and lower thresholds taking into account the stoppage of the sensors 300.

As described above, according to this embodiment, in a network monitoring system consisting of sensors 300 that acquire statistical information and collectors 400 that collect and analyze that statistical information, even if some of the sensors 300 stop due to a malfunction or the like, it is possible to suppress erroneous determination of traffic abnormalities in the analysis process performed by the collectors 400. This reduces the occurrence of network management problems, such as network administrators performing unnecessary countermeasures or being unable to perform necessary problem-solving tasks.

<Embodiment 2>
A configuration example of the network monitoring system in the second embodiment is obtained by replacing the sensor 300 with a sensor 350 according to a second embodiment described later, and the collector 400 with a collector 450 according to a second embodiment described later, in the network monitoring system of the first embodiment shown in Fig. 1. Note that in the second embodiment, the same configurations and functions as those in the first embodiment are denoted by the same reference numerals, and descriptions thereof will be omitted.

In the second embodiment, even if there is a missed transmission of flow statistics due to a problem such as insufficient performance of the sensor 350 or the collector 450, the collector 450 uses the packet statistics information in addition to the flow statistics to estimate and correct the original flow statistics, thereby suppressing problems such as failure to issue an alarm that should have been issued. Here, packet statistics is statistical information that indicates the amount of data in the network traffic, such as the number of packets and the total number of bytes (the sum of the lengths of each packet) of the entire traffic received per unit time, without distinguishing flows as in the flow statistics handled by NetFlow.

Figure 10 is a block diagram showing an example configuration of a sensor 350 in the second embodiment.

The sensor 350 has a CPU 340, a memory 310, a storage unit 320, and multiple network interfaces 330-1 and 330-2 (network interfaces are abbreviated as NW-IF in the figure). The network interface 330-1 is connected to the network device 200, and the network interface 330-2 is connected to the collector 400.

The memory 310 stores programs executed by the CPU 330, namely, programs for constituting a traffic receiving unit 311, a NetFlow information generating unit 312, a NetFlow information transmitting unit 313, a MIB (Management Information Base) response unit 316, a MIB request receiving/response transmitting unit 317, a packet statistics generating unit 318, and a packet statistics transmitting unit 319. The sensor 350 may have a vital information generating unit 314, and a vital information transmitting unit 315.

The memory unit 320 stores data used by the CPU 340 when the CPU 340 executes a program (e.g., the flow information storage unit 321 that holds acquired traffic statistics information, and the packet statistics storage unit 322), and the program executed by the CPU 340.

In the configuration of FIG. 10, the process in which the sensor 350 acquires statistical information about traffic on the network 100 and transmits it to the collector 450 as NetFlow information is the same as the process in which the sensor 300 in the first embodiment transmits statistical information to the collector 400, so a description of this process is omitted.

Furthermore, the sensor 350 transfers information relating to the traffic received by the traffic receiving unit 311 from the network device 200 to the packet statistics generating unit 318. The packet statistics generating unit 318 generates packet statistics based on the traffic information and stores the generated packet statistics in the packet statistics accumulating unit 322. The generation of packet statistics does not require flow identification during measurement, so the processing load is small, and high-precision measurement is possible even when the amount of traffic being measured increases.

Furthermore, the contents stored in the packet statistics accumulation unit 322 are stored in packets of a specified format by the packet statistics transmission unit 319, and are transmitted to the collector 450 via the network interface 330-2 at specified time intervals. Alternatively, the MIB response unit 316 may receive an MIB request by a protocol such as SNMP transmitted from the collector 450 for the purpose of acquiring packet statistics information via the network interface 330-2, and the MIB response unit 316 may interpret the contents of the received MIB request and instruct the MIB request reception/response transmission unit 317 to transmit a response, and the MIB request reception/response transmission unit 317 may transmit the packet statistics information acquired from the packet statistics accumulation unit 322 to the collector 450 via the network interface 330-2 as a response to the MIB request.

Figure 11 is a block diagram showing an example configuration of the collector 450 in the second embodiment.

The collector 450 has a CPU 440, memory 410, a storage unit 420 such as a hard disk or SSD, and multiple network interfaces 430-1 and 430-2 (network interfaces are abbreviated as NW-IF in the figure). Network interface 430-1 is connected to the sensor 300, and network interface 430-2 is connected to the management terminal 500.

The memory 410 stores programs executed by the CPU 440, namely, programs for constituting a NetFlow information receiving unit 411, an MIB request sending/response receiving unit 413, a NetFlow threshold determining unit 415, a Syslog/Trap sending unit 416, a packet statistics receiving unit 417, and a NetFlow information correcting unit 418. The collector 450 may also have a vital information receiving unit 412, and a threshold information correcting unit 414.

The memory unit 420 stores data used by the CPU 440 when executing a program (e.g., the NetFlow information accumulation unit 421, the threshold information holding unit 424, the packet statistics accumulation unit 425, and the corrected NetFlow information accumulation unit 426), and the program executed by the CPU 440.

In the configuration of FIG. 11, the process of receiving NetFlow information from the sensor 350 and storing it in the NetFlow information storage unit 421 is the same as the process of the collector 400 in the first embodiment, so a description thereof will be omitted.

Furthermore, the collector 450 acquires packet statistics information from the sensor 350 and corrects the flow information already received in the NetFlow information storage unit 421. This correction process corrects the NetFlow information when traffic that exceeds the NetFlow processing capacity of the sensor 350 occurs in the network 100 and the NetFlow information received by the collector 450 regarding that traffic is expected to contain an error, thereby reducing the possibility of the collector 450 misjudging an anomaly. When estimating whether the NetFlow information contains an error, the information acquired and stored by the sensor 350 in the packet statistics storage unit 322 is used. The operation of the collector 450 for this purpose is described below.

Figure 12 is a diagram showing the operation sequence from when the collector 450 in the second embodiment receives packet statistical information from the sensor 350 to when the corrected NetFlow information is stored in the corrected NetFlow information storage unit 426. Note that the processing described here is executed independently and in parallel with the NetFlow information reception processing by the NetFlow receiving unit 441 and the threshold determination processing by the NetFlow threshold determination unit 415.

First, the sensor 350 transmits packet statistics information to the collector 450. The packet statistics receiving unit 417 of the collector 450 receives the packet statistics information through the network interface 430-1 (step S701), and stores the received packet statistics information in the packet statistics storage unit 425 (step S702).

Figure 14 is a diagram showing an example of the configuration of the packet statistics accumulation unit 425. The packet statistics accumulation unit 425 has multiple entries, and each entry includes packet statistical information measured per unit time, which is associated with a time 4251 indicating the time of statistics acquisition, the number of packets measured during the unit time 4252, and the number of bytes 4253 which is the sum of the packet lengths of all packets measured during the unit time. The statistical information shown in the figure is an example, and other statistical information may be stored.

Returning to the sequence diagram of FIG. 12, the explanation will be continued. In parallel with the processing of the packet statistics receiving unit 417, the NetFlow information correction unit 418 acquires packet statistics information from the packet statistics accumulation unit 425 at a predetermined timing (e.g., periodically) (step S703), and further acquires NetFlow information from the NetFlow information accumulation unit 421 (S704).

Figure 13 is a diagram showing an example of the configuration of the NetFlow information accumulation unit 421. The NetFlow information accumulation unit 421 has multiple entries, and each entry includes information that associates the time 4211 when the NetFlow information was acquired, a source address 4212 and a destination address 4213 that are information for identifying a flow, the number of packets for each flow 4214, and the number of bytes 4215 that is the total packet length for each flow. Note that the information for identifying a flow is an example, and may include, for example, a TCP or UDP port number included in the packet header. It may also include statistical information other than the number of packets and the number of bytes.

Returning to the sequence diagram in Figure 12, the explanation will continue. The NetFlow information correction unit 418 compares the acquired packet statistical information with the NetFlow information, and corrects the received NetFlow information based on the comparison result (step S705). Then, the corrected NetFlow information is stored in the corrected NetFlow information storage unit 426 (S706).

Figure 15 is a diagram showing an example of the configuration of the corrected NetFlow information accumulation unit 426. The configuration of the corrected NetFlow information accumulation unit 426 is the same as that of the NetFlow information accumulation unit 421, so a description thereof will be omitted.

Here, an example of the process of correcting NetFlow information (step S705) will be described using the specific numerical examples shown in Figures 13, 14, and 15.

In step S705, the NetFlow information correction unit 418 compares the packet statistical information acquired in step S703 with the statistical information in the NetFlow information acquired in step S704 for each time, and determines whether the received NetFlow information is correct (i.e., whether the NetFlow information is likely to be incorrect because the sensor 350 is not operating). If there are multiple pieces of NetFlow information relating to different flows at the same time, the total value of the statistical information for those flows is compared to make a determination.

For example, the packet statistics information at time "2020/1/1 09:59" shows the number of packets 4252 as "3" and the number of bytes 4253 as "384" (entry #1 in FIG. 14), and the total of the NetFlow information acquired at the same time also shows the number of packets 4214 as "3" and the number of bytes 4215 as "384" (entry #1 in FIG. 13). If these statistical values match, the NetFlow information correction unit 418 determines that the NetFlow information at this time is correct and no correction is required.

On the other hand, the packet count 4252 of the packet statistics information at the time "2020/1/1 10:00" is "10,000" and the byte count 4253 is "1,280,000" (entry #2 in FIG. 14), and the total of the NetFlow information acquired at the same time is the packet count 4214 of "5,000" and the byte count 4215 of "640,000" (total values of the packet counts and byte counts from entry #2 to entry #6 in FIG. 13). As described above, the packet statistics information is expected to provide statistical values with extremely small errors, but the NetFlow information may not be measured accurately at the timing of traffic that exceeds the processing capacity of the sensor 350 or collector 450, and this difference is considered to be an error that occurs for such a reason. When such a difference occurs, the NetFlow information correction unit 418 corrects the NetFlow information.

There are several methods of correction, but for example, the statistical values of the packet statistical information may be evenly distributed to each flow, and the evenly distributed statistical values may be added to each NetFlow information to calculate the corrected NetFlow information. In addition, the statistical values of the packet statistical information may be prorated to each flow in the ratio of the statistical values of the NetFlow information, and the prorated statistical values may be added to each NetFlow information to calculate the corrected NetFlow information. In the example of FIG. 13, the five pieces of NetFlow information (entries #2 to #6 in FIG. 13) at the time "2020/1/1 10:00" are as follows: the number of packets 4214 of entry #3 is 2,000, the number of bytes 4215 is 256,000, and the number of packets 4214 of the other entries is 1,000, and the number of bytes 4215 is 128,000. Therefore, the NetFlow information correction unit 418 calculates the corrected value of the NetFlow information by distributing the packet count 4252 "12,000" and byte count 4253 "7,680,000", which are the statistical values of the packet statistical information at the same time, to the five flows in the ratio of the packet count 4214 and byte count 4215, respectively. FIG. 15 shows the state in which the calculation result is stored in the NetFlow information accumulation unit after correction. Note that the above method is an example of an effective estimation method when it is expected that the loss of flow statistical information due to insufficient performance of the sensor 350 or collector 450 will occur with equal probability in each flow. The method of estimating the statistical value of each flow from the statistical value of the packet statistical information is not limited to the above.

By the above processing, the corrected NetFlow information is stored in the corrected NetFlow information accumulation unit 426. In the second embodiment, the NetFlow threshold determination unit 415 (not shown in FIG. 11) compares the value of the corrected NetFlow information accumulation unit 426 with a threshold, instead of the NetFlow information accumulation unit 421, to determine traffic abnormalities.

The NetFlow information correction unit 418 may send a packet including this notification information to the management terminal 500 via the network interface 430-2 to notify the administrator that the NetFlow information has been corrected based on the packet statistical information. The management terminal 500 displays the received notification information on a management screen such as a display.

As described above, according to this embodiment, in a network monitoring system consisting of a sensor 300 that acquires flow-based statistical information and a collector 400 that collects and analyzes that statistical information, even if the flow-based statistical information collected by the collector 400 contains errors due to insufficient performance of the sensor 300 or the collector 400, the flow-based statistical information can be estimated and corrected by using packet-based statistical information in combination, thereby suppressing erroneous determination of traffic abnormalities in the analysis process performed by the collector 400.

The unique functions of the second embodiment have been described above, but the collector 400 of the first embodiment and the collector 450 of the second embodiment may be combined to form a collector having the functions of both.

The present invention has been described in detail above with reference to the attached drawings, but the present invention is not limited to such specific configurations, and includes various modifications and equivalent configurations within the spirit of the appended claims.

100 Network to be monitored 200 Network device 300, 350 Sensor 310 Memory 311 Traffic receiver 312 NetFlow information generator 313 NetFlow information transmitter 314 Alive/dead information generator 315 Alive/dead information transmitter 316 MIB response unit 317 MIB request receiver/response transmitter 318 Packet statistics generator 319 Packet statistics transmitter 320 Storage unit 321 Flow information storage unit 322 Packet statistics storage unit 330 Network interface 340 CPU
400, 450 Collector 410 Memory 411 NetFlow receiving unit 412 Alive/dead information receiving unit 413 MIB request sending/response receiving unit 414 Threshold information correcting unit 415 NetFlow threshold determining unit 416 Syslog/Trap sending unit 417 Packet statistics receiving unit 418 NetFlow information correcting unit 420 Storage unit 421 NetFlow information accumulating unit 422 Alive/dead information accumulating unit 423 Sensor/threshold related information holding unit 424 Threshold information holding unit 425 Packet statistics accumulating unit 426 Corrected NetFlow information accumulating unit 430 Network interface 440 CPU
500 Management terminal

Claims (11)

A collector device that monitors network traffic using NetFlow information,
a NetFlow information receiving unit that acquires NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving unit that receives, from the sensor device , alive information of the sensor device as information for determination ;
a threshold information correction unit that uses the received alive information to determine whether a statistical value of the received NetFlow information is correct, and when it is determined that the statistical value of the received NetFlow information is incorrect, updates threshold information used for monitoring network traffic using sensor threshold information defined for each sensor device;
a threshold determination unit that determines whether network traffic is normal by using the updated threshold information . A collector device that monitors network traffic using NetFlow information,
a NetFlow information receiving unit that acquires NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving unit that receives statistical information indicating a data volume of network traffic as information for determination from the sensor device;
a NetFlow information correction unit that uses the received statistical information to determine whether a statistical value of the received NetFlow information is correct, and corrects the received NetFlow information when it is determined that the statistical value of the received NetFlow information is incorrect;
a corrected NetFlow information correcting unit that accumulates the corrected NetFlow information;
and a threshold determination unit that determines whether network traffic is normal using the corrected NetFlow information. 2. The collector device of claim 1,
Sensor threshold information is defined for each of the sensor devices,
The collector device according to the present invention is characterized in that the threshold information correction unit reduces the threshold information used for the monitoring by an amount equivalent to the sensor threshold information of a sensor device determined to be abnormal based on the alive/dead information. 4. A collector device according to claim 3,
a sensor/threshold related information storage unit that stores sensor threshold information including an upper limit threshold and a lower limit threshold for each sensor device and each flow condition;
a threshold information storage unit that stores threshold information including an upper limit threshold and a lower limit threshold used for monitoring the network traffic,
The collector device according to the present invention, wherein the threshold information correction unit updates the threshold information held in the threshold information holding unit by referring to the sensor/threshold association information holding unit. 3. A collector device according to claim 2,
The corrected NetFlow information correcting unit
Calculating a difference between a total value of the NetFlow information received from the sensor device and the received statistical information;
a collector device for distributing the calculated difference to NetFlow information received from each sensor device according to a ratio of the received NetFlow information, thereby correcting the NetFlow information. A network monitoring method executed by a collector device that monitors network traffic using NetFlow information, comprising:
A NetFlow information receiving procedure for acquiring NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving step of receiving, from the sensor device, alive/dead information of the sensor device as information for determination;
a threshold information correction step of determining whether a statistical value of the received NetFlow information is correct using the received alive information, and updating threshold information used for monitoring network traffic using sensor threshold information defined for each sensor device when it is determined that the statistical value of the received NetFlow information is incorrect;
a threshold determination step of determining whether network traffic is normal using the updated threshold information. A network monitoring method executed by a collector device that monitors network traffic using NetFlow information, comprising:
A NetFlow information receiving procedure for acquiring NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving step of receiving statistical information indicating a data volume of network traffic as information for determination from the sensor device;
a NetFlow information correction procedure for determining whether a statistical value of the received NetFlow information is correct using the received statistical information, and correcting the received NetFlow information if it is determined that the received NetFlow information is incorrect;
a corrected NetFlow information correcting unit that accumulates the corrected NetFlow information;
A network monitoring method comprising a threshold judgment step of judging whether network traffic is normal using the corrected NetFlow information. A program for causing a collector device to monitor network traffic using NetFlow information,
A NetFlow information receiving procedure for acquiring NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving step of receiving, from the sensor device, alive/dead information of the sensor device as information for determination;
a threshold information correction step of determining whether a statistical value of the received NetFlow information is correct using the received alive information, and updating threshold information used for monitoring network traffic using sensor threshold information defined for each sensor device when it is determined that the statistical value of the received NetFlow information is incorrect;
and a threshold determination procedure for determining whether network traffic is normal using the updated threshold information. A program for causing a collector device to monitor network traffic using NetFlow information,
A NetFlow information receiving procedure for acquiring NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving step of receiving statistical information indicating a data volume of network traffic as information for determination from the sensor device;
a NetFlow information correction procedure for determining whether a statistical value of the received NetFlow information is correct using the received statistical information, and correcting the received NetFlow information if it is determined that the statistical value of the received NetFlow information is incorrect;
a corrected NetFlow information correcting unit that accumulates the corrected NetFlow information;
and a threshold determination procedure for determining whether network traffic is normal using the corrected NetFlow information. A network monitoring system for monitoring a network state,
A sensor device that receives network traffic and generates NetFlow information;
a collector device that monitors network traffic using the NetFlow information;
The sensor device includes:
a vital information generating unit that generates vital information of the sensor device itself at a predetermined timing;
a vital information transmission unit that transmits the generated vital information to a collector device at a predetermined timing;
The collector device comprises:
a NetFlow information receiving unit that acquires NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving unit that receives, from the sensor device, alive information of the sensor device as information for determination;
a threshold information correction unit that uses the received alive information to determine whether a statistical value of the received NetFlow information is correct, and when it is determined that the statistical value of the received NetFlow information is incorrect, updates threshold information used for monitoring network traffic using sensor threshold information defined for each sensor device;
a threshold determination unit that determines whether network traffic is normal by using the updated threshold information. A network monitoring system for monitoring a network state,
A sensor device that receives network traffic and generates NetFlow information;
a collector device that monitors network traffic using the NetFlow information;
the sensor device has a statistics transmission unit that transmits statistical information indicating a traffic volume from the sensor device to a collector device at a predetermined timing;
The collector device comprises:
a NetFlow information receiving unit that acquires NetFlow information from a sensor device that receives network traffic and generates NetFlow information;
an information receiving unit that receives the statistical information as determination information from the sensor device;
a NetFlow information correction unit that uses the received statistical information to determine whether a statistical value of the received NetFlow information is correct, and corrects the received NetFlow information when it is determined that the statistical value of the received NetFlow information is incorrect;
a corrected NetFlow information correcting unit that accumulates the corrected NetFlow information;
and a threshold determination unit that determines whether network traffic is normal using the corrected NetFlow information.

Images