JP6662812B2 - Calculation device and calculation method - Google Patents

Description

  The present invention relates to a calculation device and a calculation method.

  Measurement methods of communication in a network monitor or a LAN (Local Area Network) analyzer used for the purpose of observing a computer network can be broadly classified into a measurement method using packet capture and a measurement method for measuring flow statistical information.

  The measurement method using packet capture is a technique of copying packets, which are transmission units of information exchanged between computer systems constituting a computer network, and analyzing the contents of the copied packets.

  The measurement method that measures flow statistical information is a technology that measures the amount of information flowing in a computer network by installing a function to measure the amount of communication passing through itself on network devices such as routers that constitute a computer network. It is. Here, the flow statistical information refers to a statistical numerical value such as an information amount of communication passing through a network device. This statistical value is also called a counter. One example of traffic statistics is the number of bytes.

  As typical technical specifications of the flow statistical information, sFlow (registered trademark) (for example, see Non-Patent Document 1) and netFlow (for example, see Non-Patent Document 2) are known. sFlow is a traffic measurement technology based on statistical estimation using packet sampling and a counter for each communication line interface. In addition, netFlow is a technology for measuring the number of packets and the number of bytes per flow in network devices such as routers and switches.

  As another measurement method for measuring the flow statistical information, a method of arranging the packets of the flow in both the upstream and downstream directions of the session from the head in the order of the observed time and using the packet size as an array (for example, Non-Patent Document 3) ), A method of calculating an average value, a median value, a variance of a packet length and a packet arrival interval of a session (for example, see Non-Patent Document 4), a total number of packets and a total number of bytes per session, and identification. A method of calculating the number of packets with the flag, the average and maximum size and variance of all packets (for example, see Non-Patent Document 5) and the like are known.

Traffic Monitoring using sFlow, [online], [Search May 25, 2017], Internet (http://www.sflow.org/sFlowOverview.pdf) Omar Santos, “Network Security with NetFlow and IPFIX”, Cisco Press, September 2015 Yuji Izumi, Kazuyuki Tanaka, "Web Application Identification Based on Traffic Analysis", IEICE Technical Report CS2013-40 (2013-09), pp.61-66 Tsuyoshi Kitamura, Takayuki Shizuno, Toshiya Okabe, "Application Identification Method Based on Flow Behavior Analysis", IEICE Technical Report NS2005-136 (2005-12), pp.13-16 Liu Yingqiu, Li Wei, Li Yunchun, “Network Traffic Classification Using K-means Clustering”, Second International Multisymposium on Computer and Computational Sciences, pp.360-365

  However, the conventional technology has a problem that effective communication measurement may not be performed using limited processing resources. For example, a measurement method using packet capture requires a large amount of storage resources and computation resources. Therefore, when available processing resources are limited, a measurement method using packet capture may not be possible. On the other hand, in a measurement method for measuring flow statistical information, effective communication measurement may not be performed.

  For example, since sFlow described in Non-Patent Document 1 intermittently samples packets at a fixed rate, it is detected in a flow having a low probability of being sampled, such as communication with a very small number of packets or a very short packet. Leakage and errors may occur, and the method of selecting packets and the cycle of collecting samples or counters may affect the measurement accuracy, so that effective communication measurement may not be performed. There is a point.

  Also, for example, netFlow described in Non-Patent Document 2 is for measuring the number of packets and the number of bytes in flow units by a network device such as a router or a switch. Therefore, in order for an existing router or switch to perform both the original processing such as the original packet exchange and the processing related to the measurement, extra computational resources may be required.

  The techniques described in Non-Patent Documents 3 to 5 perform measurement by referring to only header information such as the length, arrival interval, and flag of a packet without analyzing the contents of the data payload of a captured packet. The measurable information is limited, and it is necessary to perform measurement by packet capture in order to generate additional flow statistical information.

  In the computing device of the present invention, the acquisition unit that acquires the flow statistical information that is the statistical information on the traffic aggregated for each of the communication source and the communication destination and the session of the traffic based on the flow statistical information are the same. And a calculating unit that calculates statistical information on traffic for each group classified by the classifying unit based on the flow statistical information.

  According to the present invention, effective communication measurement can be performed using limited processing resources.

FIG. 1 is a diagram illustrating an example of a configuration of a calculation system according to the first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of the computing device according to the first embodiment. FIG. 3 is a diagram for explaining a calculation method by the calculation device according to the first embodiment. FIG. 4 is a flowchart illustrating a flow of processing of the router according to the first embodiment. FIG. 5 is a flowchart illustrating a flow of processing of the computing device according to the first embodiment. FIG. 6 is a diagram illustrating an example of a data configuration of the primary storage unit according to the first embodiment. FIG. 7 is a flowchart illustrating a processing flow of the computing device according to the first embodiment. FIG. 8 is a flowchart illustrating a processing flow of the computing device according to the first embodiment. FIG. 9 is a diagram illustrating an example of a data configuration of the secondary storage unit according to the first embodiment. FIG. 10 is a diagram illustrating an example of a computer that executes a calculation program.

[Configuration of First Embodiment]
Hereinafter, embodiments of a calculation device and a calculation method according to the present application will be described in detail with reference to the drawings. Note that the present invention is not limited by the embodiments described below. First, the configuration of the calculation system according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a configuration of a calculation system according to the first embodiment. As shown in FIG. 1, the computing system 1 includes a computing device 10, a client 20, a server 30, and a router 40.

  Here, the router 40 generates flow statistic information based on traffic generated by communication between the client 20 and the server 30. The flow statistics information includes the number of packets and the number of bytes. Further, the communication for which the flow statistical information is generated in the calculation system 1 is not limited to communication between a client and a server, and may be communication between clients, communication between servers, or client communication. Alternatively, the communication may be performed by a device other than the server. The device that generates the flow statistical information is not limited to the router 40, and may be any network device.

  Here, the flow statistical information divides information exchanged between the communicating computer systems into units called flows based on a source, a destination, a protocol, a port, and the like for identifying the computer systems, and performs communication for each flow. It can be said that statistical information is obtained by measuring and calculating the amount. For example, since a flow includes an attribute related to a direction in which information of a communication source and a destination flows, there are usually two types of flows: transmission (going) and receiving (return). In the present embodiment, a unit combining two types of flows, transmission and reception, is called a session. In the following description, in the computing system 1, the direction from the client 20 to the server 30 is called up, and the direction from the server 30 to the client 20 is called down.

  In the present embodiment, as described above, the router 40, which is a network device, generates flow statistical information. The router 40 may generate the flow statistic information for each communication line interface provided in the computer system, or may generate the flow statistic information by dividing it into smaller communication units.

  The computing device 10 performs a statistical operation on the session based on the flow statistical information generated by the router 40, thereby obtaining information that cannot be obtained only by the flow statistical information, for example, information other than the number of packets and the number of bytes. Obtainable.

  Next, the configuration of the computing device 10 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a configuration of the computing device according to the first embodiment. As illustrated in FIG. 2, the computing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

  The communication unit 11 performs data communication with another device via a network. For example, the communication unit 11 is a NIC (Network Interface Card). The communication unit 11 performs data communication with, for example, the router 40.

  The storage unit 12 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), and an optical disk. Note that the storage unit 12 may be a rewritable semiconductor memory such as a random access memory (RAM), a flash memory, or a non-volatile random access memory (NVSRAM). The storage unit 12 stores an OS (Operating System) executed by the computing device 10 and various programs. Further, the storage unit 12 stores various information used in executing the program. The storage unit 12 has a primary storage unit 121 and a secondary storage unit 122.

  The control unit 13 controls the entire computing device 10. The control unit 13 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Further, the control unit 13 has an internal memory for storing programs and control data defining various processing procedures, and executes each process using the internal memory. Further, the control unit 13 functions as various processing units when various programs operate. For example, the control unit 13 includes an acquisition unit 131, a classification unit 132, a calculation unit 133, and a storage unit 134.

  The acquiring unit 131 acquires flow statistical information that is statistical information on traffic aggregated for each communication source and each communication destination. The acquisition unit 131 acquires flow statistical information generated by a network device such as the router 40.

  In addition, the classification unit 132 classifies the flow statistical information into groups such that the sessions of the underlying traffic are the same. The classification unit 132 extracts, for example, information that can identify a session from the flow statistics information acquired by the acquisition unit 131, and classifies the flow statistics information for each session of the original traffic based on the extracted information. . The information that can identify the session includes the source and destination included in the flow statistical information, the generation time of the flow statistical information, and the like.

For example, the classification unit 132 determines that the transmission source and the transmission destination included in the first flow statistics information are the same as the transmission destination and the transmission source included in the second flow statistics information, respectively , and that the first flow statistics If the information and the second flow statistical information are both based on traffic generated within a predetermined period, the first flow statistical information and the second flow statistical information are classified into the same group. As described above, the computing device 10 classifies the flow statistical information into groups for each session.

  The calculating unit 133 calculates statistical information on traffic for each group classified by the classifying unit 132 based on the flow statistical information. For example, when the obtaining unit 131 obtains the number of packets and the number of bytes as flow statistical information, the calculating unit 133 calculates, as the statistical information, the average of the packet size for each group, the maximum value of the average of the packet size, and the average of the average of the packet size. The minimum value, the standard deviation of the average of the packet size, the time average of the number of bytes, and the information indicating the presence or absence of the transmitted / received packet at each time are calculated. Further, the calculation unit 133 can calculate, as statistical information, a co-occurrence matrix based on the presence or absence of transmitted / received packets at each time.

  The storage unit 134 stores the flow statistical information acquired by the acquisition unit, the calculation result by the calculation unit 133, and the like in the primary storage unit 121 or the secondary storage unit 122. In the following description, the flow statistical information obtained by the obtaining unit 131 is referred to as input information. The statistical information calculated by the calculating unit 133 is called session statistical information.

  Here, the calculation of the input information and the session statistical information will be specifically described with reference to FIG. FIG. 3 is a diagram for explaining a calculation method by the calculation device according to the first embodiment.

The acquisition unit 131 generates , as input information, a session identifier INP_1 that uniquely identifies a session . The acquiring unit 131 acquires the generation time INP_2 of the input information, the number of upstream packets INP_3, the number of downstream packets INP_4, the number of upstream bytes INP_5, the number of downstream bytes INP_6, and the elapsed time INP_7 after the session is established.

  The session identifier INP_1 is a value that can uniquely identify one session established between a pair of computer systems performing communication. The acquisition unit 131 can generate a session identifier by performing a bit operation, a hash operation, or the like on an address, a protocol number, a port, a time, or the like for identifying a computer system.

  The generation time INP_2 is the time at which the input information acquired by the acquisition unit 131 was generated. The uplink packet count INP_3 is the total number of packets in the uplink direction (an integer value on 0) from the session start time to the generation time INP_2 in the session identified by the session identifier INP_1. The number of downlink packets INP_4 is the total number of packets (an integer value of 0 or more) in the downlink direction from the session start time to the generation time INP_2 in the session identified by the session identifier INP_1. Further, the number of upstream bytes INP_5 is the total number of bytes in the upward direction (integer value on 0) from the session start time to the generation time INP_2 in the session identified by the session identifier INP_1. In addition, the number of downstream bytes INP_6 is the total number of bytes (an integer value of 0 or more) in the downstream direction from the session start time to the generation time INP_2 in the session identified by the session identifier INP_1. The elapsed time INP_7 is the elapsed time from the session start time to the generation time INP_2. In order to perform calculations with higher accuracy, it is preferable that the generation time INP_2 and the elapsed time INP_7 include a unit of microsecond or millisecond.

Here, the acquisition unit 131 acquires the input information generated every fixed time dT from the session start time. For example, as shown in FIG. 3, the acquisition unit 131 first acquires the time T ₁ to the generated input information INP (T _1), then the time T _2, from the time T ₁ has elapsed time dT Obtain the generated input information INP (T ₂ ). Thus, obtaining unit 131 sequentially obtains the input information to acquire the input information generated at time T _n is the session end time. Here, representing the inputs to the generation time T _k as INP (T _k). INP (T _k ) includes INP_1 (T _k ), INP_2 (T _k ), INP_3 (T _k ), INP_4 (T _k ), INP_5 (T _k ), INP_6 (T _k ), INP_7 (T _k ) included. Further, it is desirable that the time interval dT at which the input information is generated is constant, but may be different.

Next, the calculation unit 133 calculates two variables of the average packet size AVE_1 and the average packet size AVE_2 for each session as the session statistical information. Calculation unit 133, INP (T _k), and the time T on the basis of the input information INP (T _k-1) in the previous time T _k-1 of _k, uplink average packet size AVE_1 at time T _k (T _k ) and the average downlink packet size AVE_2 (T _k ) are calculated as in equations (1) and (2), respectively.
AVE_1 (T _k ) = {INP_4 (T _k ) −INP_4 (T _k−1 )} ÷ {INP_3 (T _k ) −INP_3 (T _k−1 )} (1)
AVE_2 (T _k ) = {INP_6 (T _k ) −INP_6 (T _k−1 )} ÷ {INP_5 (T _k ) −INP_5 (T _k−1 )} (2)

As described above, the calculation unit 133 can calculate the average packet size by averaging the difference in the number of bytes by the difference in the number of packets between the time _Tk and the time _Tk-1 . Here, representing the average packet size in generation time T _k as AVE of (T _k). AVE (T _k ) includes AVE_1 (T _k ) and AVE_2 (T _k ).

Each average packet size is an average number of bytes per packet of a packet flowing between time _Tk and time _Tk-1 . When INP (T _k−1 ) does not exist (for example, when k = 1, that is, when INP (T _k ) is the first input information generated after the start of the session), the calculation unit 133 sets the INP_3 (T k _{k-1), INP_3 (T} k-1), INP_3 (T k-1), performs calculations INP_3 a (T _k-1) as 0. Further, the storage unit 134 stores INP (T _k ) in the primary storage unit 121 until AVE_1 (T _{k + 1} ) is calculated. Then, after calculating AVE_1 (T _{k + 1} ), storage unit 134 may discard INP (T _k ).

Further, the calculating unit 133 uses AVE (T _k ) to calculate the maximum uplink average packet size AVE_MAX_1 (T _k ), the maximum downlink average packet size AVE_MAX_2 (T _k ), the minimum uplink average packet size AVE_MIN_1 (T _k ), minimum average packet size AVE_MIN_2 (T _k), calculated as the standard deviation of the uplink average packet size AVE_SD_1 (T _k), and a downlink standard deviation of the mean packet size AVE_SD_2 a (T _k) from equation (3) (8) I do.
_{AVE_MAX_1 (T k) = {AVE_1} (T 1), AVE_1 (T 2), ..., AVE_1 (T k)} maximum value (3) of
AVE_MAX_2 (T _k ) = {AVE_2 (T ₁ ), the maximum value of AVE_2 (T ₂ ),…, AVE_2 (T _k )} (4)
_{AVE_MIN_1 (T k) = {AVE_1} (T 1), AVE_1 (T 2), ..., AVE_1 (T k)} minimum ... (5)
AVE_MIN_2 (T _k ) = {AVE_2 (T ₁ ), the minimum value of AVE_2 (T ₂ ),…, AVE_2 (T _k )} (6)
AVE_SD_1 (T _k ) = standard deviation of {AVE_1 (T ₁ ), AVE_1 (T ₂ ),..., AVE_1 (T _k )} (7)
AVE_SD_2 (T _k ) = {standard deviation of AVE_2 (T ₁ ), AVE_2 (T ₂ ), ..., AVE_2 (T _k )} ・ ・ ・ (8)

The calculating unit 133 also calculates the time average of the number of bytes flowing in each of the upstream and downstream directions from the start to the end of the session, that is, the upstream flow rate FRATE_1 and the downstream flow rate FRATE_2, respectively (9 ) And (10). Here, T _n is the time at which the session ended. Further, as shown in Expressions (9) and (10), the calculation unit 133 can calculate the flow rate only from the input information at the end of the session, that is, INP (T _n ).
FRATE_1 = INP_4 (T _n ) ÷ INP_7 (T _n ) ・ ・ ・ (9)
FRATE_2 = INP_6 (T _n ) ÷ INP_7 (T _n ) ・ ・ ・ (10)

The calculator 133 calculates a packet co-occurrence matrix. Here, the co-occurrence matrix is a matrix that expresses a relative relationship between pixels such as pixels and words and an appearance pattern of words, and has been generally used in image recognition, language processing, and the like. In the present embodiment, the calculation unit 133 calculates the packet co-occurrence matrix as follows. First, the calculation unit 133 indicates whether or not one or more packets have flowed between the time T _k−1 and the time T _k in a certain session by a binary value of 0 or 1. Note that the binary BOOL_1 (T _k) of the uplink, representing the binary downlink BOOL_2 and (T _k).

Here, the calculation unit 133, INP_3 (T _k), or INP_5 (T _k), respectively INP_3 (T _k-1), or INP_5 when (T _k-1) greater than the time T _k-1 It is assumed that one or more packets have flowed in the upstream direction between the time _Tk and the binary value is set to 1. This calculation method can be expressed as the following equations (11) and (12).
BOOL_1 (T _k ) =
_{INP_3 (T k) -INP_3 (T} k-1)> 0 if _{1, INP_3 (T k) -INP_3} (T k-1) = 0 if 0 (11)
BOOL_2 (T _k ) =
If INP_5 (T _k ) -INP_5 (T _k-1 )> 0, 1 if INP_5 (T _k ) -INP_5 (T _k-1 ) = 0, 0 ... (12)

Also, the binary value can be calculated using the average of the packet sizes as in the following Expressions (13) and (14).
BOOL_1 (T _k ) '= AVE_1 (T _k )> 0, 1 if AVE_1 (T _k ) = 0, 0 ... (13)
BOOL_2 (T _k ) '= AVE_2 (T _k )> 0, 1 if AVE_2 (T _k ) = 0, 0 ... (14)

The calculation unit 133 sets {T ₁ ,..., T _n } from the time immediately after the start of the session (first) to the end (n-th) from the time T ₁ to T ₁ the binary determined for each of up to _n, it is possible to obtain the n-number of digits sequence made of the 0 and 1, respectively downlink and uplink.

  Assuming that the sequence obtained by the calculation unit 133 for the upstream packet is BOOL_1 and the sequence obtained for the downstream packet is BOOL_2, BOOL_1 and BOOL_2 each have a length of n numbers. For example, BOOL_1 = {1,0, ...., 1}. BOOL_1 and BOOL_2 represent patterns of the presence / absence of packet transmission in a certain session. For example, in the case of a session in which packets are continuously transmitted from the start to the end of the session, BOOL_1 and BOOL_2 are represented as a sequence of one continuous {1,1,1,1,1, ..., 1}. become. Also, when transmitting packets intermittently beyond the time period in which the input information is generated, BOOL_1 and BOOL_2 repeat 1 and 0, for example, {1,0,1,0,1, ..., 0} or {1,0,0,1,0,0, ..., 1}.

  Further, when information is sent only immediately after the start of a session and the processing is completed in a short time, the length of the sequence becomes short, and a sequence such as {1,0} may be generated. As described above, BOOL_1 and BOOL_2 represent the packet transmission pattern in a certain session, but the length of BOOL_1 and BOOL_2 is variable depending on the duration of the session, and it is necessary to predict the length. It is difficult.

  The calculation unit 133 generates a co-occurrence matrix from BOOL_1 and BOOL_2 obtained by the above procedure. Since BOOL_1 and BOOL_2 are one-dimensional arrangements of 0s or 1s, if attention is paid to two consecutive numbers in a sequence, the combination of arrangements is {00}, {01}, {10}, There are only four ways of {11}. The calculation unit 133 extracts two numbers each from the top of BOOL_1 and BOOL_2, and totals the number of occurrences of the combination. Here, as one example, a method of generating a co-occurrence matrix by the calculation unit 133 when input information is generated ten times, that is, when n = 10, will be described. For example, when BOOL_1 = {1,1,1,1,1,1,1,1,1,1}, the calculation unit 133 calculates the co-occurrence matrix MATRIX_1 as {00} = 0, {01} = 0, Generate as {10} = 0, {11} = 9. Also, for example, in the case of BOOL_2 = {1,0,1,0,1,0,1,0,1,0}, the calculation unit 133 calculates the co-occurrence matrix MATRIX_2 as {00} = 0, {01} = Generated as 4, {10} = 5, {11} = 0.

  Further, when the session time is short, for example, when n = 4 and BOOL_1 = {1,0,1,1}, the calculation unit 133 sets the co-occurrence matrix MATRIX_1 to {00} = 0, {01}. = 1, {10} = 1, {11} = 1. As described above, the calculation unit 133 calculates the up-down co-occurrence matrix for a certain session, and since each co-occurrence matrix has 4 variables, a total of 8 variables can be obtained.

[Processing of First Embodiment]
The processing flow of the calculation system 1 will be described with reference to FIGS. FIG. 4 is a flowchart illustrating a flow of processing of the router according to the first embodiment. FIGS. 5, 7, and 8 are flowcharts illustrating the processing flow of the computing device according to the first embodiment. FIG. 6 is a diagram illustrating an example of a data configuration of the primary storage unit according to the first embodiment. FIG. 9 is a diagram illustrating an example of a data configuration of the secondary storage unit according to the first embodiment.

  As shown in FIG. 4, the router 40 waits until a certain time elapses (step S11, No), and when the certain time elapses (step S11, Yes), generates the input information (step S12).

As shown in FIG. 5, the acquisition unit 131, network devices, i.e., reading input information INP time T _k from the router 40 (T _k) (step S21). The classification unit 132 determines whether the input information INP (T _k ) acquired by the acquisition unit 131 is for a new session (Step S22).

Here, when the input information INP (T _k−1 ) is not stored in the primary storage unit 121, the classification unit 132 determines that the input information INP (T _k ) acquired by the acquisition unit 131 is for a new session. It is determined that there is (Step S22, Yes). In this case, the storage unit 134 stores the input information INP (T _k ) in the primary storage unit 121 (Step S23). As shown in FIG. 6, the primary storage unit 121 stores input information. FIG. 6 shows that the session identifier INP_1 (T _k ) of the input information INP (T _k ) stored by the storage unit 134 is “xyz001”, the generation time INP_2 (T _k ) is “20:40”, and the number of uplink packets INP_3 ( T _k) is "10", "80" number uplink bytes INP_4 (T _k) is, the downlink packet number INP_5 (T _k) is "400", the number down bytes INP_6 (T _k) is "10000", the elapsed time INP_7 (T _k ) was “2”. In this case, the storage unit 134 stores the statistical information in the primary storage unit 121. Since the statistical information in this case is the first statistical information of the session, the storage unit 134 sets each value of the statistical information in the primary storage unit 121 to 0.

On the other hand, when the input information INP (T _k−1 ) is stored in the primary storage unit 121, the classification unit 132 determines that the input information INP (T _k ) acquired by the acquisition unit 131 is not for a new session. It is determined (Step S22, No). That is, the classification unit 132 classifies the input information INP (T _k−1 ) and the input information INP (T _k ) into the same group. In this case, the storage unit 134 stores the input information INP (T _k ) in the primary storage unit 121. Further, the acquiring unit 131 reads the input information and the statistical information at the time T _k−1 from the primary storage unit 121 (Step S24). And the calculation part 133 calculates each statistical information (step S25).

  As described above, the acquiring unit 131 can acquire the flow statistical information corresponding to each of the times at the fixed time intervals in chronological order. At this time, the classifying unit 132 classifies the flow statistical information each time the obtaining unit 131 obtains the flow statistical information. The calculating unit 133 calculates statistical information on traffic for each group each time the classifying unit 132 performs classification.

As shown in FIG. 7, in step S25 of FIG. 5, the calculation unit 133, first, based on the input information of the input information and time T _k-1 at time T _k, calculate the average packet size at time T _k (Step S251). Next, calculation unit 133, based on the average packet size and average packet size at time T _k-1 at time T _k, the maximum value of the average packet size at time T _k, a minimum value, and calculates the standard deviation ( Step S252). Next, calculation unit 133, based on the input information of the input information and time T _k-1 at time T _k, calculates the co-occurrence matrix of time T _k (step S253). Then, the storage unit 134 stores the respective pieces of statistical information calculated by the calculation unit 133 in the primary storage unit 121.

Here, when the calculation regarding the average packet size is performed, the calculation unit 133 uses only the input information and the statistical information at the time T _k−1 and the time T _k without using all the input information from the start to the end of the session. Is used, the calculation can be performed as in the equations (15) to (18).
AVE_MAX_1 (T _k ) = {AVE_1 (T _k-1 ), the larger of AVE_1 (T _k )} (15)
AVE_MAX_2 (T _k ) = larger of {AVE_2 (T _k-1 ), AVE_2 (T _k )} (16)
AVE_MIN_1 (T _k ) = {AVE_1 (T _k−1 ), AVE_1 (T _k )}, whichever is smaller (17)
AVE_MIN_2 (T _k ) = {AVE_2 (T _k-1 ), the smaller of AVE_2 (T _k )} (18)

Thus, in step S254, the storage unit 134, only the statistics at the time T _k calculated by the calculation unit 133 may be as stored in the primary storage unit 121. That is, the storage unit 134 deletes the statistics at time T _k-1, may be stored statistics in time T _k, overwrites the statistics at the time T _k to the statistics at time T _k-1 May be. In this way, by discarding the statistical information of the previous time, only the input information and the statistical information of one time need be stored in the primary storage unit 121, and the storage capacity can be reduced. it can.

Here, if there is a variable X = {x1, x2, ..., x (k), ..., x (n-1), x (n)}, the variance sig (k) at the k-th, That is, the square of the standard deviation is represented by a recurrence equation shown in the following equation (19). Calculation unit 133 may calculate the (19) using the formula, AVE_SD_1 (T _k-1) and AVE_SD_2 (T _k-1) based on AVE_SD_1 (T _k) and AVE_SD_2 (T _k). Here, u (k) is an average of x (k) up to the k-th.

The calculation unit 133 may calculate the co-occurrence matrix of time T _k based on the input information and statistical information of the time T _k-1. First, the calculation unit 133 calculates BOOL_1 (T _k ) ′ and BOOL_2 (T _k ) ′ based on AVE_1 (T _k ) and AVE_2 (T _k ) according to Expressions (13) and (14). Here, if BOOL_1 (T _k−1 ) and BOOL_2 (T _k−1 ) are stored in the primary storage unit 121, the calculation unit 133 calculates BOOL_1 (T _k−1 ) and BOOL_1 (T _k ) ′. , Or by concatenating BOOL_2 (T _k-1 ) and BOOL_2 (T _k ) ', it is possible to obtain which of {00}, {01}, {10}, {11} is generated. BOOL_1 and BOOL_2 can be calculated.

  As described above, when the statistical information of the group classified by the classifying unit 132 has already been calculated, the calculating unit 133 calculates the statistical information based on the calculated statistical information and the flow statistical information acquired by the acquiring unit 131. , Calculate traffic statistics for each group.

  Here, the calculation unit 133 determines whether or not the session has ended (Step S26). If it is determined that the session has ended (step S26, Yes), the calculation unit 133 calculates statistical information in session units (step S27), substitutes k + 1 for k (step S28), and sets the next time. Proceed to processing. The statistical information for each session is, for example, a flow rate. If it is determined that the session has not ended (No at Step S26), the calculation unit 133 substitutes k + 1 for k (Step S28), and proceeds to the process at the next time.

Here, the calculation unit 133 can determine whether or not the session has ended, based on whether or not the input information at the time T _{k + 1} has been generated in the router 40. That is, if INP (T _{k + 1} ) is input information having the same session identifier as INP_1 (T _k ), calculation unit 133 determines that the session has not ended.

  Furthermore, the calculation unit 133 may determine whether or not the session has ended by referring to a flag included in the header portion of the packet. For example, the calculation unit 133 refers to whether a FIN flag indicating the end of transmission is ON or OFF in a TCP (Transmission Control Protocol) header, and determines that the session has ended if the FIN flag is ON. can do. Note that this method can be realized by adding a FIN flag to the input information.

As shown in FIG. 8, in step S27 in FIG. 5, the calculation unit 133, first, based on the input information of the time T _k, to calculate the flow rate of the session (step S271). Then, the storage unit 134 stores the statistical information and the flow rate of the primary storage unit 121 in the secondary storage unit 122, and deletes the input information and the statistical information of the primary storage unit 121 (Step S272).

As shown in FIG. 9, the secondary storage unit 122 stores input information and statistical information. FIG. 9 shows that the session identifier INP_1 (T _n ) of the input information INP (T _k ) stored in the secondary storage unit 122 is “abc123”, the generation time INP_2 (T _n ) is “20:29”, and the upstream packet number INP_3 (T _n) is "20", the uplink number of bytes INP_4 (T _n) is "120", the number downlink packet INP_5 (T _n) is "650", the number downlink bytes INP_6 (T _n) is "30000", This indicates that the elapsed time INP_7 (T _k ) is “5”. FIG. 9 shows that the statistical information stored in the secondary storage unit 122 has an average packet size AVE_1 (T _n ) of “30”, an average packet size AVE_2 (T _n ) of “200”, and an average packet size of up The maximum size value AVE_MAX_1 (T _n ) is “60”, the maximum downlink average packet size value AVE_MAX_2 (T _n ) is “500”, the minimum uplink average packet size value AVE_MIN_1 (T _n ) is “2”, and the downlink average The minimum packet size value AVE_MIN_2 (T _n ) is “50”, the standard deviation AVE_SD_1 (T _n ) of the uplink average packet size is “30”, and the standard deviation AVE_SD_2 (T _n ) of the downlink average packet size is “300”. The co-occurrence matrix MATRIX_1 (T _n ) is `` 0,0,0,9 '', the upstream co-occurrence matrix MATRIX_2 (T <sub>n</sub> ) is `` 0,2,2,5 '', and the upstream flow rate FRATE_1 (T _n ) is `` 19 And that the downstream flow rate FRATE_2 (T _n ) is “1300”. Further, for example, storage unit 134, below the line session identifier INP_1 (T _n) is "abc123" session identifier INP_1 (T _n) to create a line of "xyz001", the input information and statistics May be saved.

[Example]
An example based on the first embodiment will be described. In this embodiment, the router 40 collects flow statistical information using a method called NetFlow. The router 40 may use a method other than NetFlow as long as necessary information can be collected as input information. For example, the router 40 uses OpenFlow (Reference 1: OpenFlow Switch Specification (URL: https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1. 3.2.pdf)) Input information can be generated even with the information specified in "Body of reply to OFPMP_FLOW request".

  NetFlow is equipped with a function to generate flow statistical information in network equipment such as routers and switches that are used as standard on the Internet, and is one of the mechanisms for transmitting that information to remote information collectors and analyzers. One. As shown in FIG. 1, a router 40 (NetFlow-Enabled Router) is a network device that generates flow statistical information of communication passing therethrough, and a computing device 10 (NetFlow Collector) receives a Flow Record that is flow statistical information. , Collection and analysis equipment.

Here, in NetFlow Version 5, the following information is defined in the Flow Record Format.
1.srcaddr Source IP address
2.dstaddr Destination IP address
3.nexthop IP address of next hop router
4.input SNMP index of input interface
5.output SNMP index of output interface
6.dPkts Packets in the flow
7.dOctets Total number of Layer 3 bytes in the packets of the flow
8.First SysUptime at start of flow
9.Last SysUptime at the time the last packet of the flow was received
10.srcport TCP / UDP source port number or equivalent
11.dstport TCP / UDP destination port number or equivalent
12.pad1 Unused (zero) bytes
13.tcp_flags Cumulative OR of TCP flags
14.prot IP protocol type (for example, TCP = 6; UDP = 17)
15.tos IP type of service (ToS)
16.src_as Autonomous system number of the source, either origin or peer
17.dst_as Autonomous system number of the destination, either origin or peer
18.src_mask Source address prefix mask bits
19.dst_mask Destination address prefix mask bits
20.pad2 Unused (zero) bytes

The acquisition unit 131 of the computing device 10 can generate input information as described below using the information of the Flow Record Format. The acquisition unit 131 can generate INP_1 by performing a bit operation or a hash calculation by using Flow Record Format 1, 2, 8, 10, 11, and 14 as inputs. In other words, the acquisition unit 131 acquires information capable of identifying a session, and generates a session identifier from the acquired information. Further, the acquisition unit 131 may generate INP_2 with reference to the time at which the router 40 generates or transmits the Flow Record, or the computing device 10 may set the time at which the Flow Record was received as INP_2. Further, the acquisition unit 131 generates INP_3 and INP_5 with reference to 6 of the Flow Record Format. Further, the acquisition unit 131 generates INP_4 and INP_6 with reference to Flow Record Format 7. Further, the acquisition unit 131 calculates INP_7 by calculating a time difference between 9 and 8 in the Flow Record Format.

  Here, INP_3 and INP_5, and INP_4 and INP_6 are paired for up and down. Since the Flow Record of NetFlow is information about a flow in one direction, the acquisition unit 131 needs to find a Flow Record to be paired, but the Up and Down Flow Records are 1 and 2, 10 of the above Flow Record Format. And 11 are reversed. That is, the acquisition unit 131 only has to find a pair of flows in which the transmission source (source) and the destination (destination) are exchanged. Then, the acquisition unit 131 generates an INP corresponding to one session from the paired two Flow Records.

  The router 40 generates and transmits a Flow Record relating to a flow passing through itself at a certain time T, that is, all sessions. Since the transmission timing of the Flow Record can be set by the router 40, it is possible to transmit the Flow Record at fixed time intervals, for example, every 10 seconds. Further, the computing device 10 generates and updates session statistical information by receiving the Flow Record and repeating the calculation according to the procedure of the embodiment.

  When the client 20 starts communication with the server 30, the router 40 records the communication as a Flow Record. Two Flow Records are generated for the up direction and the down direction. For example, in the embodiment, the calculation system 1 performs processing according to the following flow.

(At the beginning and continuation of session)
1. The router 40 generates a Flow Record and sends it to the computing device 10 when the Flow Record is transmitted.
2. The computing device 10 receives the Flow Record and records the reception time.
3. The computing device 10 generates input information according to the procedure of the embodiment.
4. The calculation device 10 generates statistical information according to the procedure of the embodiment.
5. The computing device 10 waits for input information at the next time.
6. The router 40 generates a Flow Record and sends it to the computing device 10 when the Flow Record is transmitted.
7. The calculation device 10 receives the Flow Record and records the reception time.

(At the end of the session)
8. The client 20 or the server 30 ends the communication.
9. The router 40 detects the end of the communication, transmits the Flow Record to the computing device 10, and then deletes the ended Flow Record.
10. The computing device 10 receives the Flow Record of 9., records the reception time, and generates input information and statistical information.
11. The computing device 10 waits for input information at the next time.
12. Since the Flow Record has been deleted, the router 40 does not transmit anything (or transmits an empty Flow Record) for the Flow Record of the corresponding communication.
13. The calculation device 10 determines that the session has ended, and calculates the flow rate.
14. The calculation device 10 writes the input information and the statistical information to the secondary storage device or the like, and ends the calculation of the statistical information related to the session identified by INP_1.

[Effect of First Embodiment]
The acquiring unit 131 acquires flow statistical information that is statistical information on traffic aggregated for each communication source and each communication destination. In addition, the classification unit 132 classifies the flow statistical information into groups such that the sessions of the underlying traffic are the same. The calculating unit 133 calculates statistical information on traffic for each group classified by the classifying unit 132 based on the flow statistical information.

  As described above, the computing device 10 of the present embodiment measures communication between computer systems without using packet capture. For this reason, according to the present embodiment, it is not necessary to copy and store the capture data, and it is possible to obtain an effect that a secondary storage device for storing the capture data is not required.

  Further, in the present embodiment, since the packet is not copied, there is no problem regarding the confidentiality and privacy protection of the communication content included in the payload. Further, according to the present embodiment, it is possible to measure the communication of the encrypted communication used to protect the secret of the communication contents without reading the contents of the payload.

  Further, in the conventional measurement of the flow statistical information, the statistical information that can be measured is limited to two types of the number of packets and the number of bytes. On the other hand, the computing device 10 of the present embodiment stores eight types of statistical information (the number of packets, the number of bytes, the average packet size, the maximum value, the minimum value, the standard deviation, the flow rate, and the co-occurrence matrix of the average packet size). Generated by calculation. For this reason, if the number of values that one variable can take is M, the information amount obtained by the conventional technique is M2, whereas the information amount of M8 is obtained in the present embodiment. Can be.

  Further, in the present embodiment, it is not necessary to hold information necessary for calculation over the entire period from the start to the end of the session. That is, in the present embodiment, each piece of statistical information other than the number of packets and the number of bytes can be calculated only by temporarily storing one cycle of the flow statistical information that can be obtained periodically. As described above, according to the computing device 10 of the present embodiment, effective communication measurement can be performed using limited processing resources.

  The classification unit 132 determines that the transmission source and the transmission destination included in the first flow statistics information are the same as the transmission destination and the transmission source included in the second flow statistics information, respectively. When the second flow statistical information is based on traffic generated within a predetermined period, the first flow statistical information and the second flow statistical information can be classified into the same group. As described above, the computing device 10 classifies the flow statistical information into groups for each session. As a result, according to the present embodiment, it is possible to calculate statistical information in session units.

  The acquisition unit 131 can acquire at least the number of packets and the number of bytes as flow statistical information. In this case, the calculation unit 133 calculates, as the statistical information, the average of the packet size for each group, the maximum of the average of the packet size, the minimum of the average of the packet size, the standard deviation of the average of the packet size, and the time average of the number of bytes. , And information indicating the presence / absence of a transmitted / received packet for each time can be calculated. Thus, the computing device 10 can generate six types of statistical information from the number of packets and the number of bytes.

  The acquisition unit 131 can acquire flow statistical information corresponding to each of the times at a fixed time interval in order of time. In this case, the classifying unit 132 can classify the flow statistical information each time the obtaining unit 131 obtains the flow statistical information. Further, the calculating unit 133 can calculate statistical information on traffic for each group each time the classifying unit 132 performs classification. Thereby, the calculation device 10 can proceed with the calculation sequentially as the flow statistical information is generated.

  If the statistical information of the group classified by the classifying unit 132 has already been calculated, the calculating unit 133 calculates the statistical information of each group based on the calculated statistical information and the flow statistical information acquired by the acquiring unit 131. Statistics about traffic can be calculated. As a result, it is not necessary to hold all the flow statistical information of a session, so that the storage capacity to be used can be reduced.

  The calculation unit 133 can calculate, as statistical information, a co-occurrence matrix based on the presence or absence of transmitted / received packets at each time. This makes it possible to analyze the appearance pattern of successive packets.

[System configuration, etc.]
Each component of each device illustrated is a functional concept and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / arbitrarily divided into arbitrary units according to various loads and usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

  Further, of the processes described in the present embodiment, all or a part of the processes described as being performed automatically can be manually performed, or the processes described as being performed manually can be performed. All or part can be performed automatically by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
As one embodiment, the calculation device 10 can be implemented by installing a calculation program for performing the calculation of the above-described statistical information on a desired computer as package software or online software. For example, by causing the information processing device to execute the above calculation program, the information processing device can function as the calculation device 10. The information processing apparatus referred to here includes a desktop or notebook personal computer. In addition, the information processing apparatus includes a mobile communication terminal such as a smartphone, a mobile phone, and a PHS (Personal Handyphone System), and a slate terminal such as a PDA (Personal Digital Assistant).

  In addition, the computing device 10 can be implemented as a computing server device that provides a client with a terminal device used by a user and provides the client with a service related to the calculation of the statistical information. For example, the calculation server device is implemented as a server device that provides a calculation service that receives flow statistics information and outputs session statistics information. In this case, the calculation server device may be implemented as a Web server, or may be implemented as a cloud that provides a service related to the calculation of the statistical information by outsourcing.

  FIG. 10 is a diagram illustrating an example of a computer that executes a calculation program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1090 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. That is, a program that defines each process of the computing device 10 is implemented as a program module 1093 in which codes executable by a computer are described. The program module 1093 is stored in, for example, the hard disk drive 1090. For example, a program module 1093 for executing the same processing as the functional configuration in the computing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD.

  The setting data used in the processing of the above-described embodiment is stored as the program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as necessary and executes them.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN, Wide Area Network (WAN), or the like). Then, the program module 1093 and the program data 1094 may be read from another computer by the CPU 1020 via the network interface 1070.

DESCRIPTION OF SYMBOLS 1 Computing system 10 Computing device 11 Communication part 12 Storage part 13 Control part 20 Client 30 Server 40 Router 121 Primary storage part 122 Secondary storage part 131 Acquisition part 132 Classification part 133 Calculation part 134 Storage part

Claims (7)

An acquisition unit that acquires flow statistical information that is statistical information on traffic for each flow ;
A classification unit that classifies the flow statistics information into groups so that the sessions of the underlying traffic are the same;
For each of the sessions corresponding to the group classified by the classifying unit based on the flow statistics information, whether or not one or more upstream packets have flowed between a plurality of times when the flow statistics information is generated. A second sequence of binary values indicating whether or not at least one downstream packet has flowed during each of the plurality of times, and A calculation unit that calculates a co-occurrence matrix based on the number of appearances of each of the combinations of the arrangement methods included in the first and second arrangements, and a co-occurrence matrix based on the number of appearances of each of the combinations of the arrangement methods included in the second arrangement .
A computing device comprising: The first calculating unit indicates, for each of the sessions corresponding to the group, whether or not one or more upstream packets have flowed for each of a plurality of times at which the flow statistical information is generated, by 0 or 1 , And a second sequence of 0 or 1 indicating whether or not one or more packets in the downstream direction have flowed for each of the plurality of times, and further calculates the first sequence and the included sequence. A co-occurrence matrix based on the number of occurrences of each of the combinations {00}, {01}, {10}, and {11}, and the second and included combinations {00}, {01}, and {10} , {11}, wherein the co-occurrence matrix is calculated based on the number of occurrences of each. The classification unit may be configured such that a transmission source and a transmission destination included in the first flow statistics information are the same as a transmission destination and a transmission source included in the second flow statistics information, respectively, and the first flow statistics information And when both the second flow statistics are based on traffic generated within a predetermined period, the first flow statistics and the second flow statistics are classified into the same group. computing device of claim 1 or 2, characterized in that. The acquisition unit acquires at least the number of packets and the number of bytes as the flow statistics information,
The calculation unit may include, as the statistical information, an average of the packet size for each group, a maximum of the average of the packet size, a minimum of the average of the packet size, a standard deviation of the average of the packet size, and the number of bytes. The calculation device according to any one of claims 1 to 3 , wherein a time average and information indicating presence / absence of transmitted / received packets at each time are calculated. The acquiring unit acquires the flow statistical information corresponding to each of the times at a fixed time interval in chronological order,
The classifying unit, each time the obtaining unit obtains the flow statistical information, classifies the flow statistical information,
The calculation device according to any one of claims 1 to 4 , wherein the calculation unit calculates statistical information on traffic for each group each time the classification unit performs classification. The calculation unit, if the statistical information of the group classified by the classification unit has already been calculated, based on the calculated statistical information and the flow statistical information acquired by the acquisition unit, the The calculating device according to claim 5 , wherein statistical information on traffic for each group is calculated. A calculation method performed by a calculation device,
An acquisition step of acquiring flow statistical information that is statistical information on traffic for each flow ;
A classifying step of classifying the flow statistics into groups so that the sessions of the underlying traffic are the same;
Based on the flow statistic information, for each of the sessions corresponding to the group classified by the classification step, whether or not one or more upstream packets have flowed between a plurality of times when the flow statistic information is generated. A second sequence of binary values indicating whether or not at least one downstream packet has flowed during each of the plurality of times, and A calculating step of calculating a co-occurrence matrix based on the number of appearances of each of the combinations of the arrangement methods included in the first and the second arrangements, and
A calculation method comprising:

Images