JP6815931B2 - Statistic data processor, statistic data processing method and program - Google Patents

Description

The present invention relates to a statistic data processing apparatus, a statistic data processing method and a program .

In various fields such as transportation and healthcare, time-series data of a predetermined target is detected. It may be important to immediately detect the abnormality to be detected from such time series data.
In addition, in many cases, stream data in which the number of samples of time-series data becomes enormous and data is constantly generated one after another is processed.

The method of detecting an abnormality is roughly divided into a method of detecting an abnormality using prior knowledge and a method of detecting an outlier as an abnormality.
In the method of detecting an abnormality using prior knowledge, when time-series data satisfying a specific condition is generated, the abnormality is detected as the occurrence of the abnormality, but it is not possible to deal with an unknown event. In addition, in the method of detecting an abnormality using prior knowledge, preconditions are often assumed, so it is necessary to modify a specific condition each time the precondition changes. In addition, this method requires learning prior knowledge. Machine learning may be used to make up for this, but sufficiently high-speed processing has not been established.

Under these circumstances, it is important to establish and improve the method for detecting outliers in stream data.
For example, although techniques for detecting outliers have existed for some time, there are cases where sufficiently high-speed processing is not performed, especially when targeting stream data. In situations where it is difficult to predict when an anomaly will occur, it is necessary to constantly monitor the presence or absence of anomalies, and the processing speed is sufficiently high compared to the speed at which the data that makes up the time series data occurs one after another. It had to be realized.

As an example, Non-Patent Document 1 proposes a technique called t-digest. The t-digest has a data structure for estimating the distribution of a set of values, and for example, it is possible to estimate the distribution of stream data.
However, it may be difficult to realize sufficiently high-speed processing only by t-digest.

TED DUNNING AND OTLAR ERTL, "COMPUTING EXTREMELY ACCURATE QUANTILES USING t-DIGESTS", [online], [Search April 12, 2017], Internet <URL: https://github.com/tdunning/t-di blob / master / docs / t-digest-paper / histo.pdf ＞

In the past, it has sometimes been difficult to perform multidimensional and multiscale data analysis. In addition, in t-digest, multidimensional and multiscale were not assumed.

The present invention has been made in consideration of such circumstances, and provides a statistic data processing apparatus, a statistic data processing method, and a program capable of performing multidimensional and multiscale data analysis. Make it an issue.

As a configuration example, a plurality of different viewpoints including the first viewpoint of time , a plurality of different scales for each of the viewpoints including a plurality of different time scales of the first viewpoint , and one according to each of the time scales. With respect to the plurality of time ranges in which the upper limit value of the time range is set to a value corresponding to an integral multiple of the time scale, one time scale and one time range are specified with respect to the first viewpoint, and the time range is described. A statistic data processing unit that generates a statistic data group having statistic data based on data included in each combination in which one scale is specified for each of the viewpoints other than the first viewpoint, and the plurality of the above. shows the time range in parallel to the first direction and parallel to a direction corresponding to different respective said viewpoint and the first said the scale said plurality of different for each of the viewpoints other than the viewpoints first direction Te shown, a plurality of said statistics data chunks included in one of the time scale, the output mode shown in parallel in a direction different from the first direction about the said time scale different, the statistic It is a statistic data processing apparatus including a data output control unit that outputs the statistic data group generated by the data processing unit to a screen.
As a configuration example, the plurality of different viewpoints other than the first viewpoint include a second viewpoint different from the first viewpoint and a third viewpoint different from the first viewpoint and the second viewpoint. In the output mode, the plurality of time ranges are shown in parallel in the first direction with respect to the first viewpoint, and the plurality of scales are oriented in a second direction different from the first direction with respect to the second viewpoint. Shown in parallel, the plurality of the scales are shown in parallel in the first direction and a third direction different from the second direction with respect to the third viewpoint, and the plurality of time scales are shown with respect to the first viewpoint. May be used, which is an output mode in which is shown in parallel in the same direction as the second direction.

As a configuration example, the statistic data processing unit has a plurality of different viewpoints including a first viewpoint of time , a plurality of different scales for each of the viewpoints including a plurality of different time scales of the first viewpoint , and each of the above. With respect to the first aspect, one time scale and one time for a plurality of time ranges in which the upper limit of one time range is set to a value corresponding to an integral multiple of the time scale according to the time scale. Data output control by generating a statistic data group having statistic data based on the data included in each combination in which the range is specified and one said scale is specified for each of the viewpoints other than the first viewpoint. part is shown a plurality of said time range in parallel to the first direction, and each of the viewpoints different from the first the said scale said plurality of different for each of the viewpoints other than the viewpoints first direction The group of the plurality of statistic data included in one time scale is shown in parallel in the direction corresponding to the above, and the group of the plurality of different time scales is shown in parallel in a direction different from the first direction. In the output mode, it is a statistic data processing method that outputs the statistic data group generated by the statistic data processing unit to a screen.
As a configuration example, the statistic data processing unit has a plurality of different viewpoints including a first viewpoint of time , a plurality of different scales for each of the viewpoints including a plurality of different time scales of the first viewpoint , and each of the above. With respect to the first aspect, one time scale and one time for a plurality of time ranges in which the upper limit of one time range is set to a value corresponding to an integral multiple of the time scale according to the time scale. A step of generating a statistic data group having statistic data based on data included in each combination in which a range is specified and one said scale is specified for each of the viewpoints other than the first viewpoint , and data. output control section, shows a plurality of said time range in parallel to the first direction and different respectively from the first said the scale said plurality of different for each of the viewpoints other than the viewpoints first direction Shown in parallel in the direction corresponding to the viewpoint, a group of the plurality of statistic data included in one said time scale is arranged in parallel in a direction different from the first direction for the plurality of different time scales. This is a program for causing a computer to execute a step of outputting the statistic data group generated by the statistic data processing unit to a screen in the output mode shown in the above.

According to the present invention, it is possible to perform multidimensional and multiscale data analysis.

It is a block diagram which shows the schematic structure of the data processing system which concerns on one Embodiment (1st Embodiment) of this invention. It is a block diagram which shows the schematic structure of the statistic data processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows typically the data structure of an example of the statistical data group which concerns on one Embodiment of this invention. It is a figure for demonstrating an example of the additional processing with respect to the statistic data group which concerns on one Embodiment of this invention. It is a figure for demonstrating an example of the extraction processing with respect to the statistical data group which concerns on one Embodiment of this invention. It is a figure which shows an example of the process procedure for generating the statistic data group performed in the statistic data processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the process procedure for adding data to the statistic data group performed in the statistic data processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the process procedure for extracting data from the statistic data group performed in the statistic data processing apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the schematic structure of the data processing system which concerns on one Embodiment (second Embodiment) of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First Embodiment)
[Data processing system]
FIG. 1 is a block diagram showing a schematic configuration of a data processing system 1 according to an embodiment (first embodiment) of the present invention.
The data processing system 1 includes n (n is an integer of 2 or more) terminal devices 11-1 to 11-n, a statistic data processing device 12, a database 13, and a network 21.
As the network 21, any wired or wireless network may be used, and for example, the Internet or a Wi-Fi (registered trademark) network may be used.
In the present embodiment, the database 13 is provided separately from the statistic data processing device 12, but as another example, the database 13 may be integrated with the statistic data processing device 12.

The statistic data processing device 12 acquires and analyzes predetermined target data regarding the terminal devices 11-1 to 11-n. In the present embodiment, it is assumed that the data is stream data in which the number of samples becomes enormous (big data) and data is always generated one after another for a predetermined target time-series data.

As the data of the predetermined target, arbitrary data may be used, for example, data relating to IoT (Internet of Things) may be used, or other data may be used.
As a specific example, data in an arbitrary system may be used as the data of a predetermined target, for example, data related to a transportation system such as a vehicle, data related to a healthcare system such as a person, a factory system for producing a product, or the like. Data about financial systems such as securities, data about wired or wireless communication systems, and the like may be used. Further, in an arbitrary system, various data may be used as the predetermined target data, for example, temperature-related data, humidity-related data, velocity-related data, acceleration-related data, image-related data, oxygen, and the like. Data on the concentration of substances, data on quality, data on stock prices, data on communication signals, data on the position of terminal devices (terminal devices 11-1 to 11-n in this embodiment), data on areas where terminal devices exist. Etc. may be used.

In addition, various methods may be used as a method for detecting data of a predetermined target.
As an example, in each terminal device 11-1 to 11-n, a configuration for detecting predetermined target data regarding the respective terminal device 11-11 to 11-n may be used. In this configuration, each terminal device 11-1 to 11-n includes a detection unit that detects data of a predetermined target, and processes the data detected by the detection unit as statistical data via the network 21. It is transmitted to the device 12. The detection unit may be, for example, a sensor or an image pickup device (camera). Here, the image pickup device (camera) may also be regarded as an example of the sensor.
In addition, each terminal device 11-1 to 11-n may be an IoT terminal device, or may be another terminal device.

As another example, the data processing system 1 may include a detection device (not shown) different from the terminal devices 11-11 to 11-n. In this configuration, the detection device has a detection unit that detects predetermined target data regarding each terminal device 11-1 to 11-n, and the data detected by the detection unit is transmitted to the network 21. It is transmitted to the statistic data processing device 12 via.

The detection device may include, for example, an imaging device (camera) that captures an image of a predetermined area, and the number of terminal devices 11-1 to 11-n existing in the predetermined area based on the image. Data and the like may be detected as predetermined target data.
The detection device may include, for example, a communication unit that wirelessly or wiredly communicates with terminal devices 11-1 to 11-n existing in a predetermined area, and based on the result of this communication, the detection device may be located in the predetermined area. Data of the number of existing terminal devices 11-1 to 11-n may be detected as predetermined target data.
The detection device may include, for example, a signal acquisition unit that acquires a signal transmitted from the terminal devices 11-1 to 11-n, and based on the status of the signal, the frequency of occurrence or the degree of delay of the signal, etc. Data representing the above may be detected as predetermined target data.

In the present embodiment, each terminal device 11-11 to 11-n can communicate with another device (for example, a statistic data processing device 12) via the network 21. Each terminal device 11-1 to 11-n communicates with the network 21 by wire or wirelessly.
As another example, each terminal device 11-1 to 11-n does not have to have a communication function.

Further, as the respective terminal devices 11-1 to 11-n, for example, terminal devices having the same configuration may be used, or terminal devices having different configurations may be included.
Further, each terminal device 11-1 to 11-n may be added to or attached to an object, or may be carried or attached by a person. The product may be any one, for example, a vehicle such as an automobile, an electric appliance, or the like.

Further, a detection device other than the terminal devices 11-1 to 11-n may be provided in, for example, the statistic data processing device 12. In this configuration, the statistic data processing device 12 acquires and analyzes the data detected by the detection device.

[Statistical data processing device]
FIG. 2 is a block diagram showing a schematic configuration of a statistic data processing device 12 according to an embodiment of the present invention.
The statistic data processing device 12 includes an input unit 111, an output unit 112, a storage unit 113, a communication unit 114, and a control unit 115.
The control unit 115 includes a data acquisition unit 131, a statistic data processing unit 132, and a data output control unit 133.
The statistic data processing unit 132 includes a viewpoint setting unit 151, a scale setting unit 152, a statistic data group generation unit 153, an additional unit 154, and an extraction unit 155.

The input unit 111 inputs information from the outside. The input unit 111 has, for example, an operation unit that accepts an operation performed by a user (person), and inputs information according to the operation received by the operation unit. The input unit 111 is connected to, for example, an external device (for example, a recording medium) and inputs information output from the external device.
The output unit 112 outputs information. The output unit 112 has, for example, a screen, and displays (outputs) information on the screen. The output unit 112 is connected to, for example, an external device (for example, a recording medium) and outputs information to the external device.
The storage unit 113 stores information. In this embodiment, the storage unit 113 and the database 13 may be used arbitrarily.
The communication unit 114 communicates information. In the present embodiment, the communication unit 114 communicates information with another device (for example, terminal device 11-1 to 11-n or another detection device) via the network 21.

The control unit 115 performs various controls on the statistic data processing device 12.
In the present embodiment, the storage unit 113 stores information on a predetermined control program and its parameters. Further, the control unit 115 is configured by using a CPU (Central Processing Unit). Then, in the control unit 115, the CPU executes various controls by executing the control program stored in the storage unit 113 using the parameters stored in the storage unit 113.

The configuration of the statistic data processing device 12 including the processing units 111 to 115 shown in FIG. 2 is an example, and other configurations may be used. For example, the functional divisions of the processing units 111 to 115 are for convenience of explanation and are not necessarily limited to the configuration shown in FIG.

The function of the control unit 115 will be described.
The data acquisition unit 131 acquires a predetermined target data as the data to be analyzed.
As an example, the data acquisition unit 131 acquires data received from another device (for example, terminal device 11-1 to 11-n or another detection device) by the communication unit 114 as data to be analyzed. May be good.
As another example, the data acquisition unit 131 sequentially stores the data acquired in the past in the database 13, and when processing the data, acquires the data from the database 13 as the data to be analyzed. You may.
Further, as another example, a configuration may be used in which the data of a predetermined target is stored in the database 13 without going through the statistic data processing device 12, and in this case, the data acquisition unit 131 of the data At the time of processing, the data may be acquired from the database 13 as data to be analyzed.

The statistic data processing unit 132 performs statistical processing on the data acquired by the data acquisition unit 131, and acquires the resulting data (also referred to as “statistical data” in the present embodiment).
In addition, the statistic data processing unit 132 may process, for example, data acquired one after another in real time by the data acquisition unit 131 (a group of newly increasing data), and data acquisition. There may be a case where processing is performed on the past data (a collection of data that does not newly increase) acquired by the unit 131.

The data output control unit 133 controls the output unit 112 to output the data to be output. As this output, for example, a display output such as a character, a figure, or a graph is used.
Arbitrary data may be used as the data to be output. For example, the data acquired by the data acquisition unit 131 or the result data obtained by the processing by the statistic data processing unit 132 is used. May be done. Further, as the result data obtained by the processing by the statistic data processing unit 132, for example, the result data obtained by the processing by the statistic data group generation unit 153 and the result data obtained by the processing by the additional unit 154. One or more of the data or the result data obtained by the processing by the extraction unit 155 may be used.

The statistic data processing unit 132 will be described.
The viewpoint setting unit 151 sets a viewpoint (item) for the data to be analyzed. The viewpoint setting unit 151 may set two or more viewpoints for the data to be analyzed.
Here, as the viewpoint, any viewpoint may be used, and for example, time (time), area (region), device type, and the like may be used.
Further, the viewpoint setting unit 151 may set, for example, a predetermined viewpoint, or may set a viewpoint instructed by a user or the like. When the viewpoint is predetermined, for example, information for identifying the viewpoint is stored in the storage unit 113.

The scale setting unit 152 sets the scale (grain size) for each viewpoint. The scale setting unit 152 may set two or more scales for each viewpoint.
Here, as the scale, a scale having an arbitrary size may be used.
For example, as the time scale, a 1-second scale, a 1-minute scale, a 1-hour scale, a 1-day scale, or any other scale of any size may be used.
For example, the area scale may be a district scale, a city scale, a prefecture scale, a national scale, or any other scale of any size.
For example, as the scale of the device type, the scale of the model, the scale of the manufacturer, the scale of the OS (Operating System), or the scale of any other type (attribute) may be used.
For example, the scale setting unit 152 may set a predetermined scale for each viewpoint, or may set a scale instructed by a user or the like. When the scale is predetermined, for example, information for identifying the scale is stored in the storage unit 113.

The statistic data group generation unit 153 performs statistical processing on the data acquired by the data acquisition unit 131 based on the viewpoint set by the viewpoint setting unit 151 and the scale set by the scale setting unit 152. Then, a set of a plurality of statistic data (also referred to as "statistical data group" in the present embodiment) is generated.
In the present embodiment, the statistic data group generation unit 153 collects the data of the result of performing the multidimensional and multiscale data analysis into the statistic data group (in the present embodiment, the "multidimensional multiscale statistic data group"). Also called.).
Here, in the present embodiment, the dimension represents a viewpoint, and the multidimensional represents a plurality of viewpoints.
Further, in the present embodiment, the multi-scale indicates that there are a plurality of scales.

The additional unit 154 is a viewpoint when adding data (new data) acquired by the data acquisition unit 131 to the already generated statistic data group (for example, a multidimensional multi-scale statistic data group). By performing statistical processing based on the viewpoint set by the setting unit 151 and the scale set by the scale setting unit 152, the statistic data group after data addition (for example, the multidimensional multi-scale statistic data group) ) Is generated.

For example, when data that becomes a time series as a whole is independently transmitted from a plurality of devices (detection devices, etc.), the order of the data acquired by the data acquisition unit 131 may not be the time series. .. In this case, if the statistic data processing device 12 performs analysis after all of such time-series data has been acquired, the output of the processing result may be delayed. In such a case, the statistic data group generation unit 153 generates and outputs the statistic data group based on the already acquired data, and after the corresponding data newly arrives thereafter, the additional unit 154 generates the statistic data group. It is effective to acquire and output (for example, update the output contents) the statistic data group after adding the data.

The extraction unit 155 may use data that meets predetermined extraction conditions (for example, statistic data or statistic) based on the already generated statistic data group (for example, multidimensional multiscale statistic data group). It may be a data group.) Is extracted.

[Example of data structure of statistic data group]
FIG. 3 is a diagram schematically showing a data structure of an example of the statistic data group 201 according to the embodiment of the present invention.
In the example of FIG. 3, the statistic data group 201 is a multidimensional multi-scale statistic data group which is the result of processing time series data by the statistic data processing unit 132. The statistic data group 201 has a structure of distribution data (statistical data in the present embodiment) extended in multiple dimensions and multiscales for time series data.
In the example of FIG. 3, time, area, and device type are used as a plurality of viewpoints. In addition, a plurality of scales (multi-scales) are used for each viewpoint.

In the example of FIG. 3, a time scale s0, a time scale s1, and a time scale s2 are used as a plurality of scales related to time. As an example, the time scale s0 is "10 seconds", the time scale s1 is "20 seconds", and the time scale s2 is "40 seconds".
Further, as a plurality of scales related to the region, the region scale A1, the region scale A2, and the region scale A3 are used. As an example, the area scale A1 is "Tokyo", the area scale A2 is "Kanto", and the area scale A3 is "Japan (nationwide)".
Further, as a plurality of scales related to the device type, the device type scale D1 and the device type scale D2 are used. As an example, the device type scale D1 is a "specific model A1", and the device type scale D2 is a "specific manufacturer B1".

Further, in the example of FIG. 3, the direction (arrow) of the axis representing the time is shown. As the time range (time division), the time range t0, the time range t1, the time range t2, and the time range t3 are used. The time range t0 is "0 seconds or more and less than 10 seconds", the time range t1 is "10 seconds or more and less than 20 seconds", the time range t2 is "20 seconds or more and less than 30 seconds", and the time range t3 is " 30 seconds or more and less than 40 seconds. "
Any timing may be used as the initial value of the time (0 seconds in this embodiment).

Here, when one time scale, one area scale, and one device type scale are specified, and one time range corresponding to the time scale is specified, the statistic becomes a unit. Data (also referred to as "unit statistic data" in this embodiment) is specified.
One time range corresponding to the time scale is, in the present embodiment, a time range in which the upper limit value of the time range is a value corresponding to an integral multiple of the time scale. As a specific example, in the case of the time scale s0 (= 10 seconds), the time range t0 (= 0 seconds or more and less than 10 seconds), the time range t1 (= 10 seconds or more and less than 20 seconds), and the time range t2 (= 20 seconds). It is either the time range t3 (= 30 seconds or more and less than 40 seconds). In the case of the time scale s1 (= 20 seconds), it is either the time range t1 (= 10 seconds or more and less than 20 seconds) or the time range t3 (= 30 seconds or more and less than 40 seconds). In the case of the time scale s2 (= 40 seconds), the time range is t3 (= 30 seconds or more and less than 40 seconds).

The unit statistic data specified by one time scale, one area scale, one device type scale, and one time range according to the time scale is the maximum time in the time range. By performing a statistical calculation by the statistic data processing unit 132 for data belonging to the time scale from to the time retroactive to the time, belonging to the region scale, and belonging to the device type scale. Corresponds to the obtained statistic data.

In the example of FIG. 3, six unit statistic data 211 to 213 and 221 to 223 corresponding to the time scale s0 and the time range t3 and six unit statistic data 311 corresponding to the time scale s1 and the time range t3. Only 313 and 321 to 223 and the six unit statistic data 411 to 413 and 421 to 423 corresponding to the time scale s2 and the time range t3 are coded, and the other unit statistic data are coded. Is omitted.

As an example, the unit statistic data 211 corresponds to the time scale S0 (10 seconds), the area scale A1 (Tokyo), the device type scale D1 (specific model A1), and the time range t3 (30 seconds or more and less than 40 seconds). .. The unit statistic data 211 is statistic data obtained based on data in which the time belongs to 30 seconds or more and less than 40 seconds, the area belongs to Tokyo, and the device type belongs to a specific model A1. Is. That is, the data is generated in the terminal devices 11-1 to 11-n existing in Tokyo, the time is 30 seconds or more and less than 40 seconds, and the device type is the specific model A1. It means that the data is generated in ~ 11-n. The unit statistic data is statistic data obtained by using a set of such data.

As another example, the unit statistic data 322 is set to the time scale S1 (20 seconds), the area scale A2 (Kanto), the device type scale D2 (specific manufacturer B1), and the time range t3 (30 seconds or more and less than 40 seconds). Applicable. The unit statistic data 322 is statistic data obtained based on data in which the time belongs to 20 seconds or more and less than 40 seconds, the area belongs to Kanto, and the device type belongs to a specific manufacturer B1. Is. That is, the data is generated in the terminal devices 11-1 to 11-n existing in the Kanto region, the time is 20 seconds or more and less than 40 seconds, and the device type is the specific manufacturer B1. It means that the data is generated in ~ 11-n. The unit statistic data is statistic data obtained by using a set of such data.

As another example, the unit statistic data 413 is set to the time scale S2 (40 seconds), the area scale A3 (Japan), the device type scale D1 (specific model A1), and the time range t3 (30 seconds or more and less than 40 seconds). Applicable. The unit statistic data 413 is statistic data obtained based on data in which the time belongs to 0 seconds or more and less than 40 seconds, the area belongs to Japan, and the device type belongs to a specific model A1. Is. That is, the data is generated in the terminal devices 11-1 to 11-n existing in Japan, the time is 0 seconds or more and less than 40 seconds, and the device type is the specific model A1. It means that the data is generated in ~ 11-n. The unit statistic data is statistic data obtained by using a set of such data.

Here, as the unit statistic data, data of an arbitrary statistic may be used, for example, data of an arbitrary value related to order statistics may be used, or data of an average value may be used. May be good.
As the value related to the order statistic, for example, the median value may be used. In general, when a plurality of data to be processed are the same, it is considered that the processing time for acquiring the median value is shorter than the processing for acquiring (calculating) the average value.
Further, as the value related to the order statistics, for example, the value of the cumulative distribution function (CDF: Cumulative Distribution Function) may be used, or the value of the probability distribution function (PDF: Probability Density Function) may be used. ..

In the example of FIG. 3, the unit of each square (rectangular parallelepiped or cube) is unit statistic data (unit statistic data 211-213, 221-223, 31-13, 321-23, 411-413, 421-423, etc. ) Corresponds to.
The statistic data group generation unit 153 may acquire unit statistic data by using an arbitrary method. For example, refer to an existing technique of t-digest (for example, Non-Patent Document 1). .) May be used to calculate and obtain unit statistic data. In the present embodiment, the statistic data group is configured to include a plurality of unit statistic data.

Further, as the plurality of scales used for one viewpoint, for example, a plurality of scales having an inclusive relationship with each other may be used, or a plurality of scales having an inclusive relationship with each other may be used. Alternatively, a plurality of scales that are inclusively related to only a part may be used.
As an example of the scale of the area, there are "Tokyo", "Kanto", and "Japan" as a plurality of scales that are inclusive of each other.
As a plurality of scales that are not inclusive of each other, for example, "Tokyo", "Chiba", and "Ibaraki" are examples of the area scale.
As an example of the scale of the area, there are "Tokyo", "Kanto" (including Tokyo), and "Osaka" as a plurality of scales that are inclusively related to only a part.

[Example of additional processing]
FIG. 4 is a diagram for explaining an example of additional processing for the statistic data group 201 according to the embodiment of the present invention.
The additional process is, for example, a process of adding a sample (the data) to the appropriate unit statistic data when new time series data is generated. The additional processing may correspond to, for example, the processing of ADD included in the data structure in the t-digest technique (see, for example, Non-Patent Document 1 and the like). In the ADD process, the newly generated time series data is evaluated from each dimension, the subsets to which the data belongs are listed, and the operation of adding the data to each of the listed subsets is executed.

In the example of FIG. 4, the same as the statistic data group 201 shown in FIG. 3 is shown.
When new data is added to the statistic data group 201, the addition unit 154 regenerates all the unit statistic data to which the new data belongs by recalculating, and the statistic reflecting the result is reflected. A data group (data obtained by updating the statistic data group 201) is generated.
In the example of FIG. 4, it is assumed that the new data belongs to the time range t2, the area scale A1, and the device type scale D2. In this case, the additional unit 154 refers to the unit statistic data 511 belonging to the time scale S0, the unit statistic data 512 belonging to the time scale S1, and the unit statistic data 513 belonging to the time scale S2 for the new data, respectively. Is regenerated.
In the present embodiment, the additional unit 154 performs additional processing based on the viewpoint set by the viewpoint setting unit 151 and the scale set by the scale setting unit 152.

Here, the process of adding new data to the statistic data group 201 by the addition unit 154 is performed in response to the acquisition of new data by, for example, the data acquisition unit 131 in the statistic data processing device 12 ( A configuration that is performed (automatically) may be used, or a configuration that is performed in response to a predetermined instruction by a user or the like being received by the input unit 111 may be used.

[Example of extraction process]
FIG. 5 is a diagram for explaining an example of extraction processing for the statistic data group 201 according to the embodiment of the present invention.
The extraction process is, for example, a process of extracting data (for example, statistic data or a statistic data group) from one or more unit statistic data. The extraction process may correspond to, for example, the GET process of the data structure in the t-digest technique (see, for example, Non-Patent Document 1 and the like). In the GET process, a range of extraction sources (for example, a subset in the population) is specified, and an operation of acquiring statistics in the range is executed for the data included in the range.

In the example of FIG. 5, the same as the statistic data group 201 shown in FIG. 3 is shown.
The extraction unit 155 extracts the data (data of a predetermined target) included in the range of the extraction target (in the example of FIG. 5, the extraction range 611) from the statistic data group 201, and unit statistics based on the extracted data. It is generated by calculating the statistic data, and a statistic data group that is a set of the generated unit statistic data is generated.
In the present embodiment, the extraction unit 155 performs the extraction process based on the viewpoint set by the viewpoint setting unit 151 and the scale set by the scale setting unit 152.

Here, in the present embodiment, when a group of a plurality of unit statistic data and a group of another one or more unit statistic data have a one-to-one correspondence, the extraction unit 155 may use these plurality of units. The extraction process may be performed based on the statistic data, or the extraction process may be performed based on the other one or more unit statistic data, and the same result can be obtained in either case. Specifically, in the example of FIG. 5, a set of a plurality of unit statistic data is included in the time range t2, t3, the time scale s0, the area scales A1, A2, A3, and the device type scales D1, D2. It corresponds to a set of unit statistic data, and one or more other sets of unit statistic data are time range t2, t3 and time scale s1 and area scales A1, A2, A3 and device type scales D1 and D2. Corresponds to a set of 6 unit statistic data included in. In the extraction unit 155, the smaller the number of unit statistic data to be extracted, the more efficient the processing is.

The process of extracting the data included in the range (extraction range) of the extraction target from the statistic data group 201 by the extraction unit 155 is performed by the statistic data processing device 12, for example, under conditions predetermined in a control program or the like. A configuration that is (automatically) performed according to the determination that the condition is satisfied may be used, or a configuration that is performed according to the reception of a predetermined instruction by the user or the like by the input unit 111 may be used. It may be used.

[Example of processing performed in the statistic data processing device]
FIG. 6 is a diagram showing an example of a processing procedure for generating a statistic data group performed by the statistic data processing device 12 according to the embodiment of the present invention.
Note that this example is an example, and any other processing procedure may be used.

(Step S1)
The data acquisition unit 131 acquires the data to be analyzed.
(Step S2)
The viewpoint setting unit 151 sets the viewpoint used for analysis.
(Step S3)
The scale setting unit 152 sets the scale used for the analysis for each viewpoint.
(Step S4)
The statistic data group generation unit 153 generates a statistic data group for the data acquired by the data acquisition unit 131 based on the viewpoint set by the viewpoint setting unit 151 and the scale set by the scale setting unit 152. ..
(Step S5)
The data output control unit 133 controls to output arbitrary data (for example, a statistic data group) related to such processing to the screen.

Here, as the timing for performing the process for generating the statistic data group, an arbitrary timing may be used, for example, a predetermined fixed cycle timing may be used, or a user or the like. The timing according to the instruction received from may be used.
Further, the process for generating the statistic data group may be performed at different timings for each scale, for example, for any one or more viewpoints. As an example, when the scale "Tokyo" and the scale "Osaka" are used for the viewpoint "area", the timing of processing to generate the unit statistic data corresponding to the scale "Tokyo" and the unit statistics corresponding to the scale "Osaka". A configuration may be used in which the timing of the processing for generating the quantitative data is different, and after both of these processings are completed, a statistical data group in which the results of both of these processings are combined is generated.

FIG. 7 is a diagram showing an example of a processing procedure for adding data to a statistic data group performed by the statistic data processing device 12 according to the embodiment of the present invention.
Note that this example is an example, and any other processing procedure may be used.

(Step S11)
The addition unit 154 acquires a statistic data group to which data is added.
(Step S12)
The data acquisition unit 131 acquires the data to be analyzed (additional target data) to be added.
(Step S13)
The viewpoint setting unit 151 sets the viewpoint used for analysis.
(Step S14)
The scale setting unit 152 sets the scale used for the analysis for each viewpoint.
(Step S15)
The additional unit 154 transfers the data acquired by the data acquisition unit 131 to the acquired statistic data group based on the viewpoint set by the viewpoint setting unit 151 and the scale set by the scale setting unit 152. By adding, a statistic data group (statistic data group after data addition) is generated.
(Step S16)
The data output control unit 133 controls to output arbitrary data related to such processing (for example, a statistic data group after data addition) to the screen.

Here, as the timing for performing the process for adding data to the statistic data group, an arbitrary timing may be used, for example, a predetermined fixed cycle timing may be used. Alternatively, the timing according to the instruction received from the user or the like may be used.
As an example, first, a process for generating the statistic data group shown in FIG. 6 is performed, and then data is added to the statistic data group shown in FIG. 7 by repeating at a fixed cycle timing. A configuration may be used in which processing is performed. In this configuration, first, a statistic data group is generated based on the already acquired data, and then the statistic data group is updated using the newly acquired data at regular intervals. Can be done.

FIG. 8 is a diagram showing an example of a processing procedure for extracting data from a statistic data group performed by the statistic data processing device 12 according to the embodiment of the present invention.
Note that this example is an example, and any other processing procedure may be used.

(Step S21)
The extraction unit 155 acquires a statistic data group from which the data is extracted.
(Step S22)
The extraction unit 155 sets the extraction range. The extraction range is specified, for example, using a time range and a scale for each viewpoint.
(Step S23)
The viewpoint setting unit 151 sets the viewpoint used for analysis.
(Step S24)
The scale setting unit 152 sets the scale used for the analysis for each viewpoint.
(Step S25)
The extraction unit 155 extracts data included in the set extraction range from the acquired statistic data group. Then, the extraction unit 155 sets the statistic data group (statistical data group in the extraction range) of the extracted data based on the viewpoint set by the viewpoint setting unit 151 and the scale set by the scale setting unit 152. Generate. The data included in the extraction range may be, for example, extracted from the extraction range, or the corresponding data (substantially the same data) may be extracted from another range.
(Step S26)
The data output control unit 133 controls to output arbitrary data related to such processing (for example, a statistic data group in the extraction range) to the screen.

Here, as the timing for performing the process for extracting data from the statistic data group, an arbitrary timing may be used, for example, a predetermined fixed cycle timing may be used, or The timing according to the instruction received from the user or the like may be used.

[Summary of the first embodiment]
As described above, in the data processing system 1 according to the present embodiment, it is possible to perform multidimensional and multiscale data analysis in the statistic data processing device 12.
When the statistic data processing device 12 according to the present embodiment can calculate and acquire a value distribution (statistic) for a multidimensional and multiscale subset of a population of time series data. One or more of the series data, the acquired statistic (unit statistic data), or the statistic data group is stored and stored in the database 13 (or the storage unit 113).

In the data structure of the statistic data group according to the present embodiment, for example, it is possible to grasp the statistic data or search the statistic data in multidimensional and multiscale.
In the data structure of the statistic data group according to the present embodiment, for example, it is possible to monitor and detect an event such as the occurrence of an abnormality from a plurality of viewpoints and a plurality of scales, and various viewpoints and various scales. It is possible to monitor and detect scale events in parallel. In this case, in the data structure of the statistic data group according to the present embodiment, for example, it is possible to determine which viewpoint and scale the event has occurred. As a specific example, when an event occurs on a wide area scale, it can be estimated that the event is caused by a wide area, and when an event occurs on a specific area scale, the specific event occurs. It can be presumed that the cause is limited to the area of.

Here, in the present embodiment, the configuration for processing the statistic data for the values related to the terminal devices 11-1 to 11-n is set, but the configuration for processing the statistic data for other arbitrary values may be implemented. ..

(Second Embodiment)
[Data processing system]
FIG. 9 is a block diagram showing a schematic configuration of a data processing system 1001 according to an embodiment (second embodiment) of the present invention.
The data processing system 1001 includes n terminal devices 11-11 to 11-n, a statistic data processing device 1011, a database 1012, a unit statistic data generation device 1021, and a network 21.
Here, the terminal devices 11-11 to 11-n and the network 21 are the same as those shown in FIG. 1, and are designated by the same reference numerals for convenience of explanation.
Further, the database 1012 has a function of storing data in the same manner as the database 13 shown in FIG.

The difference between the data processing system 1001 according to the present embodiment and the data processing system 1 shown in FIG. 1 will be described.
In the example of FIG. 9, the point that the unit statistic data generation device 1021 is provided and a part of the processing performed by the statistic data processing device 1011 are different from the example of FIG.

The unit statistic data generation device 1021 has a function of performing all or a part of the processing that can be executed by the t-digest technique (see, for example, Non-Patent Document 1 and the like).
In the present embodiment, the unit statistic data generation device 1021 uses the t-digest technique to unitize the data based on the data to be analyzed, the information that identifies the viewpoint, and the information that specifies the scale of each viewpoint. It has a function to generate statistic data.
Further, in the present embodiment, the unit statistic data generation device 1021 has a function of adding data to the unit statistic data by using the t-digest technique.
Further, in the present embodiment, the unit statistic data generation device 1021 has a function of extracting data from the unit statistic data by using the t-digest technique. As the function, for example, the multiple distribution search function in t-digest may be used.
As the unit statistic data generation device 1021, for example, the unit statistic data generation device 1021 may be managed by a person who manages the statistic data processing device 1011 or is provided by another person. The configuration may be used.

The statistic data processing device 1011 differs from the statistic data processing device 12 shown in FIG. 1 in that the unit statistic data generation device 1021 performs the processing performed by the unit statistic data generation device 1021. It is configured to request and receive the processing result.

Here, for convenience of explanation, the functional blocks shown in FIG. 2 will be used for description.
The statistic data processing device 1011 according to the present embodiment generally has a functional block similar to that shown in FIG.

In the present embodiment, the statistic data group generation unit 153 transmits a signal requesting a process for generating unit statistic data to the unit statistic data generation device 1021 by the communication unit 114 via the network 21. The signal contains information necessary to generate unit statistic data, such as data to be analyzed (or information that identifies it), information that identifies one or more viewpoints, and each. Contains information that identifies the scale of the viewpoint.
When the unit statistic data generation device 1021 receives the signal of such a request, the unit statistic data generation device 1021 generates the unit statistic data in response to the request, and transmits the signal including the generated unit statistic data via the network 21. , Transmit to the statistic data processing device 1011.
The statistic data group generation unit 153 uses (uses) the unit statistic data received from the unit statistic data generation device 1021 by the communication unit 114 to generate a statistic data group.

In the present embodiment, the additional unit 154 transmits a signal requesting a process of adding data to the unit statistic data by the communication unit 114 to the unit statistic data generation device 1021 via the network 21. The signal contains information necessary to add data to the unit statistic data, for example, the unit statistic data to be added (or information that identifies it), and the data to be added (or information to be added). , Information that identifies it) is included.
When the unit statistic data generation device 1021 receives the signal of such a request, the unit statistic data generation device 1021 generates the unit statistic data after adding the data in response to the request, and transmits the signal including the generated unit statistic data to the network. It is transmitted to the statistic data processing device 1011 via 21.
The addition unit 154 uses (uses) the unit statistic data received by the communication unit 114 from the unit statistic data generation device 1021 to generate a statistic data group after data addition.

In the present embodiment, the extraction unit 155 sends a signal requesting a process for extracting the unit statistic data (or data contained therein) to the unit statistic data generation device 1021 by the communication unit 114 via the network 21. Send. The signal contains information necessary for extracting unit statistic data (or data contained therein), for example, information for identifying unit statistic data (or data contained therein) to be extracted (or data contained therein). For example, information that specifies the extraction range) is included.
When the unit statistic data generation device 1021 receives the signal of such a request, the unit statistic data generation device 1021 transmits a signal including the unit statistic data (or data contained therein) extracted in response to the request via the network 21. , Transmit to the statistic data processing device 1011.
The extraction unit 155 uses (uses) the unit statistic data received by the communication unit 114 from the unit statistic data generation device 1021 to generate a statistic data group in the extraction range.

[Summary of the second embodiment]
As described above, similarly to the first embodiment, the statistic data processing device 1011 according to the present embodiment can perform multidimensional and multiscale data analysis.

(Summary of the above embodiments)
As a configuration example, a statistic data processing unit that generates a statistic data group having statistic data based on data included in each combination of viewpoints and scales for a plurality of different viewpoints and a plurality of different scales for each viewpoint (a statistic data processing unit ( In the example of FIG. 2, it is a statistic data processing apparatus (statistical data processing apparatus 12, 1011 in the examples of FIGS. 1 and 9) including the statistic data processing unit 132).
As a configuration example, the statistic data processing unit generates a statistic data group in which new data is added to the statistic data group (in the example of FIG. 2, the function of the additional unit 154).
As a configuration example, the statistic data processing unit generates a statistic data group based on the data included in the extraction range from the statistic data group (in the example of FIG. 2, the function of the extraction unit 155).
As a configuration example, the statistic data processing unit is a unit statistic data generation device that generates unit statistic data which is statistic data in which one scale is specified for each viewpoint (in the example of FIG. 9, the unit is a unit). A statistic data group is generated using the unit statistic data generated by the statistic data generation device 1021).
As a configuration example, the statistic data processing unit generates a statistic data group having statistic data based on the data included in each combination of viewpoints and scales for a plurality of different viewpoints and a plurality of different scales for each viewpoint. This is a statistic data processing method (in the examples of FIGS. 1 and 9, the processing method performed by the statistic data processing devices 12 and 1011).
As a configuration example, the statistic data processing unit generates a statistic data group having statistic data based on the data included in each combination of viewpoints and scales for a plurality of different viewpoints and a plurality of different scales for each viewpoint. This is a program for causing a computer (in the example of FIGS. 1 and 9, the computer constituting the statistic data processing devices 12 and 1011) to execute the step.
As a configuration example, it is a data structure of a statistic data group, and for a plurality of different viewpoints and a plurality of different scales for each viewpoint, for each combination of viewpoints and scales, a statistic based on the data included in each combination. It is a data structure having data (in the example of FIG. 3, the data structure of the statistic data group 201).

A program for realizing the functions of the devices (for example, the statistic data processing devices 12, 1011 and the like) according to the above-described embodiment is recorded on a computer-readable recording medium (storage medium), and the recording medium is recorded. The processing may be performed by loading the program recorded in the computer system into a computer system and executing the program.
The term "computer system" as used herein may include hardware such as an operating system or peripheral devices.
The "computer-readable recording medium" includes a flexible disk, a photomagnetic disk, a ROM (Read Only Memory), a writable non-volatile memory such as a flash memory, and a portable medium such as a DVD (Digital Versaille Disc). A storage device such as a hard disk built into a computer system.
Further, the "computer-readable recording medium" is a volatile memory (for example, DRAM (for example, DRAM) inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. It also includes those that hold the program for a certain period of time, such as Dynamic Random Access Memory)).
Further, the above program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the above-mentioned functions. Further, the above program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like within a range not deviating from the gist of the present invention are also included.

1,1001 ... Data processing system, 11-1 to 11-n ... Terminal device, 12, 1011 ... Statistics data processing device, 13, 1012 ... Database, 21 ... Network, 111 ... Input unit, 112 ... Output unit, 113 ... Storage unit, 114 ... Communication unit, 115 ... Control unit, 131 ... Data acquisition unit, 132 ... Statistics data processing unit, 133 ... Data output control unit, 151 ... Viewpoint setting unit, 152 ... Scale setting unit, 153 ... Statistics Quantity data group generation unit, 154 ... Addition unit, 155 ... Extraction unit, 201 ... Statistics data group, 211-213, 221-223, 31-13, 312-323, 411-413, 421-423, 511-513 ... Unit statistic data, 611 ... Extraction range, 1021 ... Unit statistic data generator

Claims (4)

A plurality of different viewpoints including the first viewpoint of time , a plurality of different scales for each of the viewpoints including a plurality of different time scales of the first viewpoint , and an upper limit of one time range according to each of the time scales. With respect to the plurality of time ranges having values corresponding to integral multiples of the time scale, one time scale and one time range are specified with respect to the first viewpoint, and other than the first viewpoint. A statistic data processing unit that generates a statistic data group having statistic data based on data included in each combination in which one scale is specified for each of the viewpoints .
Shown in parallel the plurality of the time range in the first direction and corresponding to the each of the viewpoint which is different from the first the said scale said plurality of different for each of the viewpoints other than the viewpoints first direction In an output mode in which the plurality of statistical data included in one time scale are shown in parallel in a direction, and the group of the statistic data is shown in parallel in a direction different from the first direction for the plurality of different time scales. A data output control unit that outputs the statistic data group generated by the statistic data processing unit to a screen.
Statistic data processor. The plurality of different viewpoints other than the first viewpoint include a second viewpoint different from the first viewpoint and a third viewpoint different from the first viewpoint and the second viewpoint.
In the output mode, the plurality of time ranges are shown in parallel in the first direction with respect to the first viewpoint, and the plurality of scales are arranged in parallel in a second direction different from the first direction with respect to the second viewpoint. The plurality of scales are shown in parallel in the first direction and a third direction different from the second direction with respect to the third viewpoint, and the plurality of time scales are shown with respect to the first viewpoint. This is an output mode shown in parallel in the same direction as the second direction.
The statistic data processing apparatus according to claim 1. The statistic data processing unit responds to a plurality of different viewpoints including the first viewpoint of time , a plurality of different scales for each of the viewpoints including a plurality of different time scales of the first viewpoint , and each of the time scales. With respect to the plurality of time ranges in which the upper limit value of one time range is set to a value corresponding to an integral multiple of the time scale, one time scale and one time range are specified with respect to the first viewpoint. At the same time , a statistic data group having statistic data based on the data included in each combination in which one scale is specified for each of the viewpoints other than the first viewpoint is generated.
Data output control unit, said plurality of said time range shown in parallel in the first direction, and each different from each of the viewpoints of the plurality of different said the scale first direction about the other than the first aspect In parallel in the direction corresponding to the above-mentioned viewpoint, a group of the plurality of the statistic data included in the one time scale is displayed in parallel in a direction different from the first direction for the plurality of different time scales. The statistic data group generated by the statistic data processing unit is output to the screen in the output mode shown in the above.
Statistic data processing method. The statistic data processing unit responds to a plurality of different viewpoints including the first viewpoint of time , a plurality of different scales for each of the viewpoints including a plurality of different time scales of the first viewpoint , and each of the time scales. With respect to the plurality of time ranges in which the upper limit value of one time range is set to a value corresponding to an integral multiple of the time scale, one time scale and one time range are specified with respect to the first viewpoint. And a step of generating a statistic data group having statistic data based on the data included in each combination in which one scale is specified for each of the viewpoints other than the first viewpoint .
Data output control unit, said plurality of said time range shown in parallel in the first direction, and each different from each of the viewpoints of the plurality of different said the scale first direction about the other than the first aspect In parallel in the direction corresponding to the above-mentioned viewpoint, a group of the plurality of the statistic data included in the one time scale is displayed in parallel in a direction different from the first direction for the plurality of different time scales. In the output mode shown in the above, the step of outputting the statistic data group generated by the statistic data processing unit to the screen, and
A program that lets your computer run.

Images