KR101757253B1 - Method and apparatus for managing multidimensional data - Google Patents

Abstract

According to the present invention, optimized distributed management of multidimensional data is performed on the basis of a hash technique. Thus, new multidimensional data, which enables high-speed data loading by minimizing a loading time and cost unnecessarily consumed for data loading, The present invention proposes an apparatus and method for managing multidimensional data that realizes a distributed management plan.

Description

[0001] METHOD AND APPARATUS FOR MANAGING MULTIDIMENSIONAL DATA [0002]

The present invention relates to a data management technique based on a multi-dimensional database, and more particularly, to a new multi-dimensional data distribution management method capable of performing optimized distributed management of multi-dimensional data based on a hash technique .

A multidimensional database is a database that manages data having a large number of attributes (attributes) in order to model large-scale multidimensional data such as scientific information and medical information, and to support query processing and computation.

Although it is possible to analyze and store large-scale multi-dimensional data based on such a multi-dimensional database, since the capacity of data used in recent experimental data and scientific research is increasing due to the development of observation equipment and related technologies, It must take a long time to complete.

That is, in the case of multidimensional data, since a single record has various characteristics, in order to load it into the database, conversion to a format of the multidimensional database to be loaded, distribution process, and sorting process must be performed.

In the case of a multidimensional database, data is managed using a cluster, and multidimensional data is managed in a data file format such as CSV in general. If multidimensional data is input as HDF5 data, the format change must be performed before loading. Thereafter, the multivariate data of the CSV form is distributed and placed in the corresponding instance DB server. In this case, a round robin method is mainly used as a distributed method.

When the data distribution is completed, the DB server of each instance of the multidimensional database arranges each chunk by sorting the distributed data. In this case, the distributed multidimensional data can not be managed. After performing the redistribution process of transferring the corresponding data, which is difficult to manage, to another instance DB server, the multidimensional data is sorted, converted into the database format, and loaded into the corresponding instance DB server.

As can be seen from the foregoing description, since the multidimensional data that can not be easily managed by the DB server of each instance of the multidimensional database can be distributed when the existing storing process is performed, an unnecessary relocation process must be performed in order to relocate the corresponding data. There is a problem that network bottleneck occurs due to occurrence of I / O.

In addition, since the communication between the instance DB servers occurs in order to relocate the distributed multidimensional data, there is a limit to increase the overhead, thereby increasing the loading cost and time.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an apparatus and method for performing optimized distributed management of multidimensional data based on a hash technique, Dimensional data distribution management method that enables high-speed data loading by minimizing the loading cost and time.

According to an aspect of the present invention, there is provided a multidimensional data management apparatus for generating multidimensional data, the multidimensional data including at least one specific dimension data and at least one chunk, A distribution management unit for distributing the distribution data; A sort management unit for sorting the specific dimension data distributed to each chunk based on the dimension of the specific dimension data distributed by the chunks; And a storage management unit for converting the sorted specific dimension data into a predetermined format and storing the converted data in an instance DB server associated with the chunk.

Wherein the distribution management unit calculates a final hash value corresponding to the specific hash information based on a result of calculating a hash value for each dimension of the specific dimension data based on a specific hash function.

Wherein the specific dimension data includes a position value corresponding to a position in the dimension for each dimension, and the specific hash function is a hash value of the second dimension calculated from the hash value before the first dimension for which the present hash value is to be calculated, A prime number, a position value of the first dimension, and a chunk range of the first dimension as parameters.

Wherein the specific hash function includes a first result obtained by multiplying the hash value of the second dimension by the prime number and a difference value between the position value of the first dimension and the minimum position value of the first dimension, (XOR) with respect to a second result value obtained by dividing the result by the chunk range.

The hash value of the first dimension and the hash value of the first dimension are calculated using the hash value of the second dimension based on the specific hash function, And performing a modular operation based on the total number of servers to calculate the final solution value.

Wherein the size of the final hash value is matched with a chunk number that identifies the chunk and the distribution management unit is operable to determine whether the chunk number of the chunk number matches the last hash value, And the like.

Wherein the sorting management unit determines a sorting order of the specific dimension data divided by the chunks based on a predetermined dimension management method, arranges the position information values of the dimension having the fastest sorting order, Dimensional positional information value of the dimension.

The storage management unit converts the sorted specific dimension data based on a predetermined format to generate specific dimension conversion data and stores the specific dimension conversion data in an instance DB server related to the chunk .

According to another aspect of the present invention, there is provided a method of operating a multidimensional data management apparatus, the method comprising: storing at least one specific dimension data in at least one chunk based on specific hash information for each specific dimension data included in the multidimensional data; A distribution management step of distributing the distribution data; A sort management step of sorting specific dimension data distributed to each chunk based on a dimension of specific dimension data distributed for each chunk; And a storage management step of converting the sorted specific dimension data into a predetermined format and storing the converted data in an instance DB server associated with the chunk.

Wherein the distribution management step calculates a final solution value corresponding to the specific solution information based on a result of calculating a hash value for each dimension of the specific dimension data based on the specific solution function.

Wherein the specific dimension data includes a position value corresponding to a position in the dimension for each dimension, and the specific hash function is a hash value of the second dimension calculated from the hash value before the first dimension for which the present hash value is to be calculated, A prime number, a position value of the first dimension, and a chunk range of the first dimension as parameters.

Wherein the specific hash function includes a first result obtained by multiplying the hash value of the second dimension by the prime number and a difference value between the position value of the first dimension and the minimum position value of the first dimension, (XOR) with respect to a second result value obtained by dividing the result by the chunk range.

The hash value of the first dimension is calculated using the hash value of the second dimension based on the specific hash function based on the hash value of the first dimension and the instance managing the multidimensional data And the final solution value is calculated by performing a modular operation based on the total number of DB servers.

Wherein the size of the final hash value is matched with a chunk number that identifies the chunk, and the distribution managing step further includes the step of determining whether or not the chunk number of the chunk number matches the final hash value, Dimensional data is distributed.

Wherein the sorting management step comprises: determining a sorting order of the specific dimension data divided by the chunks based on a predetermined dimension management method; sorting the position information values of the dimension having the fastest sorting order; And arranges position information values of the dimension.

The storage management step may include converting the sorted specific dimension data based on a predetermined specific format to generate specific dimension conversion data and storing the specific dimension conversion data in an instance DB server related to the chunk. do.

According to the apparatus and method for managing multidimensional data of the present invention, optimized distribution management for multidimensional data is performed based on a hash technique, The present invention can provide a new multidimensional data distribution management method that enables high-speed data loading by minimizing the cost.

1 is a diagram illustrating a communication environment to which a multidimensional data management apparatus according to an embodiment of the present invention is applied.
2 is a diagram showing a schematic configuration diagram of a multidimensional database according to an embodiment of the present invention.
3 is a diagram showing a schematic configuration diagram of a multidimensional data management apparatus according to an embodiment of the present invention.
4 is a diagram illustrating an example of multi-dimensional data to be distributed based on a hash technique according to an embodiment of the present invention.
5 and 6 are views illustrating an example of calculating specific hash information based on a hash technique according to an embodiment of the present invention.
7 is a diagram illustrating an example of storing specific dimension data distributed to chunks according to an embodiment of the present invention.
8 is a view showing another example in which specific dimension data distributed to chunks according to an embodiment of the present invention is stored.
9 is a flowchart illustrating a hash-based data distribution management service according to an embodiment of the present invention.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings. The spirit of the present invention should be construed as extending to all modifications, equivalents, and alternatives in addition to the appended drawings.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

First, a communication environment to which an apparatus for performing data management based on a multi-dimensional database of the present invention (hereinafter referred to as a multi-dimensional data management apparatus) to which the present invention is applied will be described with reference to FIG.

1, the communication environment to which the present invention is applied may have a configuration including a multidimensional data management apparatus 100, a user terminal 200, and a multi-dimensional database 300.

The multidimensional data management apparatus 100 is an apparatus for performing optimized distributed management of multidimensional data through interworking with the multidimensional database 300. The multidimensional data management apparatus 100 is a device for providing services to be provided by the present invention, Based data distribution management service (hereinafter referred to as " hash-based data distribution management service ") that performs optimized distributed management on multi-dimensional data based on a hash technique that minimizes load time and cost unnecessarily consumed for multi- ). ≪ / RTI >

The multidimensional data management apparatus 100 may be embodied in the multidimensional database 300 or may be implemented separately as a separate apparatus. In the embodiment of the present invention, the multidimensional data management apparatus 100 is separated from the multidimensional database 300 and independently exists.

That is, when the multidimensional data management apparatus 100 receives multidimensional data from the user terminal 200, the multidimensional data management apparatus 100 performs an algorithm (hereinafter referred to as a distribution management algorithm) for multidimensional data distribution management in cooperation with the multidimensional database 300 Dimensional data is distributed and stored in each instance DB server of the multidimensional database 300 to generate a distributed management execution result and provides it to the user terminal 200. [

The user terminal 200 may be a terminal used by a user to receive a hash-based data distribution management service, and may transmit the multidimensional data to the multidimensional data management apparatus 100 to request the generation of the distributed management execution result.

Here, the multidimensional data may include all data represented by various formats (HDF5, HDF4, netCDF, CSV, etc.) as data to be stored in the multidimensional database 300 by the user. In the present invention, All data represented by multidimensional data is targeted.

The user terminal 200 may be a terminal of an organization / organization desiring to receive a hash-based data distribution management service or a terminal used by a general individual.

The multidimensional database 300 is a database that manages data having a large number of attribute items in order to model a large-capacity multidimensional data (e.g., scientific information, medical information, etc.) and support query processing and calculation.

When the multidimensional database 300 is requested by the multidimensional data management apparatus 100 to perform the corresponding distributed management procedure according to the distributed management algorithm so that the hash-based data distribution management service can be provided, To disperse and store the multidimensional data.

In this regard, FIG. 2 shows an example of a multi-dimensional database 300 that operates in conjunction with the multi-dimensional data management apparatus 100 so that a hash-based data distribution management service can be provided.

2, the multidimensional database 300 of the present invention is operated by at least one instance DB server (0, 1 ... n), and each instance DB server (0, 1 ... n ) Are allocated to the array storages AS0,1 ... n, so that it is possible to store and manage the multi-dimensional data distributed according to the execution of the dispersion management procedure.

In order to perform such a distributed management procedure, multidimensional data must be divided and managed, so that chunk arrays (hereinafter referred to as chunks) corresponding to the array storages AS0,1 ... n are allocated and parallelized.

In other words, multidimensional data is distributed to each chunk of the instance DB servers (0, 1 ... n) in the process of executing the distributed management procedure, and the corresponding array storage (AS0,1 ... n ), The multidimensional transform data to be finally generated is stored after the distribution management procedure (distribution, alignment, and transformation) for the corresponding multidimensional data distributed to the chunks is completed.

Accordingly, the multidimensional database 300 can support the parallel distributed operation on the multidimensional data by substantially executing the operation corresponding to the distribution management procedure through interlocking with the multidimensional data management apparatus 100. [

In the embodiment of the present invention, the multidimensional data management apparatus 100 and the multidimensional database 300 are interlocked with each other to perform the distribution management procedure according to the distribution management algorithm. However, the present invention is not limited to this, When the function of the data management apparatus 100 is implemented, the multidimensional database 300 alone may also provide a service by performing a distribution management procedure.

Hereinafter, the configuration of a multidimensional data management apparatus according to an embodiment of the present invention will be described in detail with reference to FIG.

3, the multidimensional data management apparatus 100 according to the present invention includes at least one chunk based on hash information of at least one specific dimension data included in the multidimensional data, A sort management unit 120 for sorting specific dimension data distributed to each chunk on the basis of the dimension of specific dimension data distributed for each chunk, And a storage management unit 130 for converting the data into a predetermined format and storing the data in an instance DB server associated with the chunk.

In addition, the multidimensional data management apparatus 100 according to the present invention stores all information (e.g., multidimensional data, hash functions, distribution management results, and the like) generated and transmitted and received to provide a hash-based data distribution management service, And may further include a storage unit 140 provided on demand.

All or at least a part of the multidimensional data management apparatus 100 including the distribution management unit 110, the alignment management unit 120, the storage management unit 130 and the storage unit 140 may be a software module type Or a hardware module, or a combination of a software module and a hardware module.

As a result, the multidimensional data management apparatus 100 according to the embodiment of the present invention performs optimized distributed management of multi-dimensional data based on the hash technique, The present invention provides a new multidimensional data distribution management method that enables high-speed data loading by minimizing the cost of the multidimensional data management apparatus 100. Hereinafter, each configuration in the multidimensional data management apparatus 100 will be described in detail.

The distribution management unit 110 distributes the multidimensional data based on the hash technique.

More specifically, when the user selects the multidimensional data to be stored in the multidimensional database 300, the distribution management unit 110 generates at least one chunk based on the specific hash information for each at least one specific dimensional data included in the multidimensional data, And distributes the specific dimension data to the target.

That is, the distribution management unit 110 calculates the final hash value corresponding to the specific hash value based on the result of calculating the dimensionally hash value of the specific dimension data based on the specific hash function, And distributes the specific dimension data by checking the corresponding chunk to be distributed.

Here, the specific hash function is used to find a hash value by dimension of specific dimension data, and can be defined as Equation (1).

[Equation 1]

는, 비트 연산의 XOR이다.Here, H _i is a hash value for each dimension, and the initial value H ₀ of all dimensions is initialized to "0". H _d is the final hash value finally calculated in the d dimension based on the hash value calculated for each dimension when the number of dimensions is d. Prime is a prime number for finding non-redundant hash values, and V _i is a position data value corresponding to a position in the i-th dimension of the multidimensional data. Min _i is the minimum position data value in the corresponding dimension, ChunkInterval is the chunk range of the ith dimension, and N is the number of the instance DB servers in the multidimensional database.

Is the XOR of the bit operation.

When the calculation of the hash value for each dimension of specific dimension data is completed using Equation (1), a modular operation (Equation 2) is used to calculate the final hash value.

&Quot; (2) "

Here, H _d is the final hash value finally calculated in the d dimension based on the hash value calculated for each dimension when the number of dimensions is d, and N is the number of the instance DB servers of the multidimensional database.

In this regard, FIG. 4 shows an example of multi-dimensional data to be distributed based on a hash technique. In FIG. 4, since only information on dimensions is utilized, attribute values for multidimensional data are not separately described.

4, when the multidimensional data selected by the user is two-dimensional, the X-axis is one-dimensional, the Y-axis is two-dimensional, and the chunk size (ChunkInterval) 3 ", the number N of the instance DB servers is 4, the prime number is" 991 ", and the minimum position data value per dimension is (0, 0), at least one A description will be given of a process of distributing specific dimension data to at least one chunk based on specific hash information per specific dimension data (D00-D55) of FIG.

The distribution management unit 110 first performs a process for finding a hash value for each dimension of the specific dimension data D00-D55 included in the multidimensional data.

5, the distribution manager 110 stores a first parameter for calculating a hash value (H ₁ ) in one dimension (X axis) of the specific dimension data D00, that is, the initial value of H ₀ = "0", the one-dimensional position value corresponding to a position in the (X axis) V ₁ = "0", the Min ₁ minimum position data value in the first dimension (X-axis) = "0 Quot ;, and ChunkInterval = "3 ", which is the size of the chunk in one dimension (X axis).

Then, the distribution management unit 110 calculates the hash value (H ₁ ) in one dimension (X axis) of the specific dimension data D00 by applying the first parameter to Equation ( ₁ ) as follows.

When the calculation of the hash value H ₁ is completed as described above, the distribution management unit 110 calculates the second parameter for calculating the hash value (H ₂ ) in two dimensions (Y axis) of the specific dimension data D00, one-dimensional hash values at (X-axis) H ₁ = "0", a small number of V ₂ position value corresponding to the position on the (Prime Number) = "991", 2-D (Y-axis) = "0", Min ₂ = "0", which is the minimum position data value in the two-dimensional (Y-axis), and ChunkInterval = "3", the size of the chunk in the two-dimensional (Y-axis).

Then, the distribution management unit 110 calculates the hash value (H ₂ ) in two dimensions (Y axis) of the specific dimension data D00 by applying the second parameter to Equation (1) as follows.

After all, the second parameter, geumbeon hash value than the first, and the hash value calculated one-dimensional hash value of (X-axis) (H _1), small number of two-dimensional (Y-axis) to calculate the (H ₂₎ (Prime Number), At least one of a position value V ₂ corresponding to a position in two dimensions (Y axis), a chunk range (ChunkInterval) in two dimensions (Y axis), and a minimum position data value Min ₂ in two dimensions (Y axis) One is included.

)과, 2차원(Y축)에서의 위치에 대응하는 위치값(V₂)과 2차원(Y축)에서의 최소 위치 데이터값(Min₂)을 청크범위(ChunkInterval)로 나눗셈한 제2 결과값(

)에 대한 배타적논리합(XOR)에 대응하게 된다. Thus, when the second parameter is applied to the specific solution function H _i , the specific solution function H ₂ is set to have the hash value H ₁ in one dimension (X axis) of the specific dimension data D00, And a first result (Prime Number)

A second result obtained by dividing the position value V ₂ corresponding to the position in the two-dimensional (Y axis) and the minimum position data value Min ₂ in the two-dimensional (Y axis) by the chunk range (ChunkInterval) value(

(XOR) to the exclusive logical sum (XOR).

That is, if the second hash value (H ₂₎ of the two-dimensional (Y-axis) to calculate a geumbeon hash value of the two-dimensional (X-axis, Y-axis) included in the specific-dimensional data (D00), first, a hash value (H ₁ (Y-axis) hash value H ₂ is applied to the parameter of Equation (2), which is a modular operation to be described later, so that the specific dimension data D00), all the dimensions of the specific dimension data D00, that is, the (X-axis, Y-axis) are reflected and the chunk can be determined.

After the calculation of the hash values H ₁ and H _{2 for each} dimension of the specific dimension data D00 is completed, the distribution management unit 110 calculates the final hash value of the specific dimension data D00 (2) "

That is, the distribution management unit 110 stores the third parameter for calculating the final hash value of the specific dimension data D00, that is, the hash value (H ₂ ) in the two-dimensional (Y axis) And N = "4 ", which is the total number of the instance DB servers managing the multidimensional data.

Thereafter, the distribution management unit 110 calculates the final solution value of the specific dimension data D00 by applying the third parameter to Equation (2) as follows.

In the end, the final solution when the value is one-dimensional if using a hash value (H ₁₎ in the (X axis), a two-dimensional hash value in the (Y-axis) (H ₂₎ is output, in 2-D (Y-axis) The hash value (H ₂ ) of the instance DB server and the total number (N) of the instance DB servers.

When the calculation of the final solution value is completed, the distribution management unit 110 distributes the specific dimension data D00 related to the final solution value to the corresponding chunk having the chunk number matched with the final solution value.

At this time, it is preferable that the sizes (0, 1, 2 ...) of the final hash value and the chunk numbers (0, 1, 2, ...) for distinguishing the chunks are matched in advance. For example, if the size of the final hash value is "0 ", the chunk number is also matched with" 0 ", and if the size of the final hash value is "1 ", the chunk number is also matched with" 1 ".

If the final hash value of the specific dimension data D00 is calculated as "0 ", the corresponding chunk having the chunk number" 0 "matching the size of the final hash value is detected and the specific dimension data D00 is distributed It will be possible to do.

The process of calculating the final solution value based on the dimension-specific hash value of the specific dimension data D00 and distributing the specific dimension data D00 to the corresponding chunk after detecting the corresponding chunk is shown in FIGS. 5 and 6 The final hash value of the specific dimension data D00, D01, D02, D10, D11, D12, D20, D21, and D22 is calculated as shown in FIG. 7, (D03, D04, D05, D13, D14, D15, D23, D24, and D25) will be distributed to the chunk having the chunk number "0 " D34, D35, D43, D44, D45, D53, D54, and D55 will be distributed to the chunk having the chunk number "1 " D30, D31, D32, D40, D41, D42, D50, D51 (D30, D31, D32, D40, D51) will be distributed to chunks , D52) Jonghae when values are calculated from the "3", will be distributed is made of a chunk (Chunk3) having a chunk number "3".

Thereafter, the distribution management unit 110 notifies the sort management unit 120 that distribution of all the specific dimension data D00-D55 included in the multidimensional data to the corresponding chunks is completed.

The alignment management unit 120 arranges specific dimension data distributed by chunks.

More specifically, the alignment management unit 120 arranges the specific dimension data distributed to each chunk based on the dimension of the specific dimension data distributed for each chunk.

That is, the alignment management unit 120 determines the sorting order of the specific dimension data divided by chunks based on the preset dimension management method, aligns the position information values of the corresponding dimension with the fastest sorting order, The position information value of the corresponding dimension of the "

Here, the predetermined dimension management method may be a method that each of the instance DB servers (0, 1, ..., n) of the multidimensional database 300 sets in advance for managing the dimension.

That is, the predetermined dimension management method is a method that is two-dimensional (Y-axis) than one dimension (X-axis) of one dimension (X-axis) and two dimensions (Y-axis) when multidimensional data is two- (Y-axis) position information values on the basis of the two-dimensional (Y-axis) that has the fastest sorting order, and at the same time, one-dimensional (X-axis (X-axis) positional information based on the position information of the one-dimensional (X-axis).

Hereinafter, for convenience of description, description will be made using a chunk of chunks (Chunk 0, 1, 2, 3) shown in FIG. 7 and an example of the previously set dimension management method.

7, since the specific dimension data D00, D01, D02, D10, D11, D12, D20, D21 and D22 are allocated to the chunk 0, ) Of the instance DB server 0 corresponding to the predetermined dimension management method.

Thereafter, the alignment management unit 120 determines whether or not the predetermined dimension management method is a method in which a two-dimensional (Y-axis) sorting order precedes one dimension (X axis) of two dimensions (X axis and Y axis) (Y axis), the position information values of the two dimensions (Y axis) are aligned with respect to the two-dimensional (Y axis) It is possible to confirm that the positional information value of the position information is aligned.

After the confirmation of the predetermined dimension management method is completed, the alignment management unit 120 stores position information of specific dimension data D00, D01, D02, D10, D11, D12, D20, D21, D22 in the chunk Check.

That is, the alignment management unit 120, corresponding to the position of the position values of V ₁ = "0" and two-dimensional (Y-axis) corresponding to the position in the first dimension (X-axis) of a particular dimension data (D00) The position information (0, 0) of the specific dimension data D00 can be confirmed by using the position value V ₂ = "0". The alignment management unit 120 stores the position information of the remaining specific dimension data D01, D02, D10, D11, D12, D20, D21 and D22 in the same manner as (0,1), 1,0), (1,1), (1,2), (2,0), (2,1), and (2,2).

Thereafter, the alignment management unit 120 sorts "0" having the smallest position information value of the two-dimensional (Y axis) on the basis of the two-dimensional (Y axis) (0,0), (1,0), and (2,0) obtained by aligning one-dimensional (X-axis) positional information values on the basis of the X-axis (X-axis) (0, 1), (1,1), (2,1), and (0,2), which are the results obtained by aligning the position information values "1" D2, D10, D11, D12, D20, D21, and D22 included in the chunk (Chunk0) are arranged in the chunk (Chunk0) Change the layout structure to create an alignment chunk (Chunk0 ').

When the alignment of the specific dimension data distributed to each chunk is completed based on the dimension of the specific dimension data distributed for the remaining chunks (Chunks 1, 2, 3) in the above-described manner, the alignment corresponding to the remaining chunks Chunks (Chunk 1 ', 2', 3 ') are generated.

According to the alignment method of the present invention, an external-merge sort method or the like can be used in order to align a large-capacity multidimensional data as a synchronization method, and an appropriate sorting algorithm may be used according to the number of multidimensional data.

Meanwhile, in the embodiment of the present invention, the multidimensional data of a large capacity is sorted based on the synchronization method. However, the present invention is not limited to this, and the data may be continuously merged and sorted while the distribution of the multidimensional data is progressed by the hash technique Of course, be based on asynchronous methods such as insertion sorting, bucket sorting, and so on.

The storage management unit 130 converts and stores specific dimension data arranged in a predetermined format.

More specifically, the storage management unit 130 converts the sorted specific dimension data into a predetermined specific format when the alignment of the specific dimension data distributed by the chunks is completed from the alignment management unit 120, DB server.

That is, the storage management unit 130 converts specific dimension data sorted based on the predetermined format, generates specific dimension conversion data, and stores the specific dimension conversion data in the instance DB server associated with the chunk.

Here, the preset specific format may be a storage format for managing multidimensional data in each instance DB server (0, 1 ... n) of the multidimensional database 300.

For example, the predetermined format has a Dense Array Format in which values are present in all the cells of the array storage (AS0,1 ... n) allocated to each instance DB server (0,1 ... n) A Sparse Array Format in which values are scattered only in a few cells of the array arrays AS0,1, ..., n, and is sparsely present.

Hereinafter, for convenience of description, it will be described that the chunks (Chunk 0, 1, 2, 3) of FIG. 7 and the predetermined specific format are Dense Array Format.

When the alignment management unit 130 completes the generation of the alignment chunks (Chunk0 ', 1', 2 ', 3') corresponding to the chunks (Chunk0,1,2,3) from the alignment management unit 120, The cell values corresponding to the cells of the array storage AS0,1,2,3 are allocated based on the Dense Array Format, which is a predetermined format, Dimensional transformation data.

The storage management unit 130 then checks the array storages AS0,1,2,3 corresponding to the chunks (Chunk0,1,2,3) and stores them in the corresponding instance DB servers (0,1,2,3) . The storage management unit 130 stores the specific dimension conversion data generated for each chunk (Chunk 0, 1, 2, 3) into an instance DB server 0, 1, 2 , 3).

When the storage of the specific dimension conversion data is completed according to the above description, the storage management unit 130 completes the execution of all the distributed management procedures. Therefore, in order to notify that the distributed management of the multidimensional data is completed, And transmits it to the user terminal 200.

On the other hand, when the chunks (Chunk0,1,2,3) and the instance DB server (0,1) do not match as shown in FIG. 8, at least two array storages are allocated to each instance DB server Loading can also be performed.

For example, when the array storage systems AS0 and AS2 are allocated to the instance DB server 0 and the array storage systems AS1 and AS3 are allocated to the instance DB server 1, Chunks (0, 2) correspond to the chunks (1, 2, 3), and the chunks (1,3) The converted data can be stored in the instance DB server (0, 1) associated with the chunk.

As described above, since the multidimensional data management apparatus 100 of the present invention maps and loads the multidimensional data directly to each instance DB server based on the hash value, unlike the existing multidimensional data storage method, Can be fundamentally blocked, reducing the overhead associated with stacking, thereby minimizing the overall loading cost and time.

Therefore, the multidimensional data management apparatus 100 of the present invention performs optimized distributed management of multidimensional data based on the hash technique, thereby minimizing the time and cost of the unnecessary consumption for data loading unlike the conventional method And to provide a new multidimensional data distribution management method capable of high speed data loading.

Hereinafter, a flow of providing a hash-based data distribution management service according to an embodiment of the present invention will be described in detail with reference to FIG. Hereinafter, for convenience of explanation, reference will be made to the reference numerals mentioned in the above-mentioned Figs. 1 to 8.

When the user selects the multidimensional data to be stored in the multidimensional database 300 (S100), the multidimensional data management apparatus 100 selects at least one of the multidimensional data Specific dimension data is distributed to the chunks (S110 - S140).

That is, the multidimensional data management apparatus 100 calculates the final hash value corresponding to the specific hash information based on the result of calculating the hash value for each dimension of the specific dimension data based on the specific hash function, The specific chunk data to be distributed based on the time value is identified and the specific dimension data is distributed.

Here, the specific hash function is used to find a hash value by dimension of specific dimension data, and can be defined as Equation (1).

[Equation 1]

는, 비트 연산의 XOR이다.Here, H _i is a hash value for each dimension, and the initial value H ₀ of all dimensions is initialized to "0". H _d is the final hash value finally calculated in the d dimension based on the hash value calculated for each dimension when the number of dimensions is d. Prime is a prime number for finding non-redundant hash values, and V _i is a position data value corresponding to a position in the i-th dimension of the multidimensional data. Min _i is the minimum position data value in the corresponding dimension, ChunkInterval is the chunk range of the ith dimension, and N is the number of the instance DB servers in the multidimensional database.

Is the XOR of the bit operation.

When the calculation of the hash value for each dimension of specific dimension data is completed using Equation (1), a modular operation (Equation 2) is used to calculate the final hash value.

&Quot; (2) "

Here, H _d is the final hash value finally calculated in the d dimension based on the hash value calculated for each dimension when the number of dimensions is d, and N is the number of the instance DB servers of the multidimensional database.

In this regard, FIG. 4 shows an example of multi-dimensional data to be distributed based on a hash technique. In FIG. 4, since only information on dimensions is utilized, attribute values for multidimensional data are not separately described.

4, when the multidimensional data selected by the user is two-dimensional, the X-axis is one-dimensional, the Y-axis is two-dimensional, and the chunk size (ChunkInterval) 3 ", the number N of the instance DB servers is 4, the prime number is" 991 ", and the minimum position data value per dimension is (0, 0), at least one A description will be given of a process of distributing specific dimension data to at least one chunk based on specific hash information per specific dimension data (D00-D55) of FIG.

The multidimensional data management apparatus 100 first performs a process for finding a hash value for each dimension of the specific dimension data D00-D55 included in the multidimensional data.

That is, as shown in FIG. 5, the multidimensional data management apparatus 100 has a first parameter for calculating a hash value (H ₁ ) in one dimension (X axis) of the specific dimension data D00, that is, the initial value of the dimension H ₀ = "0", the one-dimensional position value corresponding to a position in the (X-axis) V ₁ = "0", the Min ₁ minimum position data value in the first dimension (X-axis) = Quot; 0 ", and ChunkInterval = "3 ", which is the size of the chunk in one dimension (X axis).

Thereafter, the multidimensional data management apparatus 100 calculates the hash value (H ₁ ) in one dimension (X axis) of the specific dimension data D00 by applying the first parameter to Equation ( ₁ ) as follows.

When the calculation of the hash value H ₁ is completed as described above, the multidimensional data management apparatus 100 calculates the second parameter for calculating the hash value (H ₂ ) in two dimensions (Y axis) of the specific dimension data D00 H ₁ = "0", a prime number = "991", a position value corresponding to a position in the two-dimensional (Y-axis), V ₂ = "0 Min ₂ = "0", which is the minimum position data value in two dimensions (Y axis), and ChunkInterval = "3", which is the size of the chunk in two dimensions (Y axis).

Thereafter, the multidimensional data management apparatus 100 calculates the hash value (H ₂ ) in two dimensions (Y axis) of the specific dimension data D00 by applying the second parameter to Equation (1) as follows.

After all, the second parameter, geumbeon hash value than the first, and the hash value calculated one-dimensional hash value of (X-axis) (H _1), small number of two-dimensional (Y-axis) to calculate the (H ₂₎ (Prime Number), At least one of a position value V ₂ corresponding to a position in two dimensions (Y axis), a chunk range (ChunkInterval) in two dimensions (Y axis), and a minimum position data value Min ₂ in two dimensions (Y axis) One is included.

)과, 2차원(Y축)에서의 위치에 대응하는 위치값(V₂)과 2차원(Y축)에서의 최소 위치 데이터값(Min₂)을 청크범위(ChunkInterval)로 나눗셈한 제2 결과값(

)에 대한 배타적논리합(XOR)에 대응하게 된다. Thus, when the second parameter is applied to the specific solution function H _i , the specific solution function H ₂ is set to have the hash value H ₁ in one dimension (X axis) of the specific dimension data D00, And a first result (Prime Number)

A second result obtained by dividing the position value V ₂ corresponding to the position in the two-dimensional (Y axis) and the minimum position data value Min ₂ in the two-dimensional (Y axis) by the chunk range (ChunkInterval) value(

(XOR) to the exclusive logical sum (XOR).

That is, if the second hash value (H ₂₎ of the two-dimensional (Y-axis) to calculate a geumbeon hash value of the two-dimensional (X-axis, Y-axis) included in the specific-dimensional data (D00), first, a hash value (H ₁ (Y-axis) hash value H ₂ is applied to the parameter of Equation (2), which is a modular operation to be described later, so that the specific dimension data D00), all the dimensions of the specific dimension data D00, that is, the (X-axis, Y-axis) are reflected and the chunk can be determined.

When the calculation of the hash values H ₁ and H _{2 for each} dimension of the specific dimension data D00 is completed, the multidimensional data management apparatus 100 calculates the final hash value of the specific dimension data D00, (2) "

That is, the multidimensional data management apparatus 100 has the third parameter for calculating the final hash value of the specific dimension data D00, that is, the hash value H (Y) of the specific dimension data D00 in the two-dimensional ₂ ) and N = "4 ", which is the total number of the instance DB servers managing the multidimensional data.

Thereafter, the multidimensional data management apparatus 100 calculates the final solution value of the specific dimension data D00 by applying the third parameter to Equation (2) as follows.

When the calculation of the final solution value is completed, the multi-dimensional data management device 100 distributes the specific dimension data D00 related to the final solution value to the corresponding chunk having the chunk number matched with the final solution value do.

The process of calculating the final solution value based on the dimension-specific hash value of the specific dimension data D00 and distributing the specific dimension data D00 to the corresponding chunk after detecting the corresponding chunk is shown in FIGS. 5 and 6 The final hash value of the specific dimension data D00, D01, D02, D10, D11, D12, D20, D21, and D22 is calculated as shown in FIG. 7, (D03, D04, D05, D13, D14, D15, D23, D24, and D25) will be distributed to the chunk having the chunk number "0 " D34, D35, D43, D44, D45, D53, D54, and D55 will be distributed to the chunk having the chunk number "1 " D30, D31, D32, D40, D41, D42, D50, D51 (D30, D31, D32, D40, D51) will be distributed to chunks , D52) Jonghae when values are calculated from the "3", will be distributed is made of a chunk (Chunk3) having a chunk number "3".

Thereafter, the multidimensional data management apparatus 100 determines the sorting order of the specific dimension data distributed by chunks based on the preset dimension management method, arranges the position information value of the corresponding dimension with the fastest sorting order The position information values of the corresponding dimensions of the next order are sorted (S150).

More specifically, the multidimensional data management apparatus 100 determines whether the predetermined dimension management method is a two-dimensional (Y-axis) sorting order than one dimension (X-axis) of two dimensions (Y-axis) is performed based on the two-dimensional (Y-axis), which is the fastest sorting order, It is possible to confirm that the position information value of the dimension (X axis) should be aligned.

When the confirmation of the preset dimension management method is completed, the multidimensional data management apparatus 100 reads V ₁ = "0 ", which is a position value corresponding to the position in one dimension (X axis) of the specific dimension data D00 The position information (0, 0) of the specific dimension data D00 is confirmed by using V ₂ = "0" which is a position value corresponding to the position in the two-dimensional (Y axis). Next, the multidimensional data management apparatus 100 reads position information of the remaining specific dimension data D01, D02, D10, D11, D12, D20, D21, and D22 in the same manner as (0,1) , (1,0), (1,1), (1,2), (2,0), (2,1), and (2,2).

Thereafter, the multidimensional data management apparatus 100 sequentially arranges "0" having the smallest position information value of the two-dimensional (Y axis) with respect to the two-dimensional (Y axis) (0, 0), (1, 0), and (2, 0), which are the results of aligning position information values of one dimension (X axis) with respect to one dimension (X axis) Position information values "1" and "2" of the next size in two dimensions (Y axis) are also obtained by arranging in the same manner (0,1), (1,1), (2,1) D2, D10, D11, D12, D20, D21, and D22 included in a chunk (Chunk0) are arranged in a chunk (Chunk0) ) To generate an alignment chunk (Chunk0 ').

When the sorting of the specific dimension data distributed to each chunk is completed based on the dimension of the specific dimension data distributed for the remaining chunks (Chunks 1, 2, 3) in the above-described manner, the multidimensional data management apparatus 100, And generates alignment chunks (Chunk 1 ', 2', 3 ') corresponding to the chunks (Chunks 1, 2, 3).

Thereafter, the multidimensional data management apparatus 100 converts the specific dimension data sorted based on the predetermined format, generates specific dimension conversion data, and stores the specific dimension conversion data in the instance DB server associated with the chunk (S160, S170).

Here, the preset specific format may be a storage format for managing multidimensional data in each instance DB server (0, 1 ... n) of the multidimensional database 300.

For example, the predetermined format has a Dense Array Format in which values are present in all the cells of the array storage (AS0,1 ... n) allocated to each instance DB server (0,1 ... n) A Sparse Array Format in which values are scattered only in a few cells of the array arrays AS0,1, ..., n, and is sparsely present.

Hereinafter, for convenience of description, it will be described that the chunks (Chunk 0, 1, 2, 3) of FIG. 7 and the predetermined specific format are Dense Array Format.

When the generation of the alignment chunks (Chunk0 ', 1', 2 ', 3') corresponding to the chunks (Chunk0,1,2,3) is completed, the multidimensional data management apparatus 100 updates the alignment chunks ', 2', 3 ') based on a Dense Array Format, which is a predetermined format, and assigns cell values corresponding to the cells of the array storage (AS0,1,2,3) .

Thereafter, the multidimensional data management apparatus 100 confirms the array storages AS0, 1, 2, and 3 corresponding to the chunks (Chun0, 1, 2, 3) 3). Then, the multidimensional data management apparatus 100 transmits the specific dimension conversion data generated for each chunk (Chunk 0, 1, 2, 3) to the instance DB server (0, 1 , 2, 3).

When the storage of the specific dimension conversion data is completed according to the above description, the multidimensional data management apparatus 100 has completed the execution of all the distributed management procedures. Therefore, in order to notify that the multidimensional data has been distributed, And transmits it to the user terminal 200.

As described above, according to the present invention, multidimensional data is directly mapped to each instance DB server on the basis of a hash value, so that it is possible to fundamentally block the relocation process of multidimensional data unlike the existing multidimensional data storage method. Thereby reducing overall overhead and time, thereby reducing overall overhead.

Therefore, the apparatus and method for managing multidimensional data of the present invention can perform optimized distributed management of multidimensional data based on a hash technique, thereby reducing the time and cost of unnecessary consumption for data loading The present invention can provide a new multidimensional data distribution management method that enables high-speed data loading.

Embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

According to the apparatus and method for managing multidimensional data of the present invention, optimized distributed management of multidimensional data is performed based on a hash technique, thereby reducing the time and cost of unnecessary consumption for data loading Dimensional data distribution management method that enables high-speed data loading by minimizing the size of data to be stored in the storage device. Therefore, as a result of overcoming the limitations of the existing technology, Is not only sufficient but also practically usable because it can be practically carried out clearly.

100: Multidimensional data management device
110: distribution management unit 120: alignment management unit
130: storage management unit 140: storage unit
200: user terminal
300: Multidimensional Database

Claims (16)

A distribution manager for distributing the specific dimension data to at least one chunk based on at least one specific hash information for each specific dimension data included in the multidimensional data; And
And an alignment management unit for aligning the specific dimension data distributed to each chunk based on the dimension of the specific dimension data distributed for each of the chunks,
Wherein the distribution management unit comprises:
Wherein the final hash value corresponding to the specific hash value is calculated based on a result of calculating a hash value for each dimension of the specific dimension data based on a specific hash function. The method according to claim 1,
Further comprising a storage management unit for converting the sorted specific dimension data into a predetermined format and storing the converted specific dimension data in an instance DB server associated with the chunk. The method according to claim 1,
Wherein the specific dimension data includes a position value corresponding to a position in the dimension for each dimension,
The specific hash function,
A hash value of the second dimension, a prime number, a position value of the first dimension, and a chunk range of the first dimension, which are calculated before the first dimension for which the hash value is to be calculated, Dimensional data management apparatus. The method of claim 3,
The specific hash function,
A first result obtained by multiplying the hash value of the second dimension by the prime number and a second result value obtained by dividing the difference value between the position value of the first dimension and the minimum position value of the first dimension by a chunk range of the first dimension, (XOR) with respect to the result value. The method of claim 3,
Wherein the distribution management unit comprises:
When the hash value of the first dimension is calculated using the hash value of the second dimension based on the specific hash function, the hash value of the first dimension and the total number of the instance DB servers managing the multidimensional data are Wherein the final hash value is calculated by performing a modular operation on the basis of the first hash value. 6. The method of claim 5,
The size of the final hash value is matched with a chunk number that identifies the chunk,
Wherein the distribution management unit comprises:
And distributes specific dimension data related to the final hash value to a corresponding chunk having the chunk number matched with the final hash value. The method of claim 3,
The alignment management unit,
Determining a sorting order of the specific dimension data distributed by the chunks based on the predetermined dimension management method, sorting the position information value of the corresponding dimension with the highest sorting order, Dimensional data. 3. The method of claim 2,
The storage management unit,
Transforming the specified specific dimension data based on a predetermined specific format to generate specific dimension conversion data, and storing the specific dimension conversion data in an instance DB server related to the chunk. A distribution management step of distributing the specific dimension data to at least one chunk on the basis of at least one specific identification information for each specific dimension data included in the multidimensional data; And
And a sort management step of sorting the specific dimension data distributed to each chunk based on the dimension of the specific dimension data distributed for each of the chunks,
Wherein the distribution management step comprises:
Calculating a final hash value corresponding to the specific hash value based on a result of calculating a hash value for each dimension of the specific dimension data based on a specific hash function. 10. The method of claim 9,
Further comprising a storage management step of converting the sorted specific dimension data into a predetermined format and storing the converted specific dimension data in an instance DB server associated with the chunk. 10. The method of claim 9,
Wherein the specific dimension data includes a position value corresponding to a position in the dimension for each dimension,
The specific hash function,
A hash value of the second dimension, a prime number, a position value of the first dimension, and a chunk range of the first dimension, which are calculated before the first dimension for which the hash value is to be calculated, Wherein the data management apparatus comprises: 12. The method of claim 11,
The specific hash function,
A first result obtained by multiplying the hash value of the second dimension by the prime number and a second result value obtained by dividing the difference value between the position value of the first dimension and the minimum position value of the first dimension by a chunk range of the first dimension, (XOR) with respect to the result of the comparison. 12. The method of claim 11,
Wherein the distribution management step comprises:
When the hash value of the first dimension is calculated using the hash value of the second dimension based on the specific hash function, the hash value of the first dimension and the total number of the instance DB servers managing the multidimensional data are Wherein the final hash value is calculated by performing a modular operation based on the first hash value. 14. The method of claim 13,
The size of the final hash value is matched with a chunk number that identifies the chunk,
Wherein the distribution management step comprises:
Wherein the specific dimension data related to the final hash value is distributed to a corresponding chunk having the chunk number matched with the final hash value. 12. The method of claim 11,
Wherein the alignment management step comprises:
Determining a sorting order of the specific dimension data distributed by the chunks based on the predetermined dimension management method, sorting the position information value of the corresponding dimension with the highest sorting order, Dimensional data management apparatus according to the present invention. 11. The method of claim 10,
Wherein the storage management step comprises:
Wherein the specific dimension conversion data is converted by converting the specific dimension data based on a predetermined format and stores the specific dimension conversion data in an instance DB server associated with the chunk. How it works.

Images