CN107679096B - Method and device for sharing indexes among data marts - Google Patents

Abstract

The invention discloses a method and a device for sharing indicators among data marts, and relates to the field of computers. A specific implementation of the method includes: determining at least one shared indicator data table in indicator data tables of a plurality of data marts, and creating a corresponding shared indicator data table according to any one of the at least one shared indicator data table A test case; wherein the shared indicator data table includes indicator data to be shared among the multiple data marts; copy the shared indicator data table and the test case to the shared mart, and use the test The use case tests the corresponding shared indicator data table in the shared mart; and copies any shared indicator data table that passes the test to each data mart that does not store the shared indicator data table. In this embodiment, indicator data can be shared among various data marts, so that the definition of each indicator and the mathematical model adopted by each indicator can be agreed among various data marts.

Description

Method and device for sharing indicators among data marts

technical field

The invention relates to the field of computers, and in particular, to a method and device for sharing indicators between data marts.

Background technique

With the development of enterprise business, a large amount of business data will be generated. Usually, enterprises will build a data warehouse to collect data and store the data according to the subject classification. For each department of the enterprise, the data concerned in the data warehouse are often processed logically and then pushed to the data mart of the department.

In practical applications, the success of an enterprise's data mart construction depends to a large extent on whether each data mart has a stable, consistent and comprehensive indicator model. Exact definitions of indicators and related data models used for each indicator. Establishing a stable, consistent and comprehensive indicator model between data marts can keep the indicator data of each data mart consistent, and then provide support for subsequent application analysis and data products.

In the prior art, after a data warehouse extracts original data from a business database, a corresponding data table is established, and the data table is synchronized to a relevant data mart for use by analysts. In any data mart, analysts define their own metric models and use said metric models within that data mart.

In the process of realizing the present invention, the inventor found that the prior art has at least the following problems:

1. The definitions of the same indicator in each data mart are different, which makes it easy to cause misunderstanding when the corresponding indicator data is used between data marts. For example: the order volume index of the marketing market refers to the total number of orders minus the number of invalid orders, and the order volume index of the operation market refers to the number of orders that generate waybills;

2. The data models used for the same indicator in each data mart are different, so that the corresponding indicator data cannot be used in the data marts. For example, the order amount indicator in the marketing market uses the preferential data table LA, and the order amount indicator in the sales market uses the preferential data table LB.

SUMMARY OF THE INVENTION

In view of this, embodiments of the present invention provide a method and device for sharing indicators between data marts, which can share indicator data among data marts, so that the definition of each indicator and the mathematical model used by each indicator can be used in each data mart. Agreed among markets.

To achieve the above object, according to an aspect of the present invention, a method for sharing indicators between data marts is provided.

The method for sharing indicators among data marts according to the embodiment of the present invention includes: determining at least one shared indicator data table in indicator data tables of a plurality of data marts, and creating a shared indicator data table according to any one of the at least one shared indicator data table. A test case corresponding to the shared indicator data table; wherein, the shared indicator data table includes indicator data to be shared among the multiple data marts; the shared indicator data table and the test case are copied to the shared set test the corresponding shared indicator data table in the shared mart by using the test case; and copy any shared indicator data table that passes the test to each data mart that does not store the shared indicator data table.

Optionally, the test case includes: a test step and an expected result corresponding to the test step.

Optionally, the using the test case to test the corresponding shared indicator data table in the shared mart includes: for any shared indicator data table in the shared mart, executing the test steps in the corresponding test case; when When the test result is completely consistent with the expected result in the test case, the shared indicator data table is determined to pass the test.

Optionally, the method further includes: when the test result is inconsistent with the expected result in the test case: removing the shared indicator data table, and putting it in the indicator data table of the multiple data marts The determined shared indicator data table corresponding to the shared indicator data table is copied to the shared market again, and the shared indicator data table is tested by using the test case.

Optionally, the multiple data marts are subordinate data marts based on the same data warehouse, and the multiple data marts, the shared mart, and the data warehouse are all stored in a distributed file system architecture. data.

To achieve the above object, according to another aspect of the present invention, an apparatus for sharing indicators between data marts is provided.

The apparatus for sharing indicators between data marts according to the embodiment of the present invention includes: an indicator determining unit, which can be configured to determine at least one shared indicator data table among indicator data tables of a plurality of data marts, and according to the at least one shared indicator data table Create a test case corresponding to the shared indicator data table; wherein, the shared indicator data table includes indicator data to be shared among the plurality of data marts; the indicator testing unit can be used to share the shared indicator data. The indicator data table and the test case are copied to the shared mart, and the test case is used to test the corresponding shared indicator data table in the shared mart; the indicator sharing unit can be used to copy any shared indicator data table that has passed the test. to every data mart that does not store this shared metric data table.

Optionally, the test case includes: a test step and an expected result corresponding to the test step; and the indicator test unit can be used to: execute a corresponding test for any shared indicator data table in the shared marketplace Test steps in the use case; when the test result is completely consistent with the expected result in the test case, the shared indicator data table is determined to pass the test.

Optionally, the multiple data marts are subordinate data marts based on the same data warehouse, and the multiple data marts, the shared mart, and the data warehouse are all stored in a distributed file system architecture. data.

To achieve the above object, according to yet another aspect of the present invention, an electronic device is provided.

An electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more programs make the One or more processors implement the method for sharing indicators among data marts provided by the present invention.

To achieve the above object, according to yet another aspect of the present invention, a computer-readable storage medium is provided.

A computer-readable storage medium of the present invention stores a computer program thereon, and when the program is executed by a processor, the method for sharing indicators between data marts provided by the present invention is implemented.

According to the technical solution of the present invention, an embodiment of the above invention has the following advantages or beneficial effects: by determining the shared indicator data table from each data mart, the exact definition of each indicator and the adopted index can be determined according to the shared indicator data table. Data model; by copying the shared indicator data table to the shared mart, and sharing it with each data mart after testing it in the shared mart, the technical effect of sharing indicator data in each data mart is realized, and the existing technology is solved. The problem of inconsistency in the definition of indicators and the data models adopted by the data marts in the data mart enables each data mart to provide support for business based on a stable, consistent and comprehensive indicator model; by testing the shared indicator data table in the shared mart, It avoids the impact of data loss and data damage caused by node failure, network failure, etc., and ensures the quality of the indicator data shared in each data mart.

Further effects of the above non-conventional alternatives will be described below in conjunction with specific embodiments.

Description of drawings

The accompanying drawings are used for better understanding of the present invention and do not constitute an improper limitation of the present invention. in:

1 is a schematic diagram of main steps of a method for sharing indicators between data marts according to an embodiment of the present invention;

2 is a schematic structural diagram of a data mart and a data warehouse of a method for sharing indicators between data marts according to an embodiment of the present invention;

3 is a schematic diagram of a main part of an apparatus for sharing indicators between data marts according to an embodiment of the present invention;

4 is a schematic diagram of the composition of an apparatus for sharing indicators between data marts according to an embodiment of the present invention;

5 is an exemplary system architecture diagram to which an embodiment of the present invention may be applied;

FIG. 6 is a schematic structural diagram of an electronic device for implementing the method for sharing indicators between data marts according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

The technical solution of the embodiment of the present invention determines the shared indicator data table from each data mart, so that the exact definition of each indicator and the adopted data model can be determined according to the shared indicator data table; by copying the shared indicator data table to the shared mart , and share it with each data mart after testing it in the sharing market, realizing the technical effect of sharing indicator data in each data mart, and solving the inconsistency in the definition of indicators and the data model adopted by indicators in each data mart in the prior art problems, so that each data mart can provide support for business based on a stable, consistent and comprehensive indicator model; by testing the shared indicator data table in the shared mart, data loss due to node failure, network failure, etc. is avoided. The impact of data corruption ensures the quality of indicator data shared across data marts.

FIG. 1 is a schematic diagram of main steps of a method for sharing indicators among data marts according to this embodiment.

As shown in FIG. 1 , the method for sharing indicators between data marts according to an embodiment of the present invention is performed according to the following steps:

Step S101 : Determine at least one shared indicator data table in indicator data tables of a plurality of data marts, and create a test case corresponding to the shared indicator data table according to any shared indicator data table.

In the embodiment of the present invention, the data mart is a subordinate data mart based on the same data warehouse. Among them, in the field of computer technology, data warehouse refers to a subject-oriented, integrated, relatively stable data collection that reflects historical changes and is used to support management decision-making. Enterprises usually build data warehouses within the enterprise to obtain data from business databases. data, and perform online analytical processing (OLAP, Online Analytical Processing) on the data to obtain information that supports enterprise decision-making. Data marts refer to data collections that meet specific departments or specific needs of enterprises and are oriented to departmental decision-making analysis. Enterprises usually establish department-wide data marts to store relevant data of the department, and analyze the data to support the department. decision-making information. For example, the data mart of an enterprise may include marketing mart, sales mart, operation mart, etc.

Generally, data marts can be divided into two types: independent data marts that obtain data directly from business databases, and dependent data marts that obtain data from enterprise-level data warehouses. In the embodiment of the present invention, the data mart is a subordinate data mart with a more stable architecture and stronger scalability.

FIG. 2 is a schematic structural diagram of a data mart and a data warehouse according to an embodiment of the present invention.

As shown in FIG. 2 , the business data stored in the business database 210 is extracted into the data warehouse 220 after data cleaning and data compression. Among them, data cleaning is used to filter interference data in business data. Data compression can compress data into LZO (LZO is a lossless data compression algorithm), ORC (Optimized Row Columnar file, optimized row columnar file) and other formats for storage.

The data warehouse 220 establishes a data table corresponding to the business data, and partitions the data according to the storage time. In practical applications, data can be partitioned by day. Thereafter, the data warehouse 220 may replicate the data tables to the various data marts 250 . It should be noted that "copying" in the present invention refers to copying the source data table and sending the copied data table to the target location. Preferably, the data warehouse 220 can copy the data table to the data mart 250 by means of a message queue or the like.

In this embodiment of the present invention, a shared marketplace 260 is preset among each data mart 250 for communicating with any data mart 250 . In a specific application, the shared mart 260 may be an independent special data mart, or may be composed of storage spaces divided by multiple existing data marts 250 . Preferably, the data warehouse 220, the data mart 250, and the shared mart 260 in the embodiment of the present invention all adopt a distributed file system architecture such as HDFS (Hadoop Distributed File System, Hadoop distributed file system, Hadoop is a distributed system foundation. Schema) to store data, the data warehouse 220 can be built based on Hive (Hive is a Hadoop-based data warehouse tool).

In step S101, the index data table refers to a data table containing index data. The data table is a data structure in the "table" format in the data mart, and the index data contained in the data table is stored in the corresponding table directory of HDFS. Indicator data refers to the data related to indicators such as order volume and order amount in the indicator data table. For example: the total order volume data, invalid order volume data, and order generation time data in the order details table can be used as the indicator data of the order volume indicator, and the original order amount data, discount amount data, and discount table data can be used as the order amount indicator. indicator data.

It should be noted that the indicator data table established by the data mart can reflect the data mart's definition of the corresponding indicator and the relevant data model of the indicator, that is, the indicator data table can represent the data mart's indicator model for the corresponding indicator. For example: the order details table JA of a data mart A contains data such as total order volume data, invalid order volume data, order generation time data, order original amount data, discount amount data, discount table data and other data, from the above indicators in the table From the data, you can know the exact definition of the order volume indicator and the order amount indicator in the data mart A and the related data model of the indicators. Among them, the relevant data model of the indicator refers to the data model used when using the indicator, for example: for the order amount indicator, the discount table data in the indicator data table shows that the discount table k is used when generating the order amount, then k is the order amount indicator related data model.

In practical applications, each data mart will establish its own indicator data table according to the application environment of its own department. The indicator data table is generally not inconvenient when used within the data mart, but when used between different data marts. Misunderstandings are often created, making it difficult to implement analytics applications based on multiple data marts.

In view of the above problems, in the embodiment of the present invention, a shared indicator data table is determined among multiple indicator data tables related to the same indicator in each data mart, and the shared indicator data table includes indicators to be shared among various data marts The data, that is, the indicator data contained in the shared indicator data table, are shared in each data mart after the shared indicator data table is determined and the shared indicator data table passes the test, so as to determine a common indicator model for work between data marts. That is to say, select one or more shared indicator data tables from multiple indicator data tables in each data mart—for the same indicator, only a unique shared indicator data table can be selected to correspond to it—the shared indicator data table is passed through The index data contained in it determines the definition and data model of the corresponding index, and the shared index data table and the definition of the identified index and the data model of the index will be shared and agreed in each data mart, and then work among multiple data marts be used consistently.

For example, data marts A, B, and C store order detail tables JA, JB, and JC, respectively. JA, JB, and JC determine their own definitions and data models of order volume indicators and order amount indicators. At this time, according to specific needs, the order details table JA of data mart A can be used as a shared indicator data table to be shared with each data mart including data marts B and C. When each data mart communicates with other data marts, or when an enterprise works based on multiple data marts, the relevant data of each data mart needs to conform to the indicator definitions and indicator data models determined by the shared indicator data table.

In this step, after the shared indicator data table is determined, a test case can be created according to any shared indicator data table. Test cases refer to test data and test rules created to accomplish specific goals. Generally, test cases can be flexibly set according to the application environment, and can include test steps and expected results corresponding to the test steps. Specifically, creating a test case based on the shared indicator data table can be achieved through the following steps:

1. Determine multiple test steps based on the shared indicator data table;

2. Execute the test steps in sequence on the shared indicator data table to obtain multiple test results;

3. Take multiple test results as the expected results corresponding to the test steps, respectively.

For example: create the following test case based on the order details table JA as a shared indicator data table:

Test steps:

1. Query the order quantity of the product with the product number of 5512;

2. Check the order quantity of products whose order address is Beijing;

3. Check the order amount of clothing products whose order address is in Northeast China.

The above test steps are executed in sequence in JA, and the following test results are obtained, which are regarded as the expected results:

1.1000;

2.7566986;

3.659988.

It can be understood that, test cases in practical applications usually consist of a large number of test steps and expected results, and the above examples are only examples, and do not limit the test cases in the present invention.

Step S102: Copy the shared indicator data table and the test case to the shared mart, and use the test case to test the corresponding shared indicator data table in the shared mart.

In this step, the shared indicator data table determined in each data mart and the test case created according to the shared indicator data table are copied from the data mart to the pre-launch area of the shared mart. Among them, the pre-launch area of the sharing market is the level where the sharing market is in the pre-release environment. In addition, in practical applications, the above replication process may be implemented by means of message queues and the like. After that, use test cases to test the corresponding shared indicator data tables in the pre-launch area of the shared market to prevent errors or loss of data due to data storage node failures, network failures, and other reasons during the replication process. The corresponding shared indicator data table refers to a shared indicator data table corresponding to the test case, that is, the test case is created according to the shared indicator data table.

The test process is as follows:

1. For any shared indicator data table in the shared marketplace, execute the test steps in the corresponding test case. Among them, the corresponding test case refers to the test case created according to the source data table of the shared indicator data table in the data mart;

2. Determine whether the test results are completely consistent with the expected results in the test case: if so, determine the shared indicator data table as passing the test; otherwise, remove the shared indicator data table and determine it in the indicator data table of the data mart , the shared indicator data table corresponding to the shared indicator data table is copied to the shared market again, that is, the source data table of the shared indicator data table is copied to the shared market again;

3. For the shared indicator data table copied again, execute the test steps in the test case again to determine whether the test results are completely consistent with the expected results in the test case: if so, determine the shared indicator data table as passing the test; otherwise, Remove the shared indicator data table copied again, and the test is over.

In practical applications, after the above-mentioned testing process, a shared indicator data table that has passed the test can generally be obtained in the shared market.

Step S103: Copy any shared indicator data table that passes the test to each data mart that does not store the shared indicator data table.

In this step, the shared indicator data table that has passed the test is copied to the online area of each data mart that does not store the shared indicator data table for analysts to use. Among them, the online area of the data mart refers to the level in the formal environment of the data mart. In practical applications, the data mart can use its own indicator definition and indicator data model when carrying out internal work; when multiple data marts interact or work based on multiple data marts, a shared indicator data table needs to be used. The established indicator definitions and indicator data models make the indicators in each data mart consistent and avoid ambiguity.

In specific applications, the shared indicator data tables and test cases determined in the data mart can be copied to the shared mart on a regular basis—for example, every day, and the shared indicator data tables that have passed the test can be shared with each data mart to achieve sharing. Regular updates of the indicator model.

According to the method of the embodiment of the present invention, it can be seen that by determining the shared indicator data table from each data mart, the exact definition of each indicator and the adopted data model can be determined according to the shared indicator data table; The technical means of copying the table to the shared mart, and sharing it with each data mart after testing it in the shared mart, thus realizing the technical effect of sharing indicator data in each data mart, and solving the problem of each data set in the prior art. The problem of inconsistency in the definition of market indicators and the data model adopted by the indicators enables each data mart to provide support for business based on a stable, consistent and comprehensive indicator model; because the technical means of testing the shared indicator data table in the shared mart is adopted , so as to avoid the impact of data loss and data damage caused by node failure, network failure, etc., and ensure the quality of the indicator data shared in each data mart.

FIG. 3 is a schematic diagram of a main part of an apparatus for sharing indicators between data marts according to an embodiment of the present invention.

As shown in FIG. 3 , the apparatus 300 for sharing indicators among data marts according to the embodiment of the present invention may include: an indicator determining unit 301 , an indicator testing unit 302 , and an indicator sharing unit 303 . in:

The indicator determining unit 301 can be configured to determine at least one shared indicator data table in indicator data tables of multiple data marts, and create a test case corresponding to the shared indicator data table according to any shared indicator data table; wherein, the shared indicator The data table includes indicator data to be shared among the plurality of data marts.

The indicator testing unit 302 may be configured to copy the shared indicator data table and the test case to the shared mart, and use the test case to test the corresponding shared indicator data table in the shared mart.

The indicator sharing unit 303 can be configured to copy any shared indicator data table that passes the test to each data mart that does not store the shared indicator data table.

As a preferred solution, the test case may include: test steps and expected results corresponding to the test steps. The indicator testing unit 302 can be used to: for any shared indicator data table in the shared market, execute the test steps in the corresponding test case; when the test result is completely consistent with the expected result in the test case, The shared indicator data table is determined to pass the test.

In addition, in the embodiment of the present invention, the plurality of data marts are subordinate data marts based on the same data warehouse, and the plurality of data marts, the shared mart, and the data warehouse all use distribution file system architecture to store data.

FIG. 4 is a schematic diagram of components of an apparatus for sharing indicators among data marts according to an embodiment of the present invention.

As shown in FIG. 4 , the apparatus for sharing indicators among data marts according to the embodiment of the present invention includes a data preparation unit 401 and a data import unit 402 in addition to the indicator determination unit 301 , the indicator testing unit 302 , and the indicator sharing unit 303 . The data preparation unit 401 includes a collection module and a storage module, and the data import unit 402 includes an extraction module and a scheduling module. specifically:

The collection module can be used to collect the business data of the business database, and perform data cleaning on the collected data to filter the interference data;

The storage module can be used to compress the cleaned data in LZO, ORC and other formats;

The extraction module can be used to extract the compressed data to the data warehouse through the scheduling module;

After the data warehouse establishes the data table corresponding to the business data, the data table is copied to each data mart, and then the indicator determining unit 301 can determine the shared indicator data table from the data mart.

According to the technical solution of the embodiment of the present invention, by determining the shared indicator data table from each data mart, the exact definition of each indicator and the adopted data model can be determined according to the shared indicator data table; by copying the shared indicator data table to the shared indicator data table Market, and share it with each data mart after testing it in the sharing mart, realizing the technical effect of sharing indicator data in each data mart, and solving the definition of indicators in each data mart and the data used in indicators in the prior art The problem of model inconsistency enables each data mart to provide support for the business based on a stable, consistent and comprehensive indicator model; by testing the shared indicator data table in the shared mart, data caused by node failures, network failures, etc. are avoided. The impact of loss and data corruption ensures the quality of indicator data shared in various data marts.

FIG. 5 shows an exemplary system architecture 500 of a method for sharing indicators among data marts or an apparatus for sharing indicators between data marts to which embodiments of the present invention can be applied.

As shown in FIG. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505 (this architecture is only an example, and the components included in the specific architecture can be adjusted according to the specific application). The network 504 is a medium used to provide a communication link between the terminal devices 501 , 502 , 503 and the server 505 . Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the terminal devices 501, 502, 503 to interact with the server 505 through the network 504 to receive or send messages and the like. Various communication client applications may be installed on the terminal devices 501 , 502 and 503 , such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

The terminal devices 501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The server 505 may be a server that provides various services, for example, a background management server that provides support for shopping websites browsed by the terminal devices 501 , 502 , and 503 (just an example). The background management server can analyze and process the received product information query request and other data, and feed back the processing results (such as target push information, product information—just an example) to the terminal device.

It should be noted that the method for sharing indicators among data marts provided by the embodiments of the present invention is generally performed by the server 505 .

It should be understood that the numbers of terminal devices, networks and servers in FIG. 5 are only illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.

The present invention also provides an electronic device.

An electronic device according to an embodiment of the present invention includes: one or more processors; and a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more programs cause the One or more processors implement the method for sharing indicators among data marts provided by the present invention.

Referring next to FIG. 6 , it shows a schematic structural diagram of a computer system 600 suitable for implementing an electronic device according to an embodiment of the present invention. The electronic device shown in FIG. 6 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention.

As shown in FIG. 6, a computer system 600 includes a central processing unit (CPU) 601, which can be loaded into a random access memory (RAM) 603 according to a program stored in a read only memory (ROM) 602 or a program from a storage section 608 Instead, various appropriate actions and processes are performed. In the RAM 603, various programs and data necessary for the operation of the computer system 600 are also stored. The CPU 601 , the ROM 602 , and the RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to bus 604 .

The following components are connected to the I/O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc. ; and a communication section 609 including a network interface card such as a LAN card, a modem, and the like. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I/O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

In particular, according to the disclosed embodiments of the present invention, the processes described in the main step diagrams above may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the method shown in the main step diagram. In the above-described embodiment, the computer program can be downloaded and installed from the network through the communication section 609 and/or installed from the removable medium 611 . When the computer program is executed by the central processing unit 601, the above-described functions defined in the system of the present invention are performed.

It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer readable program code therein. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium, other than a computer-readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.

The units involved in the embodiments of the present invention may be implemented in a software manner, and may also be implemented in a hardware manner. The described unit can also be set in the processor, for example, it can be described as: a processor includes an indicator determination unit, an indicator testing unit and an indicator sharing unit. Among them, the names of these units do not constitute a limitation of the unit itself under certain circumstances. For example, the indicator testing unit can also be described as "a unit that sends a shared indicator data table that has passed the test to the indicator sharing unit".

As another aspect, the present invention also provides a computer-readable medium. The computer-readable medium may be included in the device described in the above embodiments; it may also exist independently without being assembled into the device. . The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the device, the steps of causing the device to execute include: determining at least one shared indicator in indicator data tables of multiple data marts A data table, creating a test case according to any shared indicator data table; wherein the shared indicator data table includes indicator data to be shared among the multiple data marts; combining the shared indicator data table and the test Copy the use case to the shared mart, and use the test case to test the corresponding shared indicator data table in the shared mart; and copy any shared indicator data table that passes the test to each data that does not store the shared indicator data table market.

According to the technical solution of the embodiment of the present invention, by determining the shared indicator data table from each data mart, the exact definition of each indicator and the adopted data model can be determined according to the shared indicator data table; by copying the shared indicator data table to the shared indicator data table Market, and share it with each data mart after testing it in the sharing mart, realizing the technical effect of sharing indicator data in each data mart, and solving the definition of indicators in each data mart and the data used in indicators in the prior art The problem of model inconsistency enables each data mart to provide support for the business based on a stable, consistent and comprehensive indicator model; by testing the shared indicator data table in the shared mart, data caused by node failures, network failures, etc. are avoided. The impact of loss and data corruption ensures the quality of indicator data shared in various data marts.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. A method for index sharing among data marts is characterized by comprising the following steps:

determining at least one shared index data table in index data tables of a plurality of data marts, and creating a test case corresponding to the shared index data table according to any one of the at least one shared index data table; wherein the shared index data table includes index data to be shared among the plurality of data marts;

copying the shared index data sheet and the test case to a shared mart, and testing the corresponding shared index data sheet in the shared mart by using the test case; and

any shared index data table that passes the test is copied to each data mart that does not store the shared index data table.

2. The method of claim 1, wherein the test case comprises: a testing step, and an expected result corresponding to the testing step.

3. The method of claim 2, wherein the testing the corresponding shared index data table in the shared mart using the test case comprises:

executing a test step in a corresponding test case aiming at least one shared index data table in the shared mart;

and when the test result is completely consistent with the expected result in the test case, determining the shared index data table as passing the test.

4. The method of claim 3, further comprising:

when the test result is inconsistent with the expected result in the test case: and removing the shared index data table, copying the shared index data table which is determined in the index data tables of the plurality of data marts and corresponds to the shared index data table to the shared marts again, and testing the shared index data table by using the test case so as to prevent errors or loss of data in the copying process.

5. The method of any of claims 1-4, wherein the plurality of data marts are subordinate data marts based on a same data warehouse, and wherein the plurality of data marts, the shared mart, and the data warehouse all store data using a distributed file system architecture.

6. An apparatus for index sharing between data marts, comprising:

the index determining unit is used for determining at least one shared index data table in the index data tables of the data marts and creating a test case corresponding to the shared index data table according to any one of the at least one shared index data table; wherein the shared index data table includes index data to be shared among the plurality of data marts;

the index testing unit is used for copying the shared index data sheet and the test case to a shared mart and testing the corresponding shared index data sheet in the shared mart by using the test case;

and the index sharing unit is used for copying any one sharing index data table passing the test to each data mart which does not store the sharing index data table.

7. The apparatus of claim 6, wherein the test case comprises: a testing step and an expected result corresponding to the testing step; and

the index testing unit is used for: executing a test step in a corresponding test case aiming at least one shared index data table in the shared mart; and when the test result is completely consistent with the expected result in the test case, determining the shared index data table as passing the test.

8. The apparatus of claim 6 or 7, wherein the plurality of data marts are subordinate data marts based on a same data warehouse, and wherein the plurality of data marts, the shared mart, and the data warehouse all store data using a distributed file system architecture.

9. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-5.

Images