CN113553341B - Multidimensional data analysis method, device, equipment and computer-readable storage medium - Google Patents
Multidimensional data analysis method, device, equipment and computer-readable storage medium Download PDFInfo
- Publication number
- CN113553341B CN113553341B CN202110849478.1A CN202110849478A CN113553341B CN 113553341 B CN113553341 B CN 113553341B CN 202110849478 A CN202110849478 A CN 202110849478A CN 113553341 B CN113553341 B CN 113553341B
- Authority
- CN
- China
- Prior art keywords
- dimension
- target
- data cube
- business
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention relates to the technical field of data processing and discloses a multidimensional data analysis method, which comprises the steps of receiving a query request of a user; the multi-dimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data according to the requirements of the business subject on the dimension and the index in advance, the query result is obtained by querying and analyzing the structured query statement in the target data cube, and the query result is returned to the user. Through the mode, the embodiment of the invention realizes the flexibility of inquiry and improves the inquiry efficiency.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a multidimensional data analysis method, a multidimensional data analysis device, multidimensional data analysis equipment and a computer readable storage medium.
Background
Currently, as digitization advances, the amount of data accumulated continues to increase. The need for multidimensional analysis of mass data is increasingly valuable. However, conventional OLAP (online analytical processing) technology is difficult to meet the requirements of high efficiency and easy use. In the data analysis system in the prior art, multidimensional data analysis and query can be generally performed through online analytical processing (OLAP), results are calculated mainly through MapReduce, and are imported into MySQL, oracle and SQL SERVER, for example, so that interactive query of data indexes is realized. The other is to implement user interactive data analysis through SQL interfaces by relying on multidimensional analysis calculation tools such as Kylin, durid and the like, and the SQL needs to be written for multidimensional analysis.
The inventor finds that the existing multidimensional data analysis method has low execution efficiency and is difficult to realize self-service multidimensional analysis.
Disclosure of Invention
In view of the above problems, embodiments of the present invention provide a method, an apparatus, a device, and a computer readable storage medium for multidimensional data analysis, which are used for solving the technical problems in the prior art that user self-help analysis is not achieved and execution efficiency is low.
According to an aspect of an embodiment of the present invention, there is provided a multidimensional data analysis method applied to a multidimensional data analysis apparatus, the method including:
Receiving a query request of a user, wherein the query request comprises a theme, a dimension and an index of service data to be queried;
The multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the service data according to the requirements of service subject on dimension and index in advance;
Inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
In an alternative manner, the determining the target data cube and the structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
Determining target dimension column information in the target data cube according to the dimension of the query request;
determining target measurement column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analysis in the target data cube according to the structured query statement, to obtain a query result, includes:
Determining a dimension value of a target dimension column in the target data cube and an index value of the target measurement column under the target dimension column according to the target dimension column information and the target measurement column information respectively;
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative way, the query result comprises a result list, and after the query result is returned to the user, the method comprises the following steps:
receiving paging and sequencing requests of users;
According to the paging and sorting request, paging and sorting are carried out on the data results in the result list, and a processed result list is obtained;
And returning the processed result list to the user.
In an alternative manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing source data to obtain a fact table and a plurality of dimension tables, wherein the fact table comprises external keys of the dimension tables and fact data;
Acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table with the dimension table, and constructing a data model corresponding to each service theme;
configuring a dimension column and a measurement column on the data model according to the dimension and the index demand information;
According to the dimension table and the associated fact table corresponding to the service theme, determining index values of the measurement columns under the combination of all dimension columns to obtain a data cube;
And mapping the data cubes, the dimension columns and the measurement columns into topics, dimensions and indexes defined by the non-relational database respectively.
According to another aspect of an embodiment of the present invention, there is provided a multidimensional data analysis apparatus including:
The system comprises a receiving module, a query module and a query module, wherein the receiving module is used for receiving a query request of a user, and the query request comprises a theme, a dimension and an index of service data to be queried;
the multi-dimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data according to the requirements of business subjects on dimensions and indexes in advance;
the analysis module is used for carrying out query analysis in the target data cube according to the structured query statement to obtain a query result;
and the return module is used for returning the query result to the user.
In an optional manner, the query analysis in the target data cube according to the structured query statement, to obtain a query result, includes:
Determining a dimension value of a target dimension column in the target data cube and an index value of the target measurement column under the target dimension column according to the target dimension column information and the target measurement column information respectively;
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative manner, the determining the target data cube and the structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
Determining target dimension column information in the target data cube according to the dimension of the query request;
determining target measurement column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
According to another aspect of the embodiment of the invention, a multi-dimensional data analysis device is provided, which comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus, and the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation of the multi-dimensional data analysis method.
According to yet another aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored therein at least one executable instruction that, when run on a multi-dimensional data analysis device, causes the multi-dimensional data analysis device to perform the operations of the multi-dimensional data analysis method described above.
According to the embodiment of the invention, the data cube is constructed according to the service subject, and the service dimension columns and the measurement columns in the data cube are determined according to the dimension and the index corresponding to the service subject, so that the dimension and the index of the service layer can be combined in a self-service manner according to the query request input by the user, the query analysis in the data cube is realized, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present invention can be more clearly understood, and the following specific embodiments of the present invention are given for clarity and understanding.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 shows a flow chart of a multidimensional data analysis method according to an embodiment of the present invention;
Fig. 2 is a schematic structural diagram of a multidimensional data analysis device according to an embodiment of the present invention;
Fig. 3 shows a schematic structural diagram of a multidimensional data analysis device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein.
First, technical terms appearing in the embodiments of the present invention will be explained.
A data cube (cube) is a multidimensional (degree) space built from dimensions, containing all the underlying data (source data) to be analyzed, and all aggregate data operations are performed on the data cube. The data cube is just one visual representation of a multi-dimensional model, which is not limited to a three-dimensional model, but can be combined in many more dimensions.
Dimension is an angle for observing data, and one dimension corresponds to a plurality of dimension tables.
And the measurement value is the data to be analyzed and displayed, namely the index. The metrics columns may be analyzed in multiple dimensions.
Fact table the central table of the dimension database is called the fact table. Its row represents facts, the central content of which is a measurement of different instances of an activity or event, and which stores fact values, and the foreign keys of the dimension table, all of which are ultimately from the fact table.
Dimension table-dimension table places facts into a context that represents things such as time, product, customer, and location, as may be a time dimension table, a product dimension table, a customer dimension table, a location dimension table, and the like.
SQL (Structured Query Language) SQL, a structured query language, is a database query and programming language used to access data and query, update and manage relational database systems.
Fig. 1 shows a flowchart of a multidimensional data analysis method provided by an embodiment of the present invention, the method being performed by a multidimensional data analysis apparatus. The multidimensional data analysis means may be a computer device, a terminal, a distributed device, etc. As shown in fig. 1, the method comprises the steps of:
step 110, receiving a query request from a user.
In the embodiment of the invention, the query request comprises the theme, the dimension and the index of the service data. The query request may be entered by a user at a preset query interface. The query request also includes a business dimension filter condition for determining a business logic tag.
And step 120, determining a target data cube and a structured query statement according to the query request, wherein the multidimensional data analysis device comprises at least one data cube, and the at least one data cube is obtained by processing the business data according to the requirements of business subjects on dimensions and indexes in advance.
And determining a corresponding target data cube according to the business theme in the query request. The target data cube is constructed by processing source data according to the requirements of the business theme on dimensions and indexes in advance.
In the embodiment of the invention, the multidimensional data analysis device comprises a plurality of data cubes, and each data cube corresponds to one service theme. All the data cubes are constructed after processing source data in advance according to the requirements of the service theme on dimensions and indexes. The target data cube is one of a plurality of data cubes.
The structured query term may be an SQL term, where the SQL term includes target dimension column information and target measure column information.
In the embodiment of the invention, determining the target data cube and the structured query statement according to the query request comprises the following steps:
Determining a data cube with a service theme matched with the service theme of the query request as a target data cube from the at least one data cube, wherein the data cube with the service theme matched with the service theme of the query request refers to the same service theme or a certain mapping relation exists, for example, a service theme field in the query request is the same as a service theme field corresponding to the data cube in the multidimensional data analysis device, or an ID corresponding to the theme in the query request is determined, and the ID corresponding to the data cube in the multidimensional data analysis device is determined according to the ID, so that the target data cube is determined;
Determining target dimension column information in the target data cube according to the dimension of the query request;
determining target measurement column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In the embodiment of the invention, the process of constructing the data cube is as follows:
S1, collecting source data. The source data is collected according to preset data specifications, wherein the preset data specifications can be corresponding settings performed by a person skilled in the art according to service scenes.
S2, processing the source data to obtain a fact table and a dimension table. The fact table comprises foreign keys of the dimension table and fact data, and the dimension table comprises at least one dimension table dimension. The process of processing the source data comprises the processes of extracting, checking and converting the data. Specifically, when data in a certain fact table is acquired, a numerical value in the fact table is extracted, the numerical value is checked, for example, whether obvious error data exist or not is checked, and then source data are converted into a required format of the fact table and the dimension table according to a preset conversion rule. The plurality of dimension tables can be a date dimension table, a user type dimension table and the like.
And S3, acquiring each service theme required by the user and dimension and index requirement information corresponding to the service theme. Wherein, each business theme required by the user and the business theme setting the Indeltoid dimension and index requirement according to the actual requirement, the embodiment of the invention is not particularly limited. It is to be appreciated that the business topic corresponds to a plurality of dimensions that correspond to dimensions in the corresponding dimension table.
And S4, correlating the fact table with the dimension table to construct a data model corresponding to the service theme. Specifically, the fact table and the dimension table can be associated according to the service theme to obtain the data model. For example, for the communication field, the service theme may include a user theme, a charging theme, a video color ring theme, etc., for the user theme, a fact table related to the user theme and at least one dimension table related to the user theme in a plurality of dimension tables may be obtained, the related fact table and the at least one dimension table are associated to form a data model corresponding to the user theme, for the charging theme, a fact table related to the charging theme and at least one dimension table related to the charging theme in a plurality of dimension tables may be obtained, the fact table and the at least one dimension table related to the charging theme are associated to form a data model corresponding to the charging theme, and so on, thereby constructing a plurality of data models for different service themes.
And S4, configuring a dimension column and a measurement column on the data model according to the dimension and the index demand information. Taking a user theme as an example, the dimensions required to be configured include dimensions such as date, company, client, login type, user type, channel, province and the like, the metrics include user quantity, browsing times and the like, and according to the dimension table corresponding to the user theme and the associated fact table, index values of the metrics such as the user quantity, the browsing times and the like are counted under each dimension, and the index values can be obtained by inquiring corresponding fact values in the fact table and calculating. The dimension and the measurement configured on the data model are determined according to the requirement of the business theme on the dimension and the index, the requirement of the business theme on the dimension and the index is set by a user according to a specific business theme, the dimension and the index requirement information corresponding to the business theme is obtained, and the target dimension column and the target measurement column are configured on the data model according to the dimension and the index requirement information and the dimension table and the fact table corresponding to the business theme. Specifically, a Kylin multidimensional analysis engine can be relied on to set a dimension field and a measurement field of a data model according to the dimension and index requirements of a user theme. APACHE KYLIN is an open source, distributed, analytical data warehouse that provides SQL query interfaces and multidimensional analysis (OLAP) capabilities over Hadoop/Spark to support very large scale data.
And S5, determining index values of the measurement columns under the combination of all dimension columns according to the dimension table corresponding to the service theme and the associated fact table to obtain a data cube.
When the data Cube is built by relying on the Kylin multidimensional analysis engine, fields such as date, company, client, login type, user type, channel and province are correspondingly set as dimension columns of the data Cube, and fields such as user id, pv (browsing times) are set as measurement columns of the data Cube by taking the requirement of a user theme on dimensions and measurement (indexes) as examples. The combination of the individual dimension columns constitutes one dimension of the data cube. In the process of constructing the data Cube, calculating index values of the measurement columns under the combination of all dimension columns, and storing, namely, respectively combining a plurality of dimension columns of 'date, company, client, login _type' and 'user_ type, channel, province', so as to obtain corresponding measurement columns and values corresponding to the measurement columns, and storing the values corresponding to the measurement columns in the corresponding fact table. So far, the establishment of the multidimensional data Cube configuration is completed, and the second-level multidimensional analysis can be carried out on the data by writing SQL queries. Wherein the target data cube is any one of the multidimensional data cubes.
And S6, mapping the data cubes, the dimension columns and the measurement columns into topics, dimensions and indexes defined by the non-relational database respectively. Further, after the data cube is constructed, mapping the target data cube, the target dimension column and the target measure column into a service theme, a service dimension and a service index respectively:
In the embodiment of the invention, after the data Cube is constructed, the data Cube is abstractly defined in the non-relational database as a service theme, and one service theme corresponds to one data Cube, so that the abstractly defined data Cube is a theme object which comprises the following attributes of theme ID, theme name, chinese name, theme SQL, theme sequence Index, theme description and the like, so that the service theme object inquired by a user through an interface corresponds to the data Cube one by one. Wherein the non-relational database is a NoSQL database.
In the embodiment of the invention, after the data Cube is constructed, the dimension column of the data Cube is abstractly defined as the dimension of the service theme, wherein each dimension column of the data Cube is defined as the dimension object of the service theme, and the dimension object comprises the following attributes of dimension ID, dimension name, dimension Chinese, dimension column, dimension value, dimension sequence Index, dimension description and the like. The business topic dimension of the user interface query corresponds to the dimension column of the data cube. For example, if the user selects a "province" query under the user theme at the interface, the "province" is the dimension of the user theme corresponding to the user interface, and corresponds to the dimension column "precursor" column of the data cube.
In the embodiment of the invention, after the data Cube is constructed, the measurement column of the abstract definition data Cube is used as the Index of the service theme, wherein the measurement column of the definition data Cube is used as the Index of the service theme, and the Index object comprises the following attributes of Index ID, index name, index Chinese, measurement SQL, filtering SQL, index sequence Index, index caliber description, SQL pseudo code, index description and the like. By such a setting, the index input by the interface is made to correspond to the calculation result of the metric column in the data cube. For example, the user selects the "active user number" index of the user topic at the interface, and the metrics column SQL of the corresponding data cube is Count (distinct user _id). The filtering SQL is used for distinguishing business logic labels of the Cube, such as identifying the user type as active or newly added.
Through the above arrangement, the data cube, the dimension column of the data cube, and the metric column of the data cube are mapped into business topics, business dimensions, and business indexes defined by configurable NoSQL (non-relational database). That is, the user inputs a query request including a service theme, a service dimension and a service index on the interface, so that a corresponding data cube, a dimension column of the data cube and a measurement column of the data cube can be determined, and a corresponding structured query sentence is generated according to the target dimension column information and the target measurement column information. The structured query statement is a structured request statement that is statistically analyzed based on a target dimension column and a target measure column in a target data cube.
The generation process of the structured query statement comprises the step of generating a From segment of SQL by using a target data cube corresponding to a service theme. For example, the target data Cube of the query is a user_subject (user subject), the From segment corresponding to the SQL statement is a From user_subject, and the Select segment and the Group by segment of the SQL are generated by using the target dimension column of the target data Cube corresponding to the business dimension of the query. For example, the business dimension of the query is Company name, province, the dimension column in the corresponding target data cube is Company, province, the Select section corresponding to the generated SQL statement is Select Company, province, the Group by section generating the SQL statement is Group by Company, province, and the statistics of SQL is generated using the target metric column of the target data cube corresponding to the business index of the query. For example, the business index is the number of users, the metric column of the corresponding target data cube is the user_id, the statistical value of the corresponding generated SQL sentence is the count (distinct user _id), and the generated SQL segments are assembled into a complete SQL sentence, so that the structured query sentence is obtained.
And 130, carrying out query analysis in the target data cube according to the structured query statement to obtain a query result.
After the structured query sentence is determined, the structured query sentence is executed in the data analysis engine, and according to the target dimension column information and the target measurement column information, a target dimension column and a target measurement column in a target data cube can be determined, so that a dimension value of the target dimension column and an index value of the target measurement column under the target dimension column are determined. Specifically, according to the structured query statement, querying in a fact table of the target data cube, and obtaining an index value of the target measurement column under the target dimension column.
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In the embodiment of the invention, the data structure of the query result is in a semi-structured JSON data format, and comprises dimension column header information, index column header information and a data list. The dimension list head information comprises dimension information of the query and comprises dimension names, chinese names, dimension IDs, dimension indexes and the like, the Index hunting head information comprises Index information of the query and comprises label names, chinese names, index IDs, index indexes and the like, the result list comprises data results of the query, and each result object comprises all dimension values and Index values of the query.
And 140, returning the query result to the user.
When a user inquires a plurality of service indexes, the inquiry results comprise inquiry results corresponding to the service indexes.
In the embodiment of the invention, the paging and sorting request of the user for the query result is received, and the result list in the returned query result is sorted and paged according to the paging and sorting request. The method comprises the steps of receiving a paging and sorting request of a user, paging and sorting data results in a result list according to the paging and sorting request to obtain a processed result list, and returning the processed result list to the user.
In the embodiment of the invention, the implementation of the steps 110 to 130 is encapsulated by the SDK (generally referred to as a software development kit, which is a collection of development tools when some software engineers build application software for specific software packages, software frameworks, hardware platforms, operating systems and the like), so that unique data query interfaces are exposed to the outside, and the data query modes are unified and standardized, so that the invoking systems for the data query are kept consistent.
According to the embodiment of the invention, the business data is constructed into the data Cube according to the business theme, the business dimension columns and the measurement columns in the data Cube are determined according to the dimension and the index corresponding to the business theme, and the multi-dimensional analysis data model Cube is defined in an abstract mode as the theme, the dimension and the index, so that the dimension and the index of the business layer can be combined in a self-service mode according to the query request input by a user, query analysis in the data Cube is realized, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
Fig. 2 shows a schematic structural diagram of a multidimensional data analysis device according to an embodiment of the present invention. As shown in fig. 2, the apparatus 200 includes:
a receiving module 210, configured to receive a query request from a user;
A determining module 220, configured to determine a target data cube and a structured query statement according to the query request;
an analysis module 230, configured to query and analyze in the target data cube according to the structured query statement, so as to obtain a query result;
and the returning module 240 is configured to return the query result to the user.
In an alternative manner, the determining the target data cube and the structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
Determining target dimension column information in the target data cube according to the dimension of the query request;
determining target measurement column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analysis in the target data cube according to the structured query statement, to obtain a query result, includes:
Determining a dimension value of a target dimension column in the target data cube and an index value of the target measurement column under the target dimension column according to the target dimension column information and the target measurement column information respectively;
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative way, the query result comprises a result list, and after the query result is returned to the user, the method comprises the following steps:
receiving paging and sequencing requests of users;
According to the paging and sorting request, paging and sorting are carried out on the data results in the result list, and a processed result list is obtained;
And returning the processed result list to the user.
In an alternative manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing source data to obtain a fact table and a plurality of dimension tables, wherein the fact table comprises external keys of the dimension tables and fact data;
Acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table with the dimension table, and constructing a data model corresponding to each service theme;
configuring a dimension column and a measurement column on the data model according to the dimension and the index demand information;
According to the dimension table and the associated fact table corresponding to the service theme, determining index values of the measurement columns under the combination of all dimension columns to obtain a data cube;
And mapping the data cubes, the dimension columns and the measurement columns into topics, dimensions and indexes defined by the non-relational database respectively.
The working process of the multidimensional data analysis device in the embodiment of the present invention is consistent with the specific steps of the multidimensional data analysis method, and will not be described herein.
According to the embodiment of the invention, the business data is constructed into the data Cube according to the business theme, the business dimension columns and the measurement columns in the data Cube are determined according to the dimension and the index corresponding to the business theme, and the multi-dimensional analysis data model Cube is defined in an abstract mode as the theme, the dimension and the index, so that the dimension and the index of the business layer can be combined in a self-service mode according to the query request input by a user, query analysis in the data Cube is realized, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
Fig. 3 is a schematic structural diagram of a multidimensional data analysis device according to an embodiment of the present invention, and the specific embodiment of the present invention is not limited to the specific implementation of the multidimensional data analysis device.
As shown in FIG. 3, the multidimensional data analysis device may include a processor 302, a communication interface (Communications Interface) 304, a memory 306, and a communication bus 308.
Wherein the processor 302, the communication interface 304, and the memory 306 communicate with each other via a communication bus 308. A communication interface 304 for communicating with network elements of other devices, such as clients or other servers. Processor 302 is configured to execute program 310 and may specifically perform the relevant steps described above for the multidimensional data analysis method embodiment.
In particular, program 310 may include program code comprising computer-executable instructions.
The processor 302 may be a central processing unit CPU, or an Application-specific integrated Circuit ASIC (Application SPECIFIC INTEGRATED Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the multi-dimensional data analysis device may be the same type of processor, such as one or more CPUs, or may be different types of processors, such as one or more CPUs and one or more ASICs.
Memory 306 for storing programs 310. Memory 306 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
Program 310 may be specifically invoked by processor 302 to cause the multidimensional data analysis device to:
Receiving a query request of a user, wherein the query request comprises a theme, a dimension and an index of service data to be queried;
The multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the service data according to the requirements of service subject on dimension and index in advance;
Inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
In an alternative manner, the determining the target data cube and the structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
Determining target dimension column information in the target data cube according to the dimension of the query request;
determining target measurement column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analysis in the target data cube according to the structured query statement, to obtain a query result, includes:
Determining a dimension value of a target dimension column in the target data cube and an index value of the target measurement column under the target dimension column according to the target dimension column information and the target measurement column information respectively;
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative way, the query result comprises a result list, and after the query result is returned to the user, the method comprises the following steps:
receiving paging and sequencing requests of users;
According to the paging and sorting request, paging and sorting are carried out on the data results in the result list, and a processed result list is obtained;
And returning the processed result list to the user.
In an alternative manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing source data to obtain a fact table and a plurality of dimension tables, wherein the fact table comprises external keys of the dimension tables and fact data;
Acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table with the dimension table, and constructing a data model corresponding to each service theme;
configuring a dimension column and a measurement column on the data model according to the dimension and the index demand information;
According to the dimension table and the associated fact table corresponding to the service theme, determining index values of the measurement columns under the combination of all dimension columns to obtain a data cube;
And mapping the data cubes, the dimension columns and the measurement columns into topics, dimensions and indexes defined by the non-relational database respectively.
According to the embodiment of the invention, the data cube is constructed according to the service subject, and the service dimension columns and the measurement columns in the data cube are determined according to the dimension and the index corresponding to the service subject, so that the dimension and the index of the service layer can be combined in a self-service manner according to the query request input by the user, the query analysis in the data cube is realized, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
An embodiment of the present invention provides a computer readable storage medium storing at least one executable instruction that, when executed on a multidimensional data analysis device, causes the multidimensional data analysis device to perform the multidimensional data analysis method in any of the method embodiments described above.
The executable instructions may be specifically operable to cause the multi-dimensional data analysis device to:
Receiving a query request of a user, wherein the query request comprises a theme, a dimension and an index of service data to be queried;
The multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the service data according to the requirements of service subject on dimension and index in advance;
Inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
In an alternative manner, the determining the target data cube and the structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
Determining target dimension column information in the target data cube according to the dimension of the query request;
determining target measurement column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analysis in the target data cube according to the structured query statement, to obtain a query result, includes:
Determining a dimension value of a target dimension column in the target data cube and an index value of the target measurement column under the target dimension column according to the target dimension column information and the target measurement column information respectively;
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative way, the query result comprises a result list, and after the query result is returned to the user, the method comprises the following steps:
receiving paging and sequencing requests of users;
According to the paging and sorting request, paging and sorting are carried out on the data results in the result list, and a processed result list is obtained;
And returning the processed result list to the user.
In an alternative manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing source data to obtain a fact table and a plurality of dimension tables, wherein the fact table comprises external keys of the dimension tables and fact data;
Acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table with the dimension table, and constructing a data model corresponding to each service theme;
configuring a dimension column and a measurement column on the data model according to the dimension and the index demand information;
According to the dimension table and the associated fact table corresponding to the service theme, determining index values of the measurement columns under the combination of all dimension columns to obtain a data cube;
And mapping the data cubes, the dimension columns and the measurement columns into topics, dimensions and indexes defined by the non-relational database respectively.
According to the embodiment of the invention, the data cube is constructed according to the service subject, and the service dimension columns and the measurement columns in the data cube are determined according to the dimension and the index corresponding to the service subject, so that the dimension and the index of the service layer can be combined in a self-service manner according to the query request input by the user, the query analysis in the data cube is realized, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
The embodiment of the invention provides a multidimensional data analysis device which is used for executing the multidimensional data analysis method.
Embodiments of the present invention provide a computer program that is callable by a processor to cause a multidimensional data analysis device to perform the multidimensional data analysis method of any of the method embodiments described above.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when run on a computer, cause the computer to perform the multi-dimensional data analysis method of any of the method embodiments described above.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component, and they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specifically stated.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110849478.1A CN113553341B (en) | 2021-07-27 | 2021-07-27 | Multidimensional data analysis method, device, equipment and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110849478.1A CN113553341B (en) | 2021-07-27 | 2021-07-27 | Multidimensional data analysis method, device, equipment and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113553341A CN113553341A (en) | 2021-10-26 |
| CN113553341B true CN113553341B (en) | 2025-03-04 |
Family
ID=78132890
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110849478.1A Active CN113553341B (en) | 2021-07-27 | 2021-07-27 | Multidimensional data analysis method, device, equipment and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113553341B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114265887A (en) * | 2021-12-31 | 2022-04-01 | 上海金仕达软件科技有限公司 | A dimensional data processing method, device, storage medium and electronic device |
| CN114741445B (en) * | 2022-03-03 | 2024-12-13 | 北京元年科技股份有限公司 | Data export method, device, equipment and computer readable storage medium |
| CN115392799B (en) * | 2022-10-27 | 2023-04-11 | 平安科技（深圳）有限公司 | Attribution analysis method and device, computer equipment and storage medium |
| CN115840772B (en) * | 2022-11-11 | 2024-11-01 | 中电金信软件有限公司 | Passenger group data statistics method and device, electronic equipment and storage medium |
| CN117785984A (en) * | 2024-02-28 | 2024-03-29 | 广州思迈特软件有限公司 | Data extraction methods, devices, electronic equipment and storage media |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286954A (en) * | 2020-09-25 | 2021-01-29 | 北京邮电大学 | Multi-dimensional data analysis method and system based on hybrid engine |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8082239B2 (en) * | 2008-02-04 | 2011-12-20 | Microsoft Corporation | Defining sub-cube scope based upon a query |
| CN110019396B (en) * | 2017-12-01 | 2023-02-17 | 中国移动通信集团广东有限公司 | Data analysis system and method based on distributed multidimensional analysis |
-
2021
- 2021-07-27 CN CN202110849478.1A patent/CN113553341B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286954A (en) * | 2020-09-25 | 2021-01-29 | 北京邮电大学 | Multi-dimensional data analysis method and system based on hybrid engine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113553341A (en) | 2021-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113553341B (en) | Multidimensional data analysis method, device, equipment and computer-readable storage medium | |
| CN107256265B (en) | A kind of search-engine results data visualization methods of exhibiting and system | |
| CN109656963B (en) | Metadata acquisition method, device, device and computer-readable storage medium | |
| CN107861981B (en) | Data processing method and device | |
| US10204111B2 (en) | System and method for compressing data in a database | |
| CN111310052B (en) | User portrait construction method, device and computer readable storage medium | |
| CN112527783A (en) | Data quality probing system based on Hadoop | |
| CN103262076A (en) | Analytical data processing | |
| CN110362591B (en) | Report form display method and device | |
| WO2021012861A1 (en) | Method and apparatus for evaluating data query time consumption, and computer device and storage medium | |
| US20200089798A1 (en) | High volume-velocity time series data ingestion, analysis and reporting method and system | |
| US9727663B2 (en) | Data store query prediction | |
| US9727666B2 (en) | Data store query | |
| CN111198898A (en) | Big data query method and big data query device | |
| US9009161B2 (en) | Data processing | |
| CN116450890A (en) | Graph data processing method, device and system, electronic equipment and storage medium | |
| WO2017107130A1 (en) | Data query method and database system | |
| CN112100177A (en) | Data storage method and device, computer equipment and storage medium | |
| CN108874873B (en) | Data query method, device, storage medium and processor | |
| US11645274B2 (en) | Minimizing group generation in computer systems with limited computing resources | |
| CN113778996A (en) | Large data stream data processing method and device, electronic equipment and storage medium | |
| CN113986947A (en) | A method, apparatus, device and readable storage medium for displaying data flow | |
| CN110489732A (en) | Method for processing report data and equipment | |
| CN113568967B (en) | Dynamic extraction method of time sequence index data, electronic equipment and storage medium | |
| CN116737753A (en) | Service data processing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |