CN105630475A - Data label organization system and organization method - Google Patents
Data label organization system and organization method Download PDFInfo
- Publication number
- CN105630475A CN105630475A CN201410624275.2A CN201410624275A CN105630475A CN 105630475 A CN105630475 A CN 105630475A CN 201410624275 A CN201410624275 A CN 201410624275A CN 105630475 A CN105630475 A CN 105630475A
- Authority
- CN
- China
- Prior art keywords
- data
- data tag
- tag
- module
- sql
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000008520 organization Effects 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000001914 filtration Methods 0.000 claims abstract description 15
- 230000002776 aggregation Effects 0.000 claims description 33
- 238000004220 aggregation Methods 0.000 claims description 33
- 238000003780 insertion Methods 0.000 claims description 10
- 230000037431 insertion Effects 0.000 claims description 10
- 238000012986 modification Methods 0.000 claims description 8
- 230000004048 modification Effects 0.000 claims description 8
- 238000004806 packaging method and process Methods 0.000 claims description 5
- 230000003993 interaction Effects 0.000 claims 2
- 230000004927 fusion Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 13
- 238000005192 partition Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data label organization system, which comprises a data label application module, a data label compiling module and an execution storage module, wherein the data label application module is used for applying for a required business data label according to a user instruction; the data label compiling module is used for compiling the required business data label into a SQL (Structured Query Language) statement based on a standard SQL according to defined metadata information; and the execution storage module is used for executing and storing the complied SQL statement based on the standard SQL. The invention also provides a data label organization method, which comprises the following steps: applying for the data label; defining the metadata information; on the basis of the metadata information, compiling the required business data label into the SQL statement based on the standard SQL; and executing and storing the SQL statement based on the standard SQL. A data business filtering rule is defined in one time to automatically obtain the data labels assigned in various data platforms so as to meet the requirements of users that various data can be conveniently, efficiently and accurately obtained from different data platforms.
Description
Technical Field
The present disclosure relates to data tag management technologies, and in particular, to a data tag organization system and an organization method.
Background
Currently, in the open data platform trading market, there are many different data providers, providing tens of millions of data tags. Data processing is generally performed manually to obtain a specified data tag, and the data processing is roughly divided into two categories: online transaction processing (OLTP), which is the primary application of relational databases, and online analytical processing (OLAP), performance is measured by response time; OLAP is a primary application of data warehouse systems, taking throughput as a primary measure. In both data application environments, a large amount of manpower and material resources must be consumed, and various complex and variable data label requirements can be supported by manually operating a large amount of service logic codes.
In the method, the intervention of third-party technicians is needed from the requirement description service to the final result, so that the final development result is different from the requirement due to the service understanding difference; or the defects that although different requirements have commonality, the development is still required to be repeatedly developed, so that the development efficiency is not high and the universality is poor exist.
In view of the above, it is necessary to provide a system and a method for performing data tagging organization suitable for different data platforms, so as to satisfy the requirement of a user to simply, efficiently and accurately obtain various data from different data platforms.
Disclosure of Invention
The application provides a data tag organization system, comprising: the data label application module is used for applying a required service data label according to a user instruction; and the data tag compiling module is used for compiling the required business data tags into SQL statements based on standard SQL according to the defined metadata information.
The application also provides a data tag organization method, which comprises the following steps: applying for a data tag; defining metadata information; and compiling the required business data tag into SQL statements based on standard SQL according to the metadata information.
By adopting the data tag organization system and the data tag organization method, the appointed data tags in various data platforms can be automatically acquired by defining the data service filtering rule once, so that the requirement of a user for simply, efficiently and accurately acquiring various data from different data platforms can be met.
Drawings
The various aspects of the present application will become more apparent to the reader after reading the detailed description of the application with reference to the attached drawings. Wherein,
FIG. 1 is a block schematic diagram of a data tag organization system of the present application;
FIG. 2 is a sub-block diagram of a module 121 in the data tag organization system of FIG. 1;
FIG. 3 is a schematic diagram of data tag information parameters E-R in the data tag organization system of the present application;
FIG. 4 is a schematic diagram of a data tag information parameter SQL list of FIG. 3;
FIG. 5 is a schematic flow chart of a preferred method of organizing data tags according to the present application;
FIG. 6 is a preferred flow chart illustrating step 200 of the data tag organization method of FIG. 5;
FIG. 7 is a preferred flow chart illustrating step 300 of the data tag organization method of FIG. 5.
Detailed Description
In order to make the disclosure more complete and complete, reference may be made to the accompanying drawings, in which like references indicate the same or similar elements, and to the various embodiments of the disclosure described below. However, it should be understood by those skilled in the art that the examples provided below are not intended to limit the scope covered by the present application. In addition, the drawings are only for illustrative purposes and are not drawn to scale.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. Information may be computer readable instructions, data structures, program sub-units, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic hard disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transmyedia), such as modulated data signals and carrier waves.
Specific embodiments of various aspects of the present application are described in further detail below with reference to the attached figures.
Referring to fig. 1, a block diagram of a data tag organization system according to the present application is shown. The data tag organization system 1 and the user 2 are in interactive communication through a visual interface, and can organize and compile according to the instruction of the user 2 and the requirement of the user 2 on the data tag, so that a SQL statement based on standard SQL for query is provided for the user 2.
In a preferred embodiment of the present application, the data tag organization system 1 comprises a data tag application module 11, a data tag compiling module 12 and an execution storage module 13. The data tag applying module 11 is configured to apply for a required service data tag according to a user instruction, the data tag compiling module 12 is configured to compile the required service data tag into a standard SQL-based SQL statement according to defined metadata information, and the execution storage module 13 is configured to execute and store the compiled standard SQL-based SQL statement.
Specifically, in another preferred embodiment of the present application, the data tag compiling module 12 further includes: a data tag definition module 120 and a program module 121. The data tag definition module 120 is configured to define preset data tag information according to the metadata information, and the program module 121 is configured to compile a required service data tag into a standard SQL-based SQL statement according to the preset data tag information.
The execution storage module 13 further includes an execution module 130, and a storage module 131. The execution module 130 is configured to execute the compiled SQL statements based on the standard SQL, and the storage module 131 is configured to store the compiled SQL statements based on the standard SQL.
In this embodiment, the data tag definition module 120 defines the preset data tag information according to the metadata information. The metadata information includes an entity-relation graph (E-R graph) of the constructed data tag, and the logical information and the physical information of the data tag are set according to the E-R graph. For example, the E-R map includes the following default data tag information definitions:
the data label data source is used for determining the storage information of the basic data label;
the data label fusion mode is used for determining the fusion mode of the basic data label;
the data label factor logic is used for determining the relation between the data label and a basic data label and the filtering rule of the basic data label;
the business logic of the data label is used for determining the business logic relation between the data label and the data label factor; the applied data tag requirement is used for determining the applied data tag and the aggregation dimension of the data tag.
A data tag container for determining a storage location of the data tag; and
and the data label quality is used for determining that the data quality of the data label meets the requirement of the metadata information.
Specifically, please refer to fig. 1 and fig. 3 together, fig. 3 is a schematic diagram illustrating the definition of the preset data tag information in another embodiment of the present application, and briefly illustrates an example of the relationship between tag entities in a preset E-R diagram. Wherein, the data label factor logic and the data label data source are of a many-to-many (M: N) reference connection type; the fusion mode of the data label factor logic and the data label is a many-to-many (M: N) constraint connection type; the data tag factor logic and the data tag service logic define a contact type many-to-many (M: N). The applied data label requirement and data label service logic is 1-to-many (1: N) definition contact type; the applied data label requirement and label container are in a 1-to-1 (1:1) storage contact type, and the applied data label requirement and label quality are in a 1-to-1 (1:1) monitoring contact type.
Further, logic setting is performed on each entity data tag information in the E-R diagram, please refer to fig. 3 and fig. 4, which schematically show the logic setting content. In this embodiment, the logic setting includes setting a data tag factor logic table according to the data tag factor logic, setting a data tag service logic table according to the data tag service logic, setting a data tag requirement table according to the data tag requirement, setting a basic tag container table according to the data tag data source, setting a basic tag fusion table according to the tag fusion mode, setting a data tag container table according to the data tag, and setting a data tag quality table according to the data tag quality.
The data label factor logic table and the data label service logic table are in many-to-many (M: N) relation. In this embodiment, the data tag factor logic table is named as [ tags _ factor _ tab ] according to the data tag factor logic table, and is used for determining a relationship between a tag factor and a basic tag, a storage location of the basic tag, and a service filtering rule of the basic tag. At least a data tag factor identifier (factor _ id), a base tag container table identifier (src _ tab _ id), and an validity determination (is _ valid) may be included to determine the validity of the relationship and the storage location. In other embodiments of the present application, the data tag factor logic table may further include other contents, such as a continuity decision (is _ constant).
The data label service logic table and the data label requirement table are in multi-pair 1(N:1) connection. In this embodiment, the data tag service logic table is named as [ tags _ expr _ TAB ], and is used to determine the logical relationship between the data tag and the tag factor, the data tag alias, and whether to be aggregated, including at least the data tag expression definition and type (controlled on TAB or COL). In this embodiment, the data tag service logic table at least includes a data tag service logic identifier (expr _ id), a service logic type (expr _ type), a service logic factor (expr _ factor), and a validity determination (is _ valid). In other embodiments of the present application, the data tag business logic table may further include an aggregation dimension decision (is _ aggregate), and the like.
The data label requirement table is in 1-to-1 relation with the data label container table. In this embodiment, the data tag requirement table is named as [ tags _ demand _ tab ], and is used for determining an expression list of data tags and an expression list of tag aggregation, and defining a data storage location of the data tags. In this embodiment, the data tag requirement table at least includes a data tag service logic identifier (expr _ id), an aggregation service logic identifier (aggregate _ expr _ id), a data tag container table identifier (target _ tab _ id), and an validity determination (is _ valid), and is used to determine whether the expression and the storage location are valid. In other embodiments of the present application, the data tag requirement table may further include other data tag data information.
The base label container table and the data label factor logic table are in many-to-many (M: N) relation. In this embodiment, the base tag container table is named as [ tags _ src _ tab ] for determining a specific storage location of the base tag. In this embodiment, the basic tag container table at least includes an identifier (tab _ id), a table name (tab _ name), and a validity determination (is _ valid) of the table. In other embodiments of the present application, the base tag container table may further include other base tag data information.
The basic label fusion table and the data label factor logic table are in many-to-many (M: N) connection. In this embodiment, the basic tag container table is named as [ tags _ join _ tab ], and is used to determine a data fusion manner of the basic tag, that is, determine whether to perform association and association conditions thereof through natural association or one of external association and field association. In this embodiment, the basic tag fusion table at least includes a left table identifier (left _ tab _ id), a right table identifier (right _ tab _ id), a fusion type (join _ type), and a validity determination (is _ valid). In other embodiments of the present application, the basic tag fusion table may further include other basic tag fusion data information.
The data label container table is associated with the data label requirement table as 1 to 1(1: 1). In this embodiment, the data tag container table is named as [ tags _ target _ tab ] for determining a specific storage location of the data tag. In this embodiment, the data tag container table at least includes an identifier (tab _ id), a table name (tab _ name), a table type (tab _ type), and a validity determination (is _ valid) of the table. In other embodiments of the present application, the data tag container table may further include other data tag data information such as partition determination (is _ partition), partition name (partition _ name), partition duration (partition _ period), and the like.
The data label quality table is in 1-to-1 (1:1) relation with the data label requirement table. In this embodiment, the data tag container table is named as [ tags _ quality _ tab ] for determining the data quality of the data tag. In this embodiment, the data tag quality table at least includes an identifier (demand _ id), a data tag quality table name (qa _ name), and a table value (qa _ value) of the data tag requirement table. In other embodiments of the present application, the data tag quality table may further include quality data information of other data tags.
It should be noted that, in this embodiment, each of the aforementioned tables is set as an ANSISQL list suitable for standardized SQL operations after the corresponding preset data tag information parameter is logically set. Further, the ANSISQL tables and their specific parameters are physically set. In the present embodiment, a primary key identification (id) is set in all the lists; a data tag creation time record (gmt-created) to obtain the data tag creation time parameter; and a data tag modification time record (gmt-modified) to obtain the data tag modification time parameter. In this embodiment, the data tag parameter in each table is set to a STRING (STRING) type, the primary key identifier is set to an Integer (INT) type, and the data tag creation and modification time record is set to a DATE (DATE) type. In other embodiments of the present application, the character type setting of the parameters may be changed according to actual requirements and platform requirements, and the above-mentioned manner listed in the embodiment should not be construed as any limitation or restriction on the present application.
For the purpose of briefly explaining the gist of the present application and to clarify the relationship and functions of the modules in the present embodiment, please refer to fig. 1 and fig. 2, wherein fig. 2 is a schematic diagram of the sub-modules of the module 121 in the data tag organization system in fig. 1. In this embodiment, the preset metadata and data tag parameters both support SQL type settings, and are stored in a corresponding list format in computer hardware, software, and a network that can run SQL. This embodiment is described by taking the table class of ANSISQL as an example.
In this embodiment, the program modules 121 further include a selection module 1211, a source module 1213, a condition module 1215, an aggregation module 1217, an insertion module 1219, and a packaging module 1220. The selection module 1211 is configured to obtain an output field of the SQL list corresponding to the data tag information parameter, so as to obtain an output field parameter of the data tag. Selection module 1211 may interact with the user through a [ select ] instruction.
The source module 1213 is configured to obtain metadata of a data tag in the ANSISQL table (table), and determine an association mode parameter of the data tag according to the data tag data information, including dynamically obtaining a table to be associated, and dynamically obtaining a correct association order of the table according to the metadata information. The source module 1213 may interact with the user via the from command. For example, according to the data tag requirement, A, B, C, D, E5 tables need to be associated, AB is associated through inner connection, B is associated through left outer connection D, E, B is associated through left outer connection A, C, and according to the aforementioned E-R graph setup, and the corresponding algorithm, for example, through the breadth first search algorithm of the directed graph, all nodes are traversed to finally obtain five association modes as shown in the following table one:
watch 1
The table sequence which can cover all the table sequences is the final association sequence, so the 5 th table association mode is selected, and the association mode parameter of the data label is determined.
The condition module 1215 is used for obtaining the selection condition of the data label in the ANSISQL table, and determining the filtering rule parameter of the data label. The condition module 1215 may interact with the user through the where instruction.
The aggregation module 1217 is configured to obtain aggregation information of the data tags in the ANSLSQL table, and determine an aggregation dimension parameter of the data tags. The aggregation module 1217 may interact with the user via a group command; .
The inserting module 1219 is configured to determine detailed parameters of the data tag in the target table according to the insertion of the data tag in the ANSLSQL table. The insertion module 1219 may interact with the user via group instructions.
The packaging module 1220 is used for controlling the selection module 1211, the source module 1213, the condition module 1215, the aggregation module 1217 and the insertion module 1219 to compile according to the output field of the data tag, the data tag association manner, the filter rule, the aggregation dimension and the detailed parameters of the target data tag, so as to obtain the SQL statement based on the standard SQL. The packaging module 1220 may interact with the user through a package command.
Therefore, the user 2 interacts with the standard ANSESQL through a universal visual interface, and the data label organization system receives a label demand instruction and feeds a target data label back to the user 2.
Therefore, by the data tag organization system, the data tags appointed in various data platforms can be obtained by defining the data service filtering rule once, so that the requirement of a user for simply, efficiently and accurately obtaining various data from different data platforms can be met.
Fig. 5 is a schematic diagram illustrating a preferred flow chart of a data tag organization method according to an embodiment of the present application. Referring to fig. 1-5, fig. 5 is illustrated in conjunction with the data tag organization system of fig. 1-4.
In this embodiment, the data tag organization method includes the following steps:
step 100, applying for a data tag. In this embodiment, the applied data tag applies for the required service data tag according to the user instruction through the data tag application module.
Step 200, defining metadata information. In this embodiment, the metadata information is defined by the data tag definition module, and defining the metadata information includes constructing an entity-relationship diagram (E-R diagram) of a data tag, and defining an SQL list and a physical configuration of a logical configuration of data tag parameters according to the E-R diagram.
Referring to fig. 6, a preferred flowchart of step 200 in fig. 5 is shown. In another preferred embodiment of the present application, the step 200 further includes a step of defining preset data tag information according to the metadata information, specifically:
at step 2001, a data tag data source is defined to determine the storage information of the underlying data tag. In this embodiment, the data tag data source is implemented by a logically configured base tag container table. In this embodiment, the logically set corresponding data tag parameter lists are all parameter lists suitable for ANSISQL, where the corresponding data tag parameters are all set to be of a STRING (STRING) type, and the specific implementation manner thereof is described in detail with reference to fig. 1, fig. 2, fig. 3, and fig. 4, and will not be described herein again.
Step 2003, defining a data tag fusion mode to determine the fusion mode of the basic data tag. In this embodiment, the fusion manner of the data tags is implemented by a data tag fusion table configured by logic. For a specific implementation, please refer to the above description, and further description is omitted here.
Step 2005, define data tag factor logic, to determine the relationship of the data tag and the base data tag and the filtering rule of the base data tag. In this embodiment, the data tag factor logic is implemented by a logic set data tag factor logic table. For a specific implementation, please refer to the above description, and further description is omitted here.
Step 2007, defining the business logic of the data tag to determine the business logic relationship of the data tag and the data tag factor. In this embodiment, the service logic of the data tag is implemented by a data tag service logic table which is logically set. For a specific implementation, please refer to the above description, and further description is omitted here.
Step 2009, defining the requirements of the applied data tags to determine the expression parameters of the applied data tags and the aggregation of the data tags. In this embodiment, the data storage location of the data tag is further determined by defining the requirement of the applied data tag. The aggregation dimension of the data tags is realized by a data tag requirement table which is logically arranged. For a specific implementation, please refer to the above description, and further description is omitted here.
Step 2011 defines a data tag container to determine a storage location for the data tag. In this embodiment, the data tag container is implemented by a logically configured data tag container table. Please refer to the foregoing for a specific implementation manner, which is not described herein again.
Step 2013, defining the quality of the data label to ensure that the data quality of the data label meets the requirement of the metadata information. In this embodiment, the data tag quality is implemented by a logically configured data tag quality table. For a specific implementation, please refer to the above description, and further description is omitted here.
Preferably, in another embodiment of the present application, the defined data tag parameter and the corresponding SQL list thereof may also be physically set, and the content of the physical setting refers to the foregoing, and is not described herein again. The physical arrangement further comprises the steps of:
step 2015, defining the primary key identification of the SQL table. In this embodiment, the preset data tag information parameters in steps 2001 to 2013 are implemented by a corresponding SQL parameter list that is logically configured, and in this embodiment, the primary key identifier is correspondingly configured in the data tag information parameter SQL table, and the primary key identifier is configured as an Integer (INT) type, so that a user can interact with the data tag organization system and maintain the corresponding list.
Step 2017, defining a data tag creation time record to obtain the data tag creation time parameter, so as to maintain the corresponding SQL table.
Step 2019, defining a data tag creation time record to obtain the data tag creation time parameter, so as to maintain the corresponding SQL table. In this embodiment, the data tag creation and modification time record is set to a DATE (DATE) type.
And 300, compiling the required business data tag into a SQL statement based on standard SQL according to the metadata information.
Referring to fig. 7, a preferred flowchart of step 300 in another embodiment of the present application is shown. In another preferred embodiment of the present application, the step 300 further includes a step of compiling the required business data tag into a SQL statement based on standard SQL according to the preset data tag information, specifically:
step 3001, obtaining the output field parameter of the data tag. In this embodiment, this step obtains the output field parameter of the data tag through the selection module 1211. The specific working manner of the selection module is described in the foregoing, and is not described herein again.
Step 3003, obtaining metadata of the data tag, and determining an association mode parameter of the data tag according to the preset data tag information. In this embodiment, the source module 1213 is configured to obtain metadata of the data tag, and determine an association mode parameter of the data tag according to the preset data tag information. The specific working method includes obtaining the tables to be associated dynamically, and obtaining the correct association sequence of the tables dynamically according to the metadata information, and the like, please refer to the foregoing description, which is not described herein again.
Step 3005, obtaining the selection condition of the data label, and determining the filtering rule parameter of the data label. In this embodiment, the condition module 1215 is configured to obtain a selection condition of a data tag and determine a filtering rule parameter of the data tag, and please refer to the foregoing description for the specific working manner, which is not described herein again.
Step 3007, obtaining aggregation information of the data tag, and determining an aggregation dimension parameter of the data tag. In this embodiment, the aggregation module 1217 is configured to obtain aggregation information of the data tags in the ANSLSQL table, and determine aggregation dimension parameters of the data tags, and the specific working manner of the aggregation module is described in the foregoing, which is not described herein again.
Step 3009, determine the detailed parameters of the target data tag according to the insertion of the data tag. In this embodiment, the aggregation module 1217 is configured to obtain aggregation information of the data tags in the ANSLSQL table, and determine aggregation dimension parameters of the data tags, and the specific working manner of the aggregation module is described in the foregoing, which is not described herein again.
Step 3011, determine the detailed parameters of the data tag according to the insertion of the data tag. In this embodiment, the packaging module 1220 is used for controlling the selection module 1211, the source module 1213, the condition module 1215, the aggregation module 1217 and the insertion module 1219 to compile according to the output field of the data tag, the data tag association manner, the filtering rule, the aggregation dimension and the detailed parameters of the target data tag, so as to obtain the standardized SQL. The detailed working method is described above and will not be described herein.
In view of this, the user 2 interacts with the standard ANSISQL through a general visual interface, receives a tag demand instruction through the data tag organization method, and feeds back a target data tag to the user 2.
Further, in another preferred embodiment of the present application, step 400 is further included to execute and store the compiled SQL statement based on the standard SQL.
Therefore, by the data tag organization system and the data tag organization method, the data tags appointed in various data platforms can be automatically acquired by defining the data service filtering rule once, so that the requirements of a user for simply, efficiently and accurately acquiring various data from different data platforms can be met.
Hereinbefore, specific embodiments of the present application are described with reference to the drawings. However, those skilled in the art will appreciate that various modifications and substitutions can be made to the specific embodiments of the present application without departing from the spirit and scope of the application. Such modifications and substitutions are intended to be included within the scope of the appended claims.
Claims (10)
1. A data tag organization system is used for a data platform capable of data tag interaction so as to obtain a designated data tag according to user requirements, and is characterized by comprising:
the data label application module is used for applying a required service data label according to a user instruction; and
and the data tag compiling module is used for compiling the required business data tags into SQL statements based on standard SQL according to the defined metadata information.
2. The data tag organization system according to claim 1, further comprising an execution storage module for executing and storing compiled standard SQL-based SQL statements.
3. The data tag organization system of claim 1, wherein the data tag compilation module further comprises:
the data tag definition module is used for defining preset data tag information according to the metadata information; and
and the program module is used for compiling the required business data label into a SQL statement based on standard SQL according to the preset data label information.
4. The data tag organization system of claim 3, wherein the program modules further comprise:
the selection module is used for acquiring the output field parameters of the data label;
the source module is used for acquiring metadata of the data tags and determining the association mode parameters of the data tags according to the preset data tag information;
the condition module is used for acquiring the selection condition of the data label and determining the filtering rule parameter of the data label;
the aggregation module is used for acquiring aggregation information of the data tags and determining aggregation dimension parameters of the data tags;
the inserting module is used for determining detailed parameters of the data tag according to the insertion of the data tag; and
and the packaging module is used for controlling the selection module, the source module, the condition module, the aggregation module and the insertion module to compile so as to obtain the SQL statement based on the standard SQL.
5. A data tag organization method is used for a data platform capable of carrying out data tag interaction so as to obtain a designated data tag according to user requirements, and is characterized by comprising the following steps:
applying for a data tag;
defining metadata information; and
and compiling the required business data tag into SQL statements based on standard SQL according to the metadata information.
6. The data tag organization method of claim 5, wherein the step of compiling the required business data tags into SQL statements based on standard SQL according to the metadata information further comprises:
defining preset data tag information according to the metadata information; and
and compiling the required business data label into a SQL statement based on standard SQL according to the preset data label information.
7. The data tag organization method of claim 5 or 6, further comprising:
and executing and storing the compiled SQL sentences based on the standard SQL.
8. The method of claim 6, wherein the defining the predetermined data tag information according to the metadata information further comprises:
defining data label factor logic to determine the relationship between the data label and the basic data label and the filtering rule of the basic data label;
defining business logic of a data tag to determine business logic association of the data tag with the data tag factor; and
defining a data tag requirement for the application to determine an expression parameter for the data tag for the application and the aggregation of the data tags.
9. The data tag organization method of claim 6, wherein compiling the required business data tags into SQL statements based on standard SQL according to the preset data tag information further comprises:
acquiring output field parameters of the data label;
acquiring metadata of the data tags, and determining association mode parameters of the data tags according to the preset data tag information;
acquiring a selection condition of the data label, and determining a filtering rule parameter of the data label;
acquiring aggregation information of the data tags, and determining aggregation dimension parameters of the data tags;
determining detailed parameters of the data tag according to the insertion of the data tag; and
and compiling according to the output field parameters, the association mode parameters, the filtering rule parameters, the aggregation dimension parameters and the detailed parameters of the data tags to obtain the SQL statement based on the standard SQL.
10. The method of claim 6, wherein the defining the predetermined data tag information according to the metadata information further comprises:
setting an information parameter SQL list of the data tag and defining a main key identification of the data tag;
defining a data tag creation time record to obtain a creation time parameter of the data tag; and
defining a data tag modification time record to obtain a modification time parameter of the data tag.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410624275.2A CN105630475B (en) | 2014-11-06 | 2014-11-06 | A kind of data label organization system and method for organizing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410624275.2A CN105630475B (en) | 2014-11-06 | 2014-11-06 | A kind of data label organization system and method for organizing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105630475A true CN105630475A (en) | 2016-06-01 |
| CN105630475B CN105630475B (en) | 2018-12-21 |
Family
ID=56045465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410624275.2A Active CN105630475B (en) | 2014-11-06 | 2014-11-06 | A kind of data label organization system and method for organizing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105630475B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108809770A (en) * | 2018-07-26 | 2018-11-13 | 郑州云海信息技术有限公司 | A kind of resource monitoring method and system |
| CN108874971A (en) * | 2018-06-07 | 2018-11-23 | 北京赛思信安技术股份有限公司 | A tool and method for massive tagged entity data storage |
| CN109063151A (en) * | 2018-08-08 | 2018-12-21 | 中国建设银行股份有限公司 | A kind of commercial banks data fusion method and device |
| CN109189774A (en) * | 2018-09-14 | 2019-01-11 | 南威软件股份有限公司 | A kind of user tag method for transformation and system based on script rule |
| CN110765100A (en) * | 2019-09-09 | 2020-02-07 | 湖南天云软件技术有限公司 | Label generation method and device, computer readable storage medium and server |
| CN111858280A (en) * | 2020-07-16 | 2020-10-30 | 中国工商银行股份有限公司 | SQL information processing method, device, equipment and system |
| CN112000699A (en) * | 2019-05-27 | 2020-11-27 | 阿里巴巴集团控股有限公司 | Data processing mode creating method and device, and data information processing method and device |
| CN112785368A (en) * | 2020-12-24 | 2021-05-11 | 江苏苏宁云计算有限公司 | Label production method, management method, device and system |
| CN116361341A (en) * | 2023-03-20 | 2023-06-30 | 北京白驹易行科技有限公司 | Crowd-sourced circle selection method, crowd-sourced circle selection device, computer equipment and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040254948A1 (en) * | 2003-06-12 | 2004-12-16 | International Business Machines Corporation | System and method for data ETL in a data warehouse environment |
| CN101076793A (en) * | 2004-08-31 | 2007-11-21 | 国际商业机器公司 | System structure for enterprise data integrated system |
| CN101324846A (en) * | 2008-07-08 | 2008-12-17 | 国电南瑞科技股份有限公司 | Method for creating data model according to ASN.1 information dynamic state |
| CN101788992A (en) * | 2009-05-06 | 2010-07-28 | 厦门东南融通系统工程有限公司 | Method and system for converting query sentence of database |
| CN102254008A (en) * | 2011-07-18 | 2011-11-23 | 深圳证券信息有限公司 | Method and system for setting dynamic data label |
| CN103177008A (en) * | 2011-12-22 | 2013-06-26 | 北大方正集团有限公司 | Method and system used for generating and executing structured query language (SQL) statement |
| CN103559243A (en) * | 2013-10-28 | 2014-02-05 | 陶睿 | Method and system for searching users in mobile devices on basis of labels |
-
2014
- 2014-11-06 CN CN201410624275.2A patent/CN105630475B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040254948A1 (en) * | 2003-06-12 | 2004-12-16 | International Business Machines Corporation | System and method for data ETL in a data warehouse environment |
| CN101076793A (en) * | 2004-08-31 | 2007-11-21 | 国际商业机器公司 | System structure for enterprise data integrated system |
| CN101324846A (en) * | 2008-07-08 | 2008-12-17 | 国电南瑞科技股份有限公司 | Method for creating data model according to ASN.1 information dynamic state |
| CN101788992A (en) * | 2009-05-06 | 2010-07-28 | 厦门东南融通系统工程有限公司 | Method and system for converting query sentence of database |
| CN102254008A (en) * | 2011-07-18 | 2011-11-23 | 深圳证券信息有限公司 | Method and system for setting dynamic data label |
| CN103177008A (en) * | 2011-12-22 | 2013-06-26 | 北大方正集团有限公司 | Method and system used for generating and executing structured query language (SQL) statement |
| CN103559243A (en) * | 2013-10-28 | 2014-02-05 | 陶睿 | Method and system for searching users in mobile devices on basis of labels |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108874971B (en) * | 2018-06-07 | 2021-09-24 | 北京赛思信安技术股份有限公司 | A tool and method for mass labelled entity data storage |
| CN108874971A (en) * | 2018-06-07 | 2018-11-23 | 北京赛思信安技术股份有限公司 | A tool and method for massive tagged entity data storage |
| CN108809770A (en) * | 2018-07-26 | 2018-11-13 | 郑州云海信息技术有限公司 | A kind of resource monitoring method and system |
| CN109063151A (en) * | 2018-08-08 | 2018-12-21 | 中国建设银行股份有限公司 | A kind of commercial banks data fusion method and device |
| CN109063151B (en) * | 2018-08-08 | 2022-07-12 | 中国建设银行股份有限公司 | Commercial bank data fusion method and device |
| CN109189774A (en) * | 2018-09-14 | 2019-01-11 | 南威软件股份有限公司 | A kind of user tag method for transformation and system based on script rule |
| CN112000699A (en) * | 2019-05-27 | 2020-11-27 | 阿里巴巴集团控股有限公司 | Data processing mode creating method and device, and data information processing method and device |
| CN112000699B (en) * | 2019-05-27 | 2024-12-13 | 阿里巴巴集团控股有限公司 | Data processing method creation method and device, data information processing method and device |
| CN110765100A (en) * | 2019-09-09 | 2020-02-07 | 湖南天云软件技术有限公司 | Label generation method and device, computer readable storage medium and server |
| CN110765100B (en) * | 2019-09-09 | 2022-08-02 | 天云软件技术有限公司 | Label generation method and device, computer readable storage medium and server |
| CN111858280A (en) * | 2020-07-16 | 2020-10-30 | 中国工商银行股份有限公司 | SQL information processing method, device, equipment and system |
| CN111858280B (en) * | 2020-07-16 | 2024-02-27 | 中国工商银行股份有限公司 | SQL information processing method, device, equipment and system |
| CN112785368A (en) * | 2020-12-24 | 2021-05-11 | 江苏苏宁云计算有限公司 | Label production method, management method, device and system |
| CN116361341A (en) * | 2023-03-20 | 2023-06-30 | 北京白驹易行科技有限公司 | Crowd-sourced circle selection method, crowd-sourced circle selection device, computer equipment and medium |
| CN116361341B (en) * | 2023-03-20 | 2024-02-13 | 北京白驹易行科技有限公司 | Crowd-sourced circle selection method, crowd-sourced circle selection device, computer equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105630475B (en) | 2018-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105630475B (en) | A kind of data label organization system and method for organizing | |
| CN110674228B (en) | Data warehouse model construction and data query method, device and equipment | |
| US12339822B2 (en) | Method and system for migrating content between enterprise content management systems | |
| EP3353672B1 (en) | Method and apparatus for transferring data between databases | |
| US9542469B2 (en) | Data warehouse data model adapters | |
| CN110895544B (en) | Interface data processing method, device, system and storage medium | |
| CN108959510B (en) | Partition level connection method and device for distributed database | |
| CN111813804A (en) | Data query method and device, electronic equipment and storage medium | |
| CN106897342B (en) | Data verification method and equipment | |
| CN112559444B (en) | SQL file migration method, device, storage medium and equipment | |
| CN105447040B (en) | Binary file management and updating method, device and system | |
| CN114138745A (en) | Data integration method, device, storage medium and processor | |
| CN118963743A (en) | A software assembly method, device and medium based on intelligent agent | |
| CN108241620B (en) | Query script generation method and device | |
| US20170351744A1 (en) | Simulation of a synchronization of records | |
| CN108228604B (en) | Model construction method based on memory object, information query method and device | |
| CN117539981A (en) | Method, equipment and medium for constructing theme data set | |
| CN117196509A (en) | Approval opinion display method, device and medium for business form | |
| CN110019448A (en) | A kind of data interactive method and device | |
| CN114138646A (en) | CAD resource verification method and device, storage medium and processor | |
| CN113641644A (en) | Data push system | |
| CN118013084B (en) | A method, device and medium for hierarchical transformation of service list based on recursive algorithm | |
| CN112579634A (en) | Data query method, device, storage medium and equipment | |
| CN114880394B (en) | SQL script sharing method, device and medium | |
| Gajendra | Product recall in supply chain management using Neo4j graph database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211104 Address after: Room 507, floor 5, building 3, No. 969, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Patentee after: ZHEJIANG TMALL TECHNOLOGY Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Patentee before: ALIBABA GROUP HOLDING Ltd. |
|
| TR01 | Transfer of patent right |