CN104199978A - System and method for realizing metadata cache and analysis based on NoSQL and method - Google Patents
System and method for realizing metadata cache and analysis based on NoSQL and method Download PDFInfo
- Publication number
- CN104199978A CN104199978A CN201410493443.9A CN201410493443A CN104199978A CN 104199978 A CN104199978 A CN 104199978A CN 201410493443 A CN201410493443 A CN 201410493443A CN 104199978 A CN104199978 A CN 104199978A
- Authority
- CN
- China
- Prior art keywords
- metadata
- analysis
- nosql
- task
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 468
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000004891 communication Methods 0.000 claims abstract description 66
- 230000004044 response Effects 0.000 claims abstract description 25
- 238000003672 processing method Methods 0.000 claims abstract description 4
- 239000002131 composite material Substances 0.000 claims description 13
- 230000005540 biological transmission Effects 0.000 claims description 9
- 230000001419 dependent effect Effects 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000010937 topological data analysis Methods 0.000 claims description 7
- 238000012098 association analyses Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 238000007405 data analysis Methods 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 37
- 230000006870 function Effects 0.000 description 11
- 238000000151 deposition Methods 0.000 description 3
- 230000008676 import Effects 0.000 description 3
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a system for realizing metadata cache and analysis based on NoSQL. The system comprises a metadata analyzing client side, a metadata analyzing server side, a NoSQL database server and a communication module. The invention further relates to a method adopting the system for realizing metadata cache and analysis based on the NoSQL. The method comprises a metadata caching method based on the NoSQL and realized through the adoption of the system and a metadata processing method based on the NoSQL and realized through the adoption of the metadata caching method. Through the adoption of the system for realizing metadata cache and analysis based on the NoSQL and the method for realizing metadata cache and analysis based on the NoSQL, according to actual requirement of metadata analysis system under a big data environment, a NoSQL data base is adopted to cache metadata and relevant inquiry methods are provided, the data accessing performance is improved, the response speed of metadata analysis is accelerated, the user experience is improved and the application range is wider.
Description
Technical field
The present invention relates to database field of storage, relate in particular to metadata cache and analysis field based on database, specifically refer to a kind of system and method for realizing metadata cache and analysis based on NoSQL.
Background technology
Along with the development of infotech, the data of magnanimity have been produced.How effectively to manage and to organize these mass datas to become distinct issues.Metadata is the effective means addressing this problem, can back-up system administering and maintaining data.Specifically, in data warehouse, Metadata mechanism is mainly supported following five type systematic management functions: which data (1) describes in data warehouse; (2) data that definition will enter the data in data warehouse and produce from data warehouse; (3) the data pick-up working time that record carries out according to business event generation thereupon arranges; (4) requirement and the implementation status of record detection system data consistency; (5) weigh the quality of data.
In metadata management system, most important to effective analysis of metadata.Metadata analysis function mainly realizes the fundamental analysis function for metadata.Comprise consanguinity analysis (blood lineage's analysis), impact analysis, entity associated analysis, entity impact analysis, main frame topological analysis, indicator consilience analysis etc.
Searching of the data correlation relation of current metadata analysis system is all to complete by relational database, finally forms source and the whereabouts graph of a relation of data.Restriction due to relational database theory, relation between metadata entity and metadata is stored in respectively in a plurality of database tables, in general, be stored in three database tables: metadata example table, metadata combination relation table and metadata dependence table, when carrying out the inquiry of data correlation relation, need to carry out the attended operation between multilist, multilist is carried out to resource that attended operation expends computing machine very much as CPU, internal memory etc., if also stored the data of magnanimity in table, the response time of data query can be very slow.
Along with the arrival of large data age, data and flow constantly increase, and must have more computational resource to deal with the situation of data explosion.For the consideration of rate of return on investment, the Longitudinal Extension cost of computing power is very high and expansion yardstick is also limited; The cluster consisting of the cheap hardware of a plurality of uses of employing extending transversely of computing power can also improve the reliability of cluster when reducing the required cost of expansion.Yet, relational database is not to be designed to move on cluster, although can overcome by some technological means the problem of operational relation type database in cluster, solve all very complicated such as the method for burst inquiry, referential integrity, affairs, consistance control etc.
For solving the inharmonious of relevant database and cluster, researchist conducts extensive research can store new departure of data in cluster, and wherein the theory of NoSQL database and product develop rapidly, have obtained a large amount of concerns of people.A not rigorous definition of NoSQL, generally it is believed that it is " the non-relational database that does not use sql like language ", NoSQL database possesses following several feature: 1. NoSQL database is not used SQL, does not have NoSQL database to realize the sql like language of standard; 2. NoSQL database is all the project of increasing income conventionally; 3. most NoSQL databases are all in order to move in cluster environment, thus their data model and data consistency scheme different; 4. NoSQL database is not used " pattern ", need not revise in advance organization definition and just can freely add Database field.
Incidence relation between metadata entity is because the characteristic of relevant database is not supported that complex data type etc. is former thereby is stored in a plurality of tables as data in table can not have in redundancy, table data type, if moved in NoSQL database, more meet the requirement of relational query between the data entity of metadata analysis.
Summary of the invention
The object of the invention is to overcome the shortcoming of above-mentioned prior art, providing a kind of is carried out cache metadata and corresponding querying method is provided by NoSQL database, save system resource, improved the performance of data access, accelerate the response speed of metadata analysis, Optimizing Queries performance based on NoSQL, realize the system and method for metadata cache and analysis.
To achieve these goals, the system and method for realizing metadata cache and analysis based on NoSQL of the present invention has following formation:
Should based on NoSQL, realize the system of metadata cache and analysis, its principal feature is that described system comprises:
Metadata analysis client, in order to respond the metadata analysis instruction of user's input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and show described metadata analysis result;
Metadata analysis service end, the metadata analysis message sending in order to resolve described communication module, and the metadata analysis task obtaining according to parsing inquiry NoSQL database server obtains metadata analysis result, and the corresponding analysis result message of described metadata analysis result is sent to described communication module;
NoSQL database server, in order to generate NoSQL metadata table, and by the data buffer storage in metadatabase in described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response;
Communication module, in order to described metadata analysis message is sent to described metadata analysis server, and is sent to described metadata analysis client by described analysis result message.
Further, described metadata analysis client comprises graphic interface module and task parsing module, wherein:
Described graphic interface module, graphic interface in order to display element data analysis, and the metadata analysis instruction of the input of the user described in response, with the data of metadata analysis task corresponding to described metadata analysis instruction are sent to described task parsing module, and show the metadata analysis result that described task parsing module sends;
Described task parsing module, in order to the data of described metadata analysis task are converted to corresponding metadata analysis message, and described metadata analysis message is sent to described communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and described metadata analysis result is sent to described graphic interface module.
Further, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, wherein:
Described task scheduling modules, in order to resolve the metadata analysis message of described communication module transmission and to obtain corresponding metadata analysis task, and by described NoSQL database interface, inquire about described NoSQL database server according to described metadata analysis task, and obtain described metadata analysis result corresponding to metadata analysis task, and be institute's correspondence analysis results messages by corresponding metadata analysis results conversion;
Described NoSQL database interface, in order to access described NoSQL database server.
Further, it is characterized in that, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, wherein:
Described metadata management module, in order to generate NoSQL metadata table and NoSQL database, and by the data buffer storage in described metadatabase in described NoSQL database, and obtain related data information by described metadata query module inquiry, and described related data information is saved in described NoSQL metadata table, and the inquiry request of the metadata analysis service end described in response;
Described metadata query module, in order to inquire about described NoSQL database according to row key word by described application programming interfaces, and obtains related data information;
Described application programming interfaces, in order to access the application program of described NoSQL database server.
Further, described NoSQL database server comprises name module, described name module in order to the NoSQL database server to described register, add, the management of monitoring and fault detect.
Further, described NoSQL database comprises metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
Wherein, described metadata analysis instruction comprises metadata analysis function type, the information of metadata type and metadata title, described metadata analysis task is consanguinity analysis task, impact analysis task, association analysis task, topological analysis task or index analysis task, described metadata analysis task comprises client the Internet protocol address information, client end slogan information, task sequence information, task names information, task type information and mission bit stream, described metadatabase comprises metadata example table, metadata combination relation table and metadata dependence table.
In addition, the present invention also provides a kind of and utilizes described system to realize the metadata cache method based on NoSQL, and described method is specially:
Described NoSQL database server generates NoSQL metadata table, and by the data buffer storage in metadatabase to described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response.
Further, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, described NoSQL database server generate NoSQL metadata table and by the data buffer storage in metadatabase to described NoSQL metadata table, comprise the following steps:
(a) described metadata management module generates NoSQL metadata table and NoSQL database;
(b) described metadata management module by the data buffer storage in described metadatabase to described NoSQL database;
(c) described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information;
(d) described metadata management module is saved to described related data information in described NoSQL metadata table.
Further, described metadata management module generates NoSQL metadata table and NoSQL database, is specially:
Described metadata management module generates NoSQL metadata table, metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
Further, described metadata management module to described NoSQL database, comprises the following steps the data buffer storage in described metadatabase:
(b.1) the metadata example table in described metadata management module query metadata storehouse, and by the data buffer storage of described metadata example table to described metadata instance objects storehouse;
(b.2) the metadata dependence table in the described metadatabase of described metadata management module inquiry, and by the data buffer storage of described metadata dependence table to described metadata dependence library of object;
(b.3) the metadata combination relation table in the metadatabase described in described metadata management module inquiry, and the data buffer storage that described metadata is combined to relation table is to described metadata combination relationship object storehouse.
Further, described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information, comprises the following steps:
(c.1) described metadata query module be take metadata exemplary sequences number for the metadata instance objects storehouse described in described row key word is inquired about by described application programming interfaces, and obtains in metadata instance objects storehouse the data message except other attribute of metadata metadata exemplary sequences number;
(c.2) described metadata query module be take dependence metadata sequence number as described row key word is by the described described metadata dependence library of object of application programming interfaces inquiry, and obtains the data message that relies on metadata sequence number;
(c.3) described metadata query module be take composite unit data sequence number as described row key word is by the described metadata combination relationship object storehouse of described application programming interfaces inquiry, and obtains the data message of composite unit data sequence number.
Further, described metadata management module is saved to described related data information in described NoSQL metadata table, is specially:
Described metadata management module is saved to the data message of the data message of the data message of described other attribute of metadata, described dependence metadata sequence number and described composite unit data sequence number in described NoSQL metadata table.
In addition, the present invention is also a kind of utilizes above-mentioned method to realize the metadata processing method based on NoSQL, and described method comprises the following steps:
(1) the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module;
(2) described communication module is sent to described metadata analysis server by described metadata analysis message;
(3) described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result;
(4) described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result;
(5) described communication module is sent to described metadata analysis client by described analysis result message;
(6) the analysis result message that the described described communication module of metadata analysis client parsing sends obtains described metadata analysis result, and the metadata analysis result described in showing.
Further, described metadata analysis client comprises graphic interface module and task parsing module, the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, comprise the following steps:
(1.1) described graphic interface module display graphics interface;
(1.2) the metadata analysis instruction that described graphic interface module responds user inputs also obtains corresponding metadata analysis task;
(1.3) described graphic interface module is sent to described task parsing module by the data of metadata analysis task corresponding to described metadata analysis instruction;
(1.4) described task parsing module is converted to corresponding metadata analysis message by the data of described metadata analysis task, and described metadata analysis message is sent to described communication module.
Further, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result, comprise the following steps:
(3.1) described task scheduling modules is resolved the metadata analysis message of described communication module transmission and is obtained corresponding metadata analysis task;
(3.2) described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtains described metadata analysis result corresponding to metadata analysis task;
(3.3) described task scheduling modules is institute's correspondence analysis results messages by corresponding metadata analysis results conversion.
Further, described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtain described metadata analysis result corresponding to metadata analysis task, comprise the following steps:
(3.2.1) described task scheduling modules generates corresponding query task according to described query task;
(3.2.2) described task scheduling modules, according to described query task, by the described described NoSQL metadata table of NoSQL database interface inquiry, and obtains the target metadata exemplary sequences number that will analyze;
(3.2.3) described task scheduling modules generates analysis result object according to the data in described NoSQL metadata table, and take described analysis result object as root object establishment analysis result object tree;
(3.2.4) described task scheduling modules reads target-dependent metadata sequence number and the objective cross metadata sequence number in described analysis result object and joins in described query task;
(3.2.5) to take described target metadata exemplary sequences number be row key word to described task scheduling modules, and by the NoSQL metadata table described in described application programming interfaces inquiry;
(3.2.6) described task scheduling modules adds metadata corresponding to described row key word in described analysis result object tree;
(3.2.7) described task scheduling modules is deleted the target metadata exemplary sequences number that has completed inquiry, target-dependent metadata sequence number or objective cross metadata sequence number from query task;
(3.2.8) described task scheduling modules is by remaining target metadata exemplary sequences number in described query task, target-dependent metadata sequence number or objective cross metadata sequence number are described row key word, return to above-mentioned steps (3.2.6), until described query task is empty.
Further, described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result, comprises the following steps:
(4.1) described metadata analysis service end is using described analysis result object tree as described analysis result;
(4.2) described metadata analysis service end encapsulates described analysis result with XML form, and obtains described analysis result message;
(4.3) described metadata analysis service end is sent to described communication module by described analysis result message.
Further, described metadata analysis client comprises graphic interface module and task parsing module, the analysis result message that described metadata analysis client is resolved described communication module transmission obtains described metadata analysis result, and show described metadata analysis result, comprise the following steps:
(6.1) described task parsing module is resolved described metadata analysis result, and described metadata analysis result is sent to described graphic interface module;
(6.2) described graphic interface module shows described metadata analysis result.
Adopted and based on NoSQL, realized the system and method for metadata cache and analysis in this invention, with NoSQL database come Preservation Metadata example and relation information data, do not need to carry out save data with a plurality of tables in relevant database, without the Mapping and Converting instrument that uses Object-Relation, there is not the problem of Impedance Mismatch.Meanwhile, NoSQL metadata table will often need adjacent the depositing of data rows of access simultaneously, the time of having reduced magnetic head tracking, improved the efficiency of magnetic disc i/o, the response time of data query greatly reduces.
In addition, the redundant storage of NoSQL database supported data, by the incidence relation between metadata is kept in metadata example information, can reduce the number of times of data base querying, reduces the resource consumption of database server.Final system framework is simple, has high scalability, can increase as required NoSQL database server; Working service is convenient and swift, stable and reliable working performance, and the scope of application is comparatively extensive, for further developing of enterprise's metadata analysis application established solid foundation.
Accompanying drawing explanation
Fig. 1 is the structural representation of realizing the system of metadata cache and analysis based on NoSQL of the present invention.
Fig. 2 is the process flow diagram of realizing the slow method of analyzing of metadata based on NoSQL of the present invention.
Embodiment
In order more clearly to describe technology contents of the present invention, below in conjunction with specific embodiment, conduct further description.
Refer to Fig. 1, in one embodiment, the system that realizes metadata cache and analysis based on NoSQL of the present invention comprises:
Metadata analysis client, in order to respond the metadata analysis instruction of user's input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and show described metadata analysis result;
Metadata analysis service end, the metadata analysis message sending in order to resolve described communication module, and the metadata analysis task obtaining according to parsing inquiry NoSQL database server obtains metadata analysis result, and the corresponding analysis result message of described metadata analysis result is sent to described communication module;
NoSQL database server, in order to generate NoSQL metadata table, and by the data buffer storage in metadatabase in described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response;
Communication module, in order to described metadata analysis message is sent to described metadata analysis server, and is sent to described metadata analysis client by described analysis result message.
Wherein, NoSQL database server provides the monitoring of NoSQL data-base cluster and management, the inquiry request of response to NoSQL database, availability and the reliability of guarantee NoSQL data-base cluster.Communication module can also provide reliable message stores and monitoring mechanism.
A kind of preferred embodiment in, described metadata analysis client comprises graphic interface module and task parsing module, wherein:
Described graphic interface module, graphic interface in order to display element data analysis, and the metadata analysis instruction of the input of the user described in response, with the data of metadata analysis task corresponding to described metadata analysis instruction are sent to described task parsing module, and show the metadata analysis result that described task parsing module sends;
Described task parsing module, in order to the data of described metadata analysis task are converted to corresponding metadata analysis message, and described metadata analysis message is sent to described communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and described metadata analysis result is sent to described graphic interface module.
A kind of preferred embodiment in, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, wherein:
Described task scheduling modules, in order to resolve the metadata analysis message of described communication module transmission and to obtain corresponding metadata analysis task, and by described NoSQL database interface, inquire about described NoSQL database server according to described metadata analysis task, and obtain described metadata analysis result corresponding to metadata analysis task, and be institute's correspondence analysis results messages by corresponding metadata analysis results conversion;
Described NoSQL database interface, in order to access described NoSQL database server.
In a kind of preferred embodiment, it is characterized in that, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, wherein:
Described metadata management module, in order to generate NoSQL metadata table and NoSQL database, and by the data buffer storage in described metadatabase in described NoSQL database, and obtain related data information by described metadata query module inquiry, and described related data information is saved in described NoSQL metadata table, and the inquiry request of the metadata analysis service end described in response;
Described metadata query module, in order to inquire about described NoSQL database according to row key word by described application programming interfaces, and obtains related data information;
Described application programming interfaces, in order to access the application program of described NoSQL database server.
In a kind of preferred embodiment, described NoSQL database server comprises name module, described name module in order to the NoSQL database server to described register, add, the management of monitoring and fault detect.
In a kind of preferred embodiment, described NoSQL database comprises metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
Wherein, described metadata analysis instruction comprises metadata analysis function type, the information of metadata type and metadata title, described metadata analysis task is consanguinity analysis task, impact analysis task, association analysis task, topological analysis task or index analysis task, described metadata analysis task comprises client the Internet protocol address information, client end slogan information, task sequence information, task names information, task type information and mission bit stream, described metadatabase comprises metadata example table, metadata combination relation table and metadata dependence table.
In addition, the present invention also provides a kind of and utilizes described system to realize the metadata cache method based on NoSQL, and described method is specially:
Described NoSQL database server generates NoSQL metadata table, and by the data buffer storage in metadatabase to described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response.
A kind of preferred embodiment in, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, described NoSQL database server generate NoSQL metadata table and by the data buffer storage in metadatabase to described NoSQL metadata table, comprise the following steps:
(a) described metadata management module generates NoSQL metadata table and NoSQL database;
(b) described metadata management module by the data buffer storage in described metadatabase to described NoSQL database;
(c) described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information;
(d) described metadata management module is saved to described related data information in described NoSQL metadata table.
A kind of preferred embodiment in, described metadata management module generates NoSQL metadata table and NoSQL database, is specially:
Described metadata management module generates NoSQL metadata table, metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
In a kind of preferred embodiment, described metadata management module to described NoSQL database, comprises the following steps the data buffer storage in described metadatabase:
(b.1) the metadata example table in described metadata management module query metadata storehouse, and by the data buffer storage of described metadata example table to described metadata instance objects storehouse;
(b.2) the metadata dependence table in the described metadatabase of described metadata management module inquiry, and by the data buffer storage of described metadata dependence table to described metadata dependence library of object;
(b.3) the metadata combination relation table in the metadatabase described in described metadata management module inquiry, and the data buffer storage that described metadata is combined to relation table is to described metadata combination relationship object storehouse.
In a kind of preferred embodiment, described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information, comprises the following steps:
(c.1) described metadata query module be take metadata exemplary sequences number for the metadata instance objects storehouse described in described row key word is inquired about by described application programming interfaces, and obtains in metadata instance objects storehouse the data message except other attribute of metadata metadata exemplary sequences number;
(c.2) described metadata query module be take dependence metadata sequence number as described row key word is by the described described metadata dependence library of object of application programming interfaces inquiry, and obtains the data message that relies on metadata sequence number;
(c.3) described metadata query module be take composite unit data sequence number as described row key word is by the described metadata combination relationship object storehouse of described application programming interfaces inquiry, and obtains the data message of composite unit data sequence number.
In a kind of preferred embodiment, described metadata management module is saved to described related data information in described NoSQL metadata table, is specially:
Described metadata management module is saved to the data message of the data message of the data message of described other attribute of metadata, described dependence metadata sequence number and described composite unit data sequence number in described NoSQL metadata table.
In addition, the present invention is also a kind of utilizes above-mentioned method to realize the metadata processing method based on NoSQL, and as shown in Figure 2, described method comprises the following steps:
(1) the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module;
(2) described communication module is sent to described metadata analysis server by described metadata analysis message;
(3) described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result;
(4) described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result;
(5) described communication module is sent to described metadata analysis client by described analysis result message;
(6) the analysis result message that the described described communication module of metadata analysis client parsing sends obtains described metadata analysis result, and the metadata analysis result described in showing.
A kind of preferred embodiment in, described metadata analysis client comprises graphic interface module and task parsing module, the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, comprise the following steps:
(1.1) described graphic interface module display graphics interface;
(1.2) the metadata analysis instruction that described graphic interface module responds user inputs also obtains corresponding metadata analysis task;
(1.3) described graphic interface module is sent to described task parsing module by the data of metadata analysis task corresponding to described metadata analysis instruction;
(1.4) described task parsing module is converted to corresponding metadata analysis message by the data of described metadata analysis task, and described metadata analysis message is sent to described communication module.
A kind of preferred embodiment in, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result, comprise the following steps:
(3.1) described task scheduling modules is resolved the metadata analysis message of described communication module transmission and is obtained corresponding metadata analysis task;
(3.2) described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtains described metadata analysis result corresponding to metadata analysis task;
(3.3) described task scheduling modules is institute's correspondence analysis results messages by corresponding metadata analysis results conversion.
In a kind of preferred embodiment, described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtain described metadata analysis result corresponding to metadata analysis task, comprise the following steps:
(3.2.1) described task scheduling modules generates corresponding query task according to described query task;
(3.2.2) described task scheduling modules, according to described query task, by the described described NoSQL metadata table of NoSQL database interface inquiry, and obtains the target metadata exemplary sequences number that will analyze;
(3.2.3) described task scheduling modules generates analysis result object according to the data in described NoSQL metadata table, and take described analysis result object as root object establishment analysis result object tree;
(3.2.4) described task scheduling modules reads target-dependent metadata sequence number and the objective cross metadata sequence number in described analysis result object and joins in described query task;
(3.2.5) to take described target metadata exemplary sequences number be row key word to described task scheduling modules, and by the NoSQL metadata table described in described application programming interfaces inquiry;
(3.2.6) described task scheduling modules adds metadata corresponding to described row key word in described analysis result object tree;
(3.2.7) described task scheduling modules is deleted the target metadata exemplary sequences number that has completed inquiry, target-dependent metadata sequence number or objective cross metadata sequence number from query task;
(3.2.8) described task scheduling modules is by remaining target metadata exemplary sequences number in described query task, target-dependent metadata sequence number or objective cross metadata sequence number are described row key word, return to above-mentioned steps (3.2.6), until described query task is empty.
In a kind of preferred embodiment, described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result, comprises the following steps:
(4.1) described metadata analysis service end is using described analysis result object tree as described analysis result;
(4.2) described metadata analysis service end encapsulates described analysis result with XML form, and obtains described analysis result message;
(4.3) described metadata analysis service end is sent to described communication module by described analysis result message.
A kind of preferred embodiment in, described metadata analysis client comprises graphic interface module and task parsing module, the analysis result message that described metadata analysis client is resolved described communication module transmission obtains described metadata analysis result, and show described metadata analysis result, comprise the following steps:
(6.1) described task parsing module is resolved described metadata analysis result, and described metadata analysis result is sent to described graphic interface module;
(6.2) described graphic interface module shows described metadata analysis result.
In a preferred specific embodiment, the method that realizes metadata cache and analysis based on NoSQL of the present invention comprises in fact two operations: metadata import operation (being metadata cache) and metadata analysis are processed operation.Data importing operation imports to the metadata being kept in relevant database in NoSQL database and carries out corresponding data-switching.Metadata analysis is processed and is operated in to carry out inquiry in the metadata of NoSQL database caches, carries out relevant metadata analysis, and operation is as follows in detail:
First, the step of metadata import operation is as follows:
1, create metadata instance objects storehouse, wherein, the field of the Property Name of object, attribute type and metadata example table is corresponding one by one;
2, query metadata example table, is saved in the value of metadatabase record in metadata instance objects storehouse;
Wherein, the form in metadata instance objects storehouse is: metadata example ID (sequence number), metadata title, metadata type, metadata attributes 1, metadata attributes 2, metadata attributes 3
3, call the API (application programming interfaces) that NoSQL database server provides, in NoSQL database server, create NoSQL metadata table, for depositing metadata example and relation data;
Wherein, NoSQL metadata tableau format is: row key word, and metadata example row family, dependence row family, syntagmatic row family, the data in row family comprise the value of a plurality of versions, the corresponding timestamp of each value;
In metadata example row family, comprise a plurality of row such as metadata title, metadata type, metadata attributes 1, metadata attributes 2, metadata attributes 3; Dependence row family comprises row of dependence metadata ID; Syntagmatic row family comprises row of composite unit data ID;
4, call the API that NoSQL database server provides, take metadata example ID as row key word, other attribute except metadata example ID in metadata instance objects storehouse is all saved in NoSQL database instance;
5, create metadata dependence library of object, wherein, the field of the Property Name of object, attribute type and metadata dependence table is corresponding one by one;
6, query metadata dependence table, is saved in the value of metadatabase record in metadata dependence library of object;
Wherein, metadata dependence library of object form is: metadata example ID, relies on metadata ID;
7, call the API that NoSQL database server provides, the metadata example ID of take in dependence object is inquired about in NoSQL database as row key word, by relying on metadata ID, is updated in the dependence metadata ID row in dependence row family;
8, create metadata combination relationship object storehouse, wherein, the field of the Property Name of object, attribute type and metadata combination relation table is corresponding one by one;
9, query metadata syntagmatic table, is saved in the value of metadatabase record in metadata combination relationship object storehouse;
Wherein, metadata combination relationship object library format is: metadata example ID, composite unit data ID;
10, call the API that NoSQL database server provides, the metadata example ID of take in syntagmatic library of object is inquired about in NoSQL database as row key word, and composite unit data ID is updated in the composite unit data ID row in syntagmatic row family.
Secondly, suppose to have a plurality of metadata analysis clients and metadata analysis service end, the step that metadata analysis is processed operation is as follows:
1, communication module initialization
11, read configuration file, obtain a plurality of analysis client ids and Analysis Service end ID;
12, according to analyzing client id, for each metadata analysis client, set up the queue of message sending and receiving;
13, according to Analysis Service end ID, for each metadata analysis service end, set up the queue of message sending and receiving;
14, watcher thread starts, the arrival waiting for the arrival of news;
2, NoSQL database server carries out initialization;
21, start naming service module in NoSQL database server;
22, start metadata management module in NoSQL database server;
23, start NoSQL metadatabase example in NoSQL database server;
24, after NoSQL metadata table starts successfully, to the name module information that reports in;
25, the information that metadata query module must all available NoSQL database instances;
26, metadata management module is communicated by letter with available NoSQL database instance, sets up the database table of NoSQL database instance and storage and the corresponding relation of data field;
27, metadata management module is inquired about all data field information, calculates unappropriated data field, and they are added to data field table to be allocated;
18, metadata management module is inquired about available NoSQL database instance information, unappropriated data separation dispensing NoSQL database instance is stored, and preserved corresponding relation;
3, metadata analysis service end is carried out initialization;
31, to communication module, send test post, if receive response, illustrate normally with being connected of communication module, confiscate to respond and provide miscue;
32, to the name module in NoSQL database server, send test post, confirm the normally operation of NoSQL data-base cluster;
33, the task scheduling thread of initiating task scheduler module, waits for the submission of metadata analysis task;
4, user submits to the data that will analyze to metadata analysis client;
41, the metadata analysis function that user is chosen metadata node and determined to carry out by graphic interface;
Wherein, metadata analysis function type comprises: consanguinity analysis, impact analysis, association analysis, topological analysis, index analysis etc.;
42, graphic interface module receives user's action and submits to related data to metadata analysis client;
Wherein, the data layout that graphic interface module is submitted to is: metadata analysis function type, metadata type, metadata title;
5, metadata analysis client is submitted to metadata analysis service end by analysis data and is analyzed;
51, the data that metadata analysis client sends to graphic interface module;
52, metadata analysis client encapsulates data in metadata analysis task;
Wherein, the form of metadata analysis task is: client ip address, client end slogan, task ID, task names, task type, mission bit stream
53, metadata analysis client sends to metadata analysis service end by metadata analysis message by communication module;
6, metadata analysis service end is carried out metadata analysis method, obtains analysis result;
61, the metadata analysis message that the task scheduling modules received communication module of metadata analysis service end is sent, starts new analysis thread execution metadata analysis task;
62, the parameter in metadata analysis service end analytical element data analysis message, obtains the metadata analysis task that will analyze;
Wherein, the form of metadata analysis message is: metadata analysis function type, metadata type, metadata title;
63, metadata analysis service end is used different query strategies according to the type of metadata analysis function, generates different query tasks;
Wherein, because the type of metadata analysis function mainly comprises: consanguinity analysis (blood lineage's analysis), impact analysis, entity associated analysis, entity impact analysis, main frame topological analysis, indicator consilience analysis etc., therefore corresponding query strategy comprises: consanguinity analysis strategy, impact analysis strategy, association analysis strategy, topological analysis strategy, consistency analysis strategy etc.;
64, metadata analysis service end is according to the query task generating, data in the inquiry API Access NoSQL metadata table that use NoSQL database server provides, the metadata example ID that acquisition will be analyzed, wherein, described inquiry API refers to the API of data query in the performing database that NoSQL provides;
65, metadata analysis service end obtains after metadata example ID, to in NoSQL database, take metadata example ID as row key word, in metadata example row family, dependence row family and syntagmatic row family, the value of the latest edition of each row generates analysis result object, and take this object as root object establishment analysis result object tree;
66, metadata analysis service end directly reads dependence metadata ID and the syntagmatic metadata ID in analysis result object and joins in query task;
67, metadata analysis service end reads the concentrated metadata ID of query task, the inquiry API that calling NoSQL database server provides inquires about the metadata record that the metadata ID of take in task-set is row key word and joins in analysis result object tree in metadata table, take that to have the metadata of mutual relationship be parent object;
68, metadata analysis service end directly reads dependence metadata ID and the syntagmatic metadata ID of metadata record in Query Result and joins query task and concentrates;
69, from query task, concentrate deletion to complete the metadata example ID of inquiry;
610, using the concentrated metadata ID of query task as row key word, carry out query steps above, until query task collection be sky
7, metadata analysis service end returns to analysis result to metadata analysis client;
71, metadata analysis service end encapsulates analysis result object tree with XML form, forms analysis result message content;
72, analysis result message is sent to metadata analysis client by communication module;
8, metadata analysis client represents;
81, metadata analysis client, from communication module receipt message, sends to graphic interface module by analysis result object tree after parsing;
82, graphic interface module represents analysis result object tree with graphical form;
Adopted and based on NoSQL, realized the system and method for metadata cache and analysis in this invention, with NoSQL database come Preservation Metadata example and relation information data, do not need to carry out save data with a plurality of tables in relevant database, without the Mapping and Converting instrument that uses Object-Relation, there is not the problem of Impedance Mismatch.Meanwhile, NoSQL metadata table will often need adjacent the depositing of data rows of access simultaneously, the time of having reduced magnetic head tracking, improved the efficiency of magnetic disc i/o, the response time of data query greatly reduces.
In addition, the redundant storage of NoSQL database supported data, by the incidence relation between metadata is kept in metadata example information, can reduce the number of times of data base querying, reduces the resource consumption of database server.Final system framework is simple, has high scalability, can increase as required NoSQL database server; Working service is convenient and swift, stable and reliable working performance, and the scope of application is comparatively extensive, for further developing of enterprise's metadata analysis application established solid foundation.
In this instructions, the present invention is described with reference to its specific embodiment.But, still can make various modifications and conversion obviously and not deviate from the spirit and scope of the present invention.Therefore, instructions and accompanying drawing are regarded in an illustrative, rather than a restrictive.
Claims (20)
1. based on NoSQL, realize a system for metadata cache and analysis, it is characterized in that, described system comprises:
Metadata analysis client, in order to respond the metadata analysis instruction of user's input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and show described metadata analysis result;
Metadata analysis service end, the metadata analysis message sending in order to resolve described communication module, and the metadata analysis task obtaining according to parsing inquiry NoSQL database server obtains metadata analysis result, and the corresponding analysis result message of described metadata analysis result is sent to described communication module;
NoSQL database server, in order to generate NoSQL metadata table, and by the data buffer storage in metadatabase in described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response;
Communication module, in order to described metadata analysis message is sent to described metadata analysis server, and is sent to described metadata analysis client by described analysis result message.
2. the system that realizes metadata cache and analysis based on NoSQL according to claim 1, is characterized in that, described metadata analysis client comprises graphic interface module and task parsing module, wherein:
Described graphic interface module, graphic interface in order to display element data analysis, and the metadata analysis instruction of the input of the user described in response, with the data of metadata analysis task corresponding to described metadata analysis instruction are sent to described task parsing module, and show the metadata analysis result that described task parsing module sends;
Described task parsing module, in order to the data of described metadata analysis task are converted to corresponding metadata analysis message, and described metadata analysis message is sent to described communication module, and resolve the analysis result message that described communication module sends and obtain metadata analysis result, and described metadata analysis result is sent to described graphic interface module.
3. the system that realizes metadata cache and analysis based on NoSQL according to claim 1, is characterized in that, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, wherein:
Described task scheduling modules, in order to resolve the metadata analysis message of described communication module transmission and to obtain corresponding metadata analysis task, and by described NoSQL database interface, inquire about described NoSQL database server according to described metadata analysis task, and obtain described metadata analysis result corresponding to metadata analysis task, and be institute's correspondence analysis results messages by corresponding metadata analysis results conversion;
Described NoSQL database interface, in order to access described NoSQL database server.
4. according to realizing the system of metadata cache and analysis based on NoSQL described in any one in claims 1 to 3, it is characterized in that, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, wherein:
Described metadata management module, in order to generate NoSQL metadata table and NoSQL database, and by the data buffer storage in described metadatabase in described NoSQL database, and obtain related data information by described metadata query module inquiry, and described related data information is saved in described NoSQL metadata table, and the inquiry request of the metadata analysis service end described in response;
Described metadata query module, in order to inquire about described NoSQL database according to row key word by described application programming interfaces, and obtains related data information;
Described application programming interfaces, in order to access the application program of described NoSQL database server.
5. the system that realizes metadata cache and analysis based on NoSQL according to claim 4, it is characterized in that, described NoSQL database server comprises name module, described name module in order to the NoSQL database server to described register, add, the management of monitoring and fault detect.
6. the system that realizes metadata cache and analysis based on NoSQL according to claim 4, is characterized in that, described NoSQL database comprises metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
7. the system that realizes metadata cache and analysis based on NoSQL according to claim 1, it is characterized in that, described metadata analysis instruction comprises metadata analysis function type, the information of metadata type and metadata title, described metadata analysis task is consanguinity analysis task, impact analysis task, association analysis task, topological analysis task or index analysis task, described metadata analysis task comprises client the Internet protocol address information, client end slogan information, task sequence information, task names information, task type information and mission bit stream.
8. the system that realizes metadata cache and analysis based on NoSQL according to claim 1, is characterized in that, described metadatabase comprises metadata example table, metadata combination relation table and metadata dependence table.
9. utilize the system described in claim 1 to realize the metadata cache method based on NoSQL, it is characterized in that, described method is specially:
Described NoSQL database server generates NoSQL metadata table, and by the data buffer storage in metadatabase to described NoSQL metadata table, and the inquiry request of the described metadata analysis service end of response.
10. the metadata cache method of realization according to claim 9 based on NoSQL, it is characterized in that, described NoSQL database server comprises metadata management module, metadata query module and application programming interfaces, described NoSQL database server generate NoSQL metadata table and by the data buffer storage in metadatabase to described NoSQL metadata table, comprise the following steps:
(a) described metadata management module generates NoSQL metadata table and NoSQL database;
(b) described metadata management module by the data buffer storage in described metadatabase to described NoSQL database;
(c) described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information;
(d) described metadata management module is saved to described related data information in described NoSQL metadata table.
The metadata cache method of 11. realizations according to claim 10 based on NoSQL, is characterized in that, described metadata management module generates NoSQL metadata table and NoSQL database, is specially:
Described metadata management module generates NoSQL metadata table, metadata instance objects storehouse, metadata dependence library of object and metadata combination relationship object storehouse.
The metadata cache method of 12. realizations according to claim 11 based on NoSQL, is characterized in that, described metadata management module to described NoSQL database, comprises the following steps the data buffer storage in described metadatabase:
(b.1) the metadata example table in described metadata management module query metadata storehouse, and by the data buffer storage of described metadata example table to described metadata instance objects storehouse;
(b.2) the metadata dependence table in the described metadatabase of described metadata management module inquiry, and by the data buffer storage of described metadata dependence table to described metadata dependence library of object;
(b.3) the metadata combination relation table in the metadatabase described in described metadata management module inquiry, and the data buffer storage that described metadata is combined to relation table is to described metadata combination relationship object storehouse.
The metadata cache method of 13. realizations according to claim 12 based on NoSQL, it is characterized in that, described metadata query module is inquired about described NoSQL database according to described row key word by described application programming interfaces, and obtains related data information, comprises the following steps:
(c.1) described metadata query module be take metadata exemplary sequences number for the metadata instance objects storehouse described in described row key word is inquired about by described application programming interfaces, and obtains in metadata instance objects storehouse the data message except other attribute of metadata metadata exemplary sequences number;
(c.2) described metadata query module be take dependence metadata sequence number as described row key word is by the described described metadata dependence library of object of application programming interfaces inquiry, and obtains the data message that relies on metadata sequence number;
(c.3) described metadata query module be take composite unit data sequence number as described row key word is by the described metadata combination relationship object storehouse of described application programming interfaces inquiry, and obtains the data message of composite unit data sequence number.
The metadata cache method of 14. realizations according to claim 13 based on NoSQL, is characterized in that, described metadata management module is saved to described related data information in described NoSQL metadata table, is specially:
Described metadata management module is saved to the data message of the data message of the data message of described other attribute of metadata, described dependence metadata sequence number and described composite unit data sequence number in described NoSQL metadata table.
15. 1 kinds are utilized the method described in any one in claim 10 to 14 to realize the metadata processing method based on NoSQL, it is characterized in that, described method comprises the following steps:
(1) the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module;
(2) described communication module is sent to described metadata analysis server by described metadata analysis message;
(3) described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result;
(4) described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result;
(5) described communication module is sent to described metadata analysis client by described analysis result message;
(6) the analysis result message that the described described communication module of metadata analysis client parsing sends obtains described metadata analysis result, and the metadata analysis result described in showing.
The metadata analysis method of 16. realizations according to claim 15 based on NoSQL, it is characterized in that, described metadata analysis client comprises graphic interface module and task parsing module, the metadata analysis instruction of described metadata analysis client end response user input, and metadata analysis message corresponding to described metadata analysis instruction is sent to communication module, comprise the following steps:
(1.1) described graphic interface module display graphics interface;
(1.2) the metadata analysis instruction that described graphic interface module responds user inputs also obtains corresponding metadata analysis task;
(1.3) described graphic interface module is sent to described task parsing module by the data of metadata analysis task corresponding to described metadata analysis instruction;
(1.4) described task parsing module is converted to corresponding metadata analysis message by the data of described metadata analysis task, and described metadata analysis message is sent to described communication module.
The metadata analysis method of 17. realizations according to claim 15 based on NoSQL, it is characterized in that, described metadata analysis service end comprises task scheduling modules and NoSQL database interface, described metadata analysis service end is resolved described metadata analysis message, and the described NoSQL database server of the metadata analysis task obtaining according to parsing inquiry obtains metadata analysis result, comprise the following steps:
(3.1) described task scheduling modules is resolved the metadata analysis message of described communication module transmission and is obtained corresponding metadata analysis task;
(3.2) described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtains described metadata analysis result corresponding to metadata analysis task;
(3.3) described task scheduling modules is institute's correspondence analysis results messages by corresponding metadata analysis results conversion.
The metadata analysis method of 18. realizations according to claim 17 based on NoSQL, it is characterized in that, described task scheduling modules is inquired about described NoSQL database server according to described metadata analysis task by described NoSQL database interface, and obtain described metadata analysis result corresponding to metadata analysis task, comprise the following steps:
(3.2.1) described task scheduling modules generates corresponding query task according to described query task;
(3.2.2) described task scheduling modules, according to described query task, by the described described NoSQL metadata table of NoSQL database interface inquiry, and obtains the target metadata exemplary sequences number that will analyze;
(3.2.3) described task scheduling modules generates analysis result object according to the data in described NoSQL metadata table, and take described analysis result object as root object establishment analysis result object tree;
(3.2.4) described task scheduling modules reads target-dependent metadata sequence number and the objective cross metadata sequence number in described analysis result object and joins in described query task;
(3.2.5) to take described target metadata exemplary sequences number be row key word to described task scheduling modules, and by the NoSQL metadata table described in described application programming interfaces inquiry;
(3.2.6) described task scheduling modules adds metadata corresponding to described row key word in described analysis result object tree;
(3.2.7) described task scheduling modules is deleted the target metadata exemplary sequences number that has completed inquiry, target-dependent metadata sequence number or objective cross metadata sequence number from query task;
(3.2.8) described task scheduling modules is by remaining target metadata exemplary sequences number in described query task, target-dependent metadata sequence number or objective cross metadata sequence number are described row key word, return to above-mentioned steps (3.2.6), until described query task is empty.
The metadata analysis method of 19. realizations according to claim 18 based on NoSQL, it is characterized in that, described metadata analysis service end is sent to described communication module by the corresponding analysis result message of described metadata analysis result, comprises the following steps:
(4.1) described metadata analysis service end is using described analysis result object tree as described analysis result;
(4.2) described metadata analysis service end encapsulates described analysis result with XML form, and obtains described analysis result message;
(4.3) described metadata analysis service end is sent to described communication module by described analysis result message.
The metadata analysis method of 20. realizations according to claim 15 based on NoSQL, it is characterized in that, described metadata analysis client comprises graphic interface module and task parsing module, the analysis result message that described metadata analysis client is resolved described communication module transmission obtains described metadata analysis result, and show described metadata analysis result, comprise the following steps:
(6.1) described task parsing module is resolved described metadata analysis result, and described metadata analysis result is sent to described graphic interface module;
(6.2) described graphic interface module shows described metadata analysis result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410493443.9A CN104199978A (en) | 2014-09-24 | 2014-09-24 | System and method for realizing metadata cache and analysis based on NoSQL and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410493443.9A CN104199978A (en) | 2014-09-24 | 2014-09-24 | System and method for realizing metadata cache and analysis based on NoSQL and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104199978A true CN104199978A (en) | 2014-12-10 |
Family
ID=52085271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410493443.9A Pending CN104199978A (en) | 2014-09-24 | 2014-09-24 | System and method for realizing metadata cache and analysis based on NoSQL and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104199978A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794003A (en) * | 2015-02-04 | 2015-07-22 | 汉鼎信息科技股份有限公司 | Large data analysis system integrating real-time mode and non-real-time mode |
| CN105787051A (en) * | 2016-02-26 | 2016-07-20 | 广州品唯软件有限公司 | Analysis method and device based on metadata model |
| WO2016192583A1 (en) * | 2015-06-04 | 2016-12-08 | 阿里巴巴集团控股有限公司 | Data processing method and device for data warehouse |
| CN109857924A (en) * | 2019-02-28 | 2019-06-07 | 重庆科技学院 | A kind of big data analysis monitor information processing system and method |
| CN110019252A (en) * | 2019-04-16 | 2019-07-16 | 成都四方伟业软件股份有限公司 | The method, apparatus and electronic equipment of information processing |
| CN111680030A (en) * | 2019-03-11 | 2020-09-18 | 阿里巴巴集团控股有限公司 | Data fusion method and device, data processing method and device based on meta-information |
| CN112699100A (en) * | 2020-12-31 | 2021-04-23 | 天津浪淘科技股份有限公司 | Management and analysis system based on metadata |
| CN112818015A (en) * | 2021-01-21 | 2021-05-18 | 广州汇通国信科技有限公司 | Data tracking method, system and storage medium based on data blood margin analysis |
| CN114741395A (en) * | 2022-04-25 | 2022-07-12 | 北京海量数据技术股份有限公司 | Method for realizing event trigger in OpenGauss database |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8195610B1 (en) * | 2007-05-08 | 2012-06-05 | IdeaBlade, Inc. | Method and apparatus for cache management of distributed objects |
| CN103019960A (en) * | 2012-12-03 | 2013-04-03 | 华为技术有限公司 | Distributed cache method and system |
| CN103646111A (en) * | 2013-12-25 | 2014-03-19 | 普元信息技术股份有限公司 | System and method for realizing real-time data association in big data environment |
| CN103853714A (en) * | 2012-11-28 | 2014-06-11 | 中国移动通信集团河南有限公司 | Data processing method and device |
-
2014
- 2014-09-24 CN CN201410493443.9A patent/CN104199978A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8195610B1 (en) * | 2007-05-08 | 2012-06-05 | IdeaBlade, Inc. | Method and apparatus for cache management of distributed objects |
| CN103853714A (en) * | 2012-11-28 | 2014-06-11 | 中国移动通信集团河南有限公司 | Data processing method and device |
| CN103019960A (en) * | 2012-12-03 | 2013-04-03 | 华为技术有限公司 | Distributed cache method and system |
| CN103646111A (en) * | 2013-12-25 | 2014-03-19 | 普元信息技术股份有限公司 | System and method for realizing real-time data association in big data environment |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794003A (en) * | 2015-02-04 | 2015-07-22 | 汉鼎信息科技股份有限公司 | Large data analysis system integrating real-time mode and non-real-time mode |
| CN104794003B (en) * | 2015-02-04 | 2019-06-04 | 汉鼎宇佑互联网股份有限公司 | It is a kind of to integrate real-time and non-real-time mode big data analysis system |
| WO2016192583A1 (en) * | 2015-06-04 | 2016-12-08 | 阿里巴巴集团控股有限公司 | Data processing method and device for data warehouse |
| CN105787051A (en) * | 2016-02-26 | 2016-07-20 | 广州品唯软件有限公司 | Analysis method and device based on metadata model |
| CN109857924A (en) * | 2019-02-28 | 2019-06-07 | 重庆科技学院 | A kind of big data analysis monitor information processing system and method |
| CN111680030A (en) * | 2019-03-11 | 2020-09-18 | 阿里巴巴集团控股有限公司 | Data fusion method and device, data processing method and device based on meta-information |
| CN111680030B (en) * | 2019-03-11 | 2025-02-25 | 阿里巴巴集团控股有限公司 | Data fusion method and device, data processing method and device based on metadata |
| CN110019252A (en) * | 2019-04-16 | 2019-07-16 | 成都四方伟业软件股份有限公司 | The method, apparatus and electronic equipment of information processing |
| CN112699100A (en) * | 2020-12-31 | 2021-04-23 | 天津浪淘科技股份有限公司 | Management and analysis system based on metadata |
| CN112818015A (en) * | 2021-01-21 | 2021-05-18 | 广州汇通国信科技有限公司 | Data tracking method, system and storage medium based on data blood margin analysis |
| CN114741395A (en) * | 2022-04-25 | 2022-07-12 | 北京海量数据技术股份有限公司 | Method for realizing event trigger in OpenGauss database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10831753B2 (en) | Query plan generation and execution in a relational database management system with a temporal-relational database | |
| CN104199978A (en) | System and method for realizing metadata cache and analysis based on NoSQL and method | |
| JP6617117B2 (en) | Scalable analysis platform for semi-structured data | |
| CN107273506B (en) | A method for joint query of multiple tables in a database | |
| US20200089666A1 (en) | Secure data isolation in a multi-tenant historization system | |
| EP2874077B1 (en) | Stateless database cache | |
| CN104516979B (en) | A kind of data query method and system based on quadratic search | |
| US20140074771A1 (en) | Query optimization | |
| US20120158655A1 (en) | Non-relational function-based data publication for relational data | |
| JP2016519810A (en) | Scalable analysis platform for semi-structured data | |
| CN104239572A (en) | System and method for achieving metadata analysis based on distributed cache | |
| CN104123374A (en) | Method and device for aggregate query in distributed databases | |
| CN103430144A (en) | Data source analytics | |
| CN107066546B (en) | MPP engine-based cross-data center quick query method and system | |
| CN112416991A (en) | Data processing method and device and storage medium | |
| CN111752920A (en) | Method, system and storage medium for managing metadata | |
| CN111897867A (en) | Database log statistical method, system and related device | |
| CN105550351B (en) | The extemporaneous inquiry system of passenger's run-length data and method | |
| US20220391368A1 (en) | Cryptography system for using associated values stored in different locations to encode and decode data | |
| CN115168361A (en) | Label management method and device | |
| US20150317330A1 (en) | Storing data to multiple storage location types in a distributed historization system | |
| US11620284B2 (en) | Backend data aggregation system and method | |
| CN110647518B (en) | Data source fusion calculation method, component and device | |
| CN107220363A (en) | It is a kind of to support the global complicated cross-region querying method retrieved and system | |
| CN106156338A (en) | The date storage method of a kind of INFORMATION DISCOVERY server and INFORMATION DISCOVERY method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141210 |
|
| WD01 | Invention patent application deemed withdrawn after publication |