CN105279281A - Internet-of-things data access method - Google Patents
Internet-of-things data access method Download PDFInfo
- Publication number
- CN105279281A CN105279281A CN201510787124.3A CN201510787124A CN105279281A CN 105279281 A CN105279281 A CN 105279281A CN 201510787124 A CN201510787124 A CN 201510787124A CN 105279281 A CN105279281 A CN 105279281A
- Authority
- CN
- China
- Prior art keywords
- internet
- module
- access method
- data access
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000001914 filtration Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 abstract description 2
- 238000013500 data storage Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/235—Update request formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An Internet-of-things data access method relates to a data access method, and particularly to an Internet-of-things data access method which supports high-frequency updating and complicated retrieval. A device according to the Internet-of-things data access method comprises the components of a data storing and importing module, namely an Importer module, a Mysql database, a Hbase database and a Query module; wherein the Importer module performs functions of analyzing all fields in data storage, extracting retrieved fields and presented fields, storing the retrieved fields in the Mysql database, and storing the presented fields into the Hbase database. The Query module performs morphology analysis and grammar analysis on a retrieved sentence, and furthermore reads related information from the Mysqul database and the Hbase database according to an analysis result. The Internet-of-things data access method comprises a storage operation, an updating operation and a querying operation. The Internet-of-things data access method can well settle problems in storage, retrieval and updating of current state information of an Internet-of-things terminal.
Description
Technical field
What a kind of Internet of Things of the present invention data access method related to is especially a kind of Internet of Things data access method supporting that high frequency upgrades and complexity is retrieved of a kind of data access method.
Background technology
Along with the flourish of technology of Internet of things and widespread use, the current state information of internet-of-things terminal has that field is many, individual fields upgrades frequent, the characteristic of querying condition complexity, and this is a very large challenge to the storage of internet-of-things terminal current state information.
Traditional method will support that complex query can adopt relevant database, that conventional is exactly the Relational DBMS Mysql increased income, it can support complicated inquiry, also the search efficiency of data can be improved by setting up index, but under Data Update ratio more frequently situation, the poor performance of Mysql, even if adopt memory table, because memory table is table lock, its performance still cannot improve.
Support that the renewal of high frequency can select the database Hbase of high-performance, high reliability, because Hbase is distributed database, read-write pressure can be distributed to each back end, so under read-write operation frequently situation, higher efficiency can be ensured, but owing to being Nosql database, so complicated inquiry cannot be supported.
Summary of the invention
The object of the invention is to provide a kind of Internet of Things data access method for above-mentioned weak point, high frequency can be supported to upgrade and the complicated Internet of Things data access retrieved.Based on the Internet of Things data access method with high frequency renewal and complicated retrieval characteristic of Mysql, Hbase, the method utilizes Mysql support complex query and Hbase to support that the characteristic of high-speed read-write can well solve the storage of the current state information of internet-of-things terminal, retrieval and a renewal difficult problem.
A kind of Internet of Things data access method takes following technical scheme to realize:
Two class fields are had in the current state information of internet-of-things terminal, one class is search field, these fields are often used for retrieving information as search condition, wherein Termination ID is fixed as search field, one class represents field, these fields are used for showing on interface substantially, also can be used for once in a while retrieving.
A kind of Internet of Things data access method comprises data and stores importing module and Importer module, Mysql database, Hbase database and inquiry Query module; Wherein Importer module is analyzed all fields when data store and is extracted search field and represent field, by search field stored in Mysql database, will represent field stored in Hbase; Inquiry Query module in charge carries out lexical analysis and grammatical analysis to query statement, and reads relevant information according in analysis result to Mysql and Hbase.
A kind of Internet of Things data access method comprises storage operation, renewal rewards theory and query manipulation;
Described storage operation comprises the steps:
1) Importer module reads data to be stored;
2) Importer module carries out field analysis to the data to be stored described in step 1), obtains search field and represents field;
3) Importer module is by step 2) described in search field be stored in Mysql database;
4) Importer module is by step 2) described in the field that represents be that line unit rowkey is stored into Hbase database with Termination ID, described Hbase database and non-relational distributed data base.
Described renewal rewards theory comprises the steps:
1) Importer module reads data to be updated;
2) Importer module analyzes search field and represents field from the data to be updated described in step 1);
3) search field is updated in Mysql database by Importer module;
4) Importer module will represent field and will be updated in Hbase database.
Described query manipulation, comprises the steps:
1), after Query module receives the inquiry request of user, lexical analysis is carried out to described inquiry request, and grammatical analysis;
2) Query module is inquired about in Mysql database according to the search condition analyzed in step 1); If Query Result is sky, terminate inquiry, and Query Result is set for empty.
3) Query module is with step 2) in the Termination ID that inquires be good in Hbase database for row and read data; If Query Result is sky, terminate inquiry, and Query Result is set for empty.
4) Query module carries out secondary filtration according to the search condition analyzed in step 1) to the data in Hbase database, and returns the data after filtration.
Inquiry request is particularly resolved into a series of word symbol according to word-building rule by lexical analysis by the lexical analysis described in step 1) and grammatical analysis, then, by syntax analyzer, described word symbol is built into parse tree, and do syntax check and grammatical analysis, finally obtain search condition and search field;
Described word symbol comprises key word, indications and operational symbol.
The invention has the beneficial effects as follows: the Internet of Things data access method of a kind of efficient retrieval based on Mysql, Hbase system of the present invention, can support that read-write upgrades database manipulation scene frequently efficiently on the one hand, the scene of complicated query manipulation can be supported on the one hand.
Accompanying drawing explanation
Below with reference to accompanying drawing, the invention will be further described:
Fig. 1 is the allomeric function schematic diagram of Internet of Things data access method of the present invention;
Fig. 2 is the storage operational flowchart of Internet of Things data access method of the present invention;
Fig. 3 is the accessing operation process flow diagram of Internet of Things data access method of the present invention;
Fig. 4 is the storage schematic diagram of Internet of Things data access method of the present invention.
Embodiment
With reference to accompanying drawing 1 ~ 3, a kind of Internet of Things data access method comprises data and stores importing module and Importer module, Mysql database, Hbase database and inquiry Query module; Wherein Importer module is analyzed all fields when data store and is extracted search field and represent field, by search field stored in Mysql database, will represent field stored in Hbase; Inquiry Query module in charge carries out lexical analysis and grammatical analysis to query statement, and reads relevant information according in analysis result to Mysql and Hbase.
A kind of Internet of Things data access method comprises storage operation, renewal rewards theory and query manipulation;
Described storage operation comprises the steps:
1) Importer module reads data to be stored;
2) Importer module carries out field analysis to the data to be stored described in step 1), obtains search field and represents field;
3) Importer module is by step 2) described in search field be stored in Mysql database;
4) Importer module is by step 2) described in the field that represents be that line unit rowkey is stored into Hbase database with Termination ID, described Hbase database and non-relational distributed data base.
Described renewal rewards theory comprises the steps:
1) Importer module reads data to be updated;
2) Importer module analyzes search field and represents field from the data to be updated described in step 1);
3) search field is updated in Mysql database by Importer module;
4) Importer module will represent field and will be updated in Hbase database.
1), after Query module receives the inquiry request of user, lexical analysis is carried out to described inquiry request, and grammatical analysis;
2) Query module is inquired about in Mysql database according to the search condition analyzed in step 1); If Query Result is sky, terminate inquiry, and Query Result is set for empty.
3) Query module is with step 2) in the Termination ID that inquires be good in Hbase database for row and read data; If Query Result is sky, terminate inquiry, and Query Result is set for empty.
4) Query module carries out secondary filtration according to the search condition analyzed in step 1) to the data in Hbase database, and returns the data after filtration.
Inquiry request is particularly resolved into a series of word symbol according to word-building rule by lexical analysis by the lexical analysis described in step 1) and grammatical analysis, then, by syntax analyzer, described word symbol is built into parse tree, and do syntax check and grammatical analysis, finally obtain search condition and search field;
Described word symbol comprises key word, indications and operational symbol.
For the storage schematic diagram of the Internet of Things data access method of Fig. 4, terminal current state information has 200 fields, and wherein TerminalID, f5, f193, f195, f197 are search field, and all the other are for representing field.Sql statement is select*fromcurrentInfowheref5=" adb " andf193=1andf2> ' dd '; Then query manipulation comprises the steps:
1) after Query module receives the inquiry request of user, after lexical analysis and grammatical analysis, obtain search field is f5 and f193, and needs to filter representing field f2;
2) in Query module to Mysql database, inquiry meets f5=" adb " terminal of andf193=1;
3) Query module is according to reading terminals information in the Termination ID set inquired in Mysql database to Hbase database.
Storage operation as shown in Figure 2, comprises the steps:
1) Importer module carries out field analysis to data record, obtains search field and represents field;
2) by search field stored in Mysql database;
3) take Termination ID as line unit, using representing field as Value stored in Hbase database;
3) if Mysql database and Hbase database all do not store successfully, then perform rolling back action, the data stored are deleted from above-mentioned database.
Query manipulation as shown in Figure 3, comprises the steps:
1) the Sql statement of user's input for retrieving;
2) lexical analyzer of Query module carries out lexical analysis to Sql statement;
The syntax analyzer of Query module carries out grammatical analysis and syntax check to the word of lexical analyzer analysis and symbol;
3) according to the result of grammatical analysis, the record reading coupling in search field to Mysql database is used;
Hbase reading demonstrating data is gathered according to the TerminalID that Mysql returns;
4) according to filtercondition, data are filtered, and return the data after filtration.
Claims (6)
1. an Internet of Things data access method, is characterized in that: comprise data and store importing module and Importer module, Mysql database, Hbase database and inquiry Query module; Wherein Importer module is analyzed all fields when data store and is extracted search field and represent field, by search field stored in Mysql database, will represent field stored in Hbase; Inquiry Query module in charge carries out lexical analysis and grammatical analysis to query statement, and reads relevant information according in analysis result to Mysql and Hbase.
2. Internet of Things data access method according to claim 1, is characterized in that: Internet of Things data access method comprises storage operation, renewal rewards theory and query manipulation.
3. Internet of Things data access method according to claim 2, is characterized in that, described storage operation comprises the steps:
1) Importer module reads data to be stored;
2) Importer module carries out field analysis to the data to be stored described in step 1), obtains search field and represents field;
3) Importer module is by step 2) described in search field be stored in Mysql database;
4) Importer module is by step 2) described in the field that represents be that line unit rowkey is stored into Hbase database with Termination ID, described Hbase database and non-relational distributed data base.
4. Internet of Things data access method according to claim 2, is characterized in that, described renewal rewards theory comprises the steps:
1) Importer module reads data to be updated;
2) Importer module analyzes search field and represents field from the data to be updated described in step 1);
3) search field is updated in Mysql database by Importer module;
4) Importer module will represent field and will be updated in Hbase database.
5. Internet of Things data access method according to claim 2, is characterized in that, described query manipulation, comprises the steps:
1), after Query module receives the inquiry request of user, lexical analysis is carried out to described inquiry request, and grammatical analysis;
2) Query module is inquired about in Mysql database according to the search condition analyzed in step 1); If Query Result is sky, terminate inquiry, and Query Result is set for empty;
3) Query module is with step 2) in the Termination ID that inquires be good in Hbase database for row and read data; If Query Result is sky, terminate inquiry, and Query Result is set for empty;
4) Query module carries out secondary filtration according to the search condition analyzed in step 1) to the data in Hbase database, and returns the data after filtration.
6. Internet of Things data access method according to claim 5, it is characterized in that, inquiry request is particularly resolved into a series of word symbol according to word-building rule by lexical analysis by the lexical analysis described in step 1) and grammatical analysis, then, by syntax analyzer, described word symbol is built into parse tree, and do syntax check and grammatical analysis, finally obtain search condition and search field;
Described word symbol comprises key word, indications and operational symbol.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510787124.3A CN105279281A (en) | 2015-11-17 | 2015-11-17 | Internet-of-things data access method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510787124.3A CN105279281A (en) | 2015-11-17 | 2015-11-17 | Internet-of-things data access method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105279281A true CN105279281A (en) | 2016-01-27 |
Family
ID=55148295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510787124.3A Withdrawn CN105279281A (en) | 2015-11-17 | 2015-11-17 | Internet-of-things data access method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105279281A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106126553A (en) * | 2016-06-16 | 2016-11-16 | 西安科技大市场有限公司 | A kind of storage method based on the big data of scientific and technological resources |
| CN108241627A (en) * | 2016-12-23 | 2018-07-03 | 北京神州泰岳软件股份有限公司 | A kind of isomeric data storage querying method and system |
| CN109299114A (en) * | 2018-11-21 | 2019-02-01 | 安徽云融信息技术有限公司 | A kind of complex database and its efficient querying method |
| CN109408513A (en) * | 2018-10-23 | 2019-03-01 | 杭州能链科技有限公司 | Data processing method, system and storage medium |
| CN109739883A (en) * | 2019-01-03 | 2019-05-10 | 广东南方数码科技股份有限公司 | Promote the method, apparatus and electronic equipment of data query performance |
| CN113407609A (en) * | 2021-06-29 | 2021-09-17 | 中国民生银行股份有限公司 | External data using method, device and equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030074375A1 (en) * | 1999-08-12 | 2003-04-17 | Nakamura Lee Evan | Data acess system |
| CN102521416A (en) * | 2011-12-28 | 2012-06-27 | 用友软件股份有限公司 | Data correlation query method and data correlation query device |
| CN102799622A (en) * | 2012-06-19 | 2012-11-28 | 北京大学 | Distributed structured query language (SQL) query method based on MapReduce expansion framework |
| CN102982075A (en) * | 2012-10-30 | 2013-03-20 | 北京京东世纪贸易有限公司 | Heterogeneous data source access supporting system and method thereof |
| CN103761080A (en) * | 2013-12-25 | 2014-04-30 | 中国农业大学 | Structured query language (SQL) based MapReduce operation generating method and system |
| CN104142957A (en) * | 2013-05-10 | 2014-11-12 | 上海联影医疗科技有限公司 | Method and system for regional medical treatment-orientated data sharing |
-
2015
- 2015-11-17 CN CN201510787124.3A patent/CN105279281A/en not_active Withdrawn
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030074375A1 (en) * | 1999-08-12 | 2003-04-17 | Nakamura Lee Evan | Data acess system |
| CN102521416A (en) * | 2011-12-28 | 2012-06-27 | 用友软件股份有限公司 | Data correlation query method and data correlation query device |
| CN102799622A (en) * | 2012-06-19 | 2012-11-28 | 北京大学 | Distributed structured query language (SQL) query method based on MapReduce expansion framework |
| CN102982075A (en) * | 2012-10-30 | 2013-03-20 | 北京京东世纪贸易有限公司 | Heterogeneous data source access supporting system and method thereof |
| CN104142957A (en) * | 2013-05-10 | 2014-11-12 | 上海联影医疗科技有限公司 | Method and system for regional medical treatment-orientated data sharing |
| CN103761080A (en) * | 2013-12-25 | 2014-04-30 | 中国农业大学 | Structured query language (SQL) based MapReduce operation generating method and system |
Non-Patent Citations (1)
| Title |
|---|
| 孙志佳: ""基于Hadoop的在线购物原型系统的设计与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106126553A (en) * | 2016-06-16 | 2016-11-16 | 西安科技大市场有限公司 | A kind of storage method based on the big data of scientific and technological resources |
| CN106126553B (en) * | 2016-06-16 | 2020-02-14 | 西安科技大市场有限公司 | Storage method based on scientific and technological resource big data |
| CN108241627A (en) * | 2016-12-23 | 2018-07-03 | 北京神州泰岳软件股份有限公司 | A kind of isomeric data storage querying method and system |
| CN109408513A (en) * | 2018-10-23 | 2019-03-01 | 杭州能链科技有限公司 | Data processing method, system and storage medium |
| CN109299114A (en) * | 2018-11-21 | 2019-02-01 | 安徽云融信息技术有限公司 | A kind of complex database and its efficient querying method |
| CN109739883A (en) * | 2019-01-03 | 2019-05-10 | 广东南方数码科技股份有限公司 | Promote the method, apparatus and electronic equipment of data query performance |
| CN113407609A (en) * | 2021-06-29 | 2021-09-17 | 中国民生银行股份有限公司 | External data using method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105279281A (en) | Internet-of-things data access method | |
| CN109299102B (en) | HBase secondary index system and method based on Elastcissearch | |
| CN109508355B (en) | A data extraction method, system and terminal device | |
| CN110275920B (en) | Data query method and device, electronic equipment and computer readable storage medium | |
| KR101083563B1 (en) | Method and System for Managing Database | |
| CN101727502A (en) | Data query method, data query device and data query system | |
| CN111930615B (en) | Code quality evaluation method and device | |
| CN106407360B (en) | Data processing method and device | |
| CN104636389A (en) | Hbase database real-time query achieving method and system | |
| CN103744913A (en) | Database retrieval method based on search engine technology | |
| CN114329082A (en) | Data blood relationship analysis method and system based on hugagraph | |
| CN118312524B (en) | Table recall method, apparatus, electronic device and medium | |
| CN111291037A (en) | Data storage and query method, device, equipment and computer storage medium | |
| CN106933869A (en) | A kind of method and apparatus of operating database | |
| CN109522271A (en) | A kind of batch insertion of B+ tree node and delet method and device | |
| CN112783927B (en) | Database query method and system | |
| CN105095436A (en) | Automatic modeling method for data of data sources | |
| EP3123360B1 (en) | Partition filtering using smart index in memory | |
| CN104408128B (en) | A kind of reading optimization method indexed based on B+ trees asynchronous refresh | |
| US6826563B1 (en) | Supporting bitmap indexes on primary B+tree like structures | |
| CN103810300A (en) | Method and device for inquiring non-index coverage data | |
| CN112612859A (en) | DDL analysis method and device based on log analysis | |
| CN110674142A (en) | Oracle database index optimization method | |
| CN104331460A (en) | Hbase-based data read-write operation method and system | |
| CN108984720B (en) | Data query method and device based on column storage, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20160127 |