CN108509636A - It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology - Google Patents
It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology Download PDFInfo
- Publication number
- CN108509636A CN108509636A CN201810316136.1A CN201810316136A CN108509636A CN 108509636 A CN108509636 A CN 108509636A CN 201810316136 A CN201810316136 A CN 201810316136A CN 108509636 A CN108509636 A CN 108509636A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- cutting
- partition table
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of big data management disaster recovery method for realizing read and write abruption based on partition table technology comprising following steps：Step S1：According to three kinds of range partition, Hash subregions, compound subregion divisional types, respective partition table is established；Step S2：Index is created to the partition table that step S1 is established；Step S3：To database into being cut into primary database and from database；Step S4：Primary database handles transactional operation, is operated from database processing SELECT query；Step S5：It will be synchronized from database and primary database.The read and write abruption of database, libraries more in this way exist simultaneously, and effectively prevent the generation of disaster, improve stability and the safety of data；In terms of data performance, on the one hand balanced I/O operation, reasonable distribution physical disk space greatly improve the performance of big data inquiry, are on the other hand convenient for the maintenance of data, enhance the availability of data.
Description
Technical field
The invention belongs to database management application fields, and in particular to a kind of to realize read and write abruption based on partition table technology
Big data manages disaster recovery method.
Background technology
In modern big data software development application scenarios, the load that oracle database software is stored as data is used mostly
Body, and a kind of important means and method that oracle database partitioning technique optimizes as oracle database performance, for answering
To big data processing, the search efficiency and data management disaster-tolerant backup for improving data play an important role.
Currently, in the application of database, data are stored in single table, and all tables correspond to the physics in computer disk
File, due to the design problem of database, over time, the data volume of single table is more and more, the speed of data query
Slower and slower, the system of eventually leading to cannot respond to even to routed.And the balanced I/O operation of partition table technology, by different subregions
It is mapped to disk, improves the performance of data query, enhances the availability of data.
For in current software development process, database is configured as an important ring, there are database design it is not perfect,
Big data storage can not be coped with, can not rapidly carry out the problems such as disaster recovery reduction, can be solved using partition table technology.
Invention content
Realizing that the big data of read and write abruption manages disaster recovery method based on partition table technology the object of the present invention is to provide a kind of.
The present invention uses following technical scheme：It is a kind of to realize that the big data of read and write abruption manages disaster tolerance based on partition table technology
Method comprising following steps：Step S1：According to three kinds of range partition, Hash subregions, compound subregion divisional types, phase is established
Answer partition table；Step S2：Index is created to the partition table that step S1 is established；Step S3：To database into being cut into primary database
And from database；Step S4：Primary database handles transactional operation, is operated from database processing SELECT query；Step S5：It will
It is synchronized from database and primary database.
In an embodiment of the present invention, the partition table includes following operation：It is inquiry, insertion, update, deletion record, new
Increase, merge and delete subregion.
In an embodiment of the present invention, the index of partition table includes creating partial indexes and global index；When going out in subregion
Show many affairs and to ensure to use global index when the uniqueness of the data record in all subregions.
In an embodiment of the present invention, the cutting of database is physics cutting, specifically includes following steps：Data are passed through
For physics segmentation rules by data distribution to different DB servers, physics segmentation rules include being cut with function for business cutting
Point, business cutting carries out cutting according to database level residing in different service application scenes, and a system can include
Multiple business special topics, each business workshop possess sub- special topic, the distribution of DB servers are carried out according to different business special topics；Work(
Energy cutting carries out cutting according to the effect that database is played in systems, and special topic has general utility functions in system, according to function
The different cuttings for carrying out DB servers；Access specific database by routing rule routing, each access plane pair just not
It is single server, but multiple servers.
In an embodiment of the present invention, the cutting of database is data cutting, specifically includes following steps：Data are passed through
Data segmentation rules, segmentation rules include type cutting and time cutting；Type cutting is cut according to the difference of data type
Point, homogeneous data is placed in same table, and inhomogeneity data are split in different tables；Time of the time cutting residing for data
Data are carried out data separating according to different periods such as dates according to data volume, so pass through different rule by Duan Jinhang cuttings
It then will be in the different tables of data distribution a to database.
In an embodiment of the present invention, the database is oracle.
Further, step S5 includes the following steps：When primary database writes library operation, synchronized update cache, every time
It reads first reading cache and reads DB again；It is synchronous with the data from database that primary database is responsible for using dataguard.
Further, there are one primary databases, have from data multiple.
Compared with prior art, the present invention carries out vertical cutting, the water of data by carrying out subregion table handling to database
Flat slicing operation realizes that the read and write abruption of database, libraries more in this way exist simultaneously, effectively prevents the generation of disaster, improves
The stability of data and safety；In terms of data performance, on the one hand balanced I/O operation, reasonable distribution physical disk are empty
Between, the performance of big data inquiry is greatly improved, is on the other hand convenient for the maintenance of data, enhances the availability of data.
Description of the drawings
Fig. 1 is database cutting flow diagram in one embodiment of the invention.
Specific implementation mode
Explanation is further explained to the present invention in the following with reference to the drawings and specific embodiments.
It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology comprising following steps：Step
Rapid S1：According to three kinds of range partition, Hash subregions, compound subregion divisional types, respective partition table is established；Step S2：To step
The partition table that S1 is established creates index；Step S3：To database into being cut into primary database and from database；Step S4：Main number
According to library processing transactional operation, operated from database processing SELECT query；Step S5：It will be carried out from database and primary database
It is synchronous.
In an embodiment of the present invention, the partition table includes following operation：It is inquiry, insertion, update, deletion record, new
Increase, merge and delete subregion.
In an embodiment of the present invention, the index of partition table includes creating partial indexes and global index；When going out in subregion
Show many affairs and to ensure to use global index when the uniqueness of the data record in all subregions.
In an embodiment of the present invention, the cutting of database is physics cutting, specifically includes following steps：Data are passed through
Physics segmentation rules access specific database, often by data distribution to different DB servers by routing rule routing
The just not instead of single server of secondary access plane pair, multiple servers.Physics segmentation rules are mainly business cutting and work(
Can cutting, mainly the level residing in different service application scenes according to database carries out cutting for business cutting, and one is
System can include multiple business special topic, and each business workshop possesses sub- special topic, and DB servers are carried out according to different business special topics
Distribution；Function cutting mainly carries out cutting according to the effect that database is played in systems, and special topic has picture in system
The general utility functions such as acquisition, message distribution, pre-alert notification, according to the different cuttings for carrying out DB servers of function.
In an alternative embodiment of the invention, the cutting of database is data cutting, specifically includes following steps：It is logical to data
Data segmentation rules are crossed, it will be in the different tables of data distribution a to database.Data segmentation rules be mainly type cutting with
Time cutting, type cutting mainly carry out cutting according to the different of data type, and data have monitoring, business, history, basic data
Etc. types, homogeneous data be placed in same table, inhomogeneity data are split in different tables；Time cutting is mainly according to data institute
The period at place carries out cutting, and data are carried out data separating according to different periods such as dates according to data volume.
In an embodiment of the present invention, the database is oracle.
Further, step S5 includes the following steps：When primary database writes library operation, synchronized update cache, every time
It reads first reading cache and reads DB again；It is synchronous with the data from database that primary database is responsible for using dataguard.
Further, there are one primary databases, have from data multiple.
The present invention is by taking oracle database as an example.Oracle database partition table technology is divided into three kinds of methods：Range partition,
Hash subregions, compound subregion.Range partition is exactly to carry out subregion to the range of some value in tables of data, according to the model of some value
It encloses, which subregion which is stored on by decision；Hash subregion is by specified partition number come the one of Uniform-distributed Data
Kind divisional type, because by carrying out hash subregion in I/O equipment so that these partition sizes are consistent, that is, only name
Partition name uniformly carries out data distribution in this way；Compound subregion is first use scope subregion, is then reused in each subregion
Hash a kind of partition method of subregion.On the other hand, partition table can establish index as general table, and partition table can create
Partial indexes and global index.When occurring many affairs in subregion and to ensure the uniqueness of the data record in all subregions
Shi Caiyong global indexes.
In a specific embodiment of the invention, key step is as follows：
1, partition table is established；According to three kinds of range partition, Hash subregions, compound subregion divisional types, respective partition is established；Subregion
Table can be inquired, be inserted into, being updated, deletion record operates, and can also be increased newly, merges, delete division operation；
2, index is established；Partition table can establish index as general table, and partition table can create partial indexes and global rope
Draw.Global index is used when occurring many affairs in subregion and to ensure the uniqueness of the data record in all subregions.
3, database cutting；Data cutting is divided into vertical cutting and horizontal cutting, data cutting can be physically,
Data will be route by routing rule in data distribution to different DB servers and access spy by a series of segmentation rules
Fixed database, so just not instead of single server of each access plane pair, N platform servers thus can be with
Reduce the load pressure of single machine.Data cutting can also be in database, to data by a series of segmentation rules,
It will be in the different tables of data distribution a to database.Point library reduces the load of single-point machine；Divide table, improves data manipulation
Efficiency, especially Write operation efficiency.Main flow schematic diagram is referring to Fig. 1.
4, data base read-write detaches；Read and write abruption, basic principle are that primary database processing transactional is allowed to increase, change, delete behaviour
Make（INSERT、UPDATE、DELETE）, and operated from database processing SELECT query, database duplication is used to an affairs
Property operation caused by change be synchronized to slave database in cluster.
5, database synchronization；By taking oracle as an example, master library is responsible for writing data, reads data.Library is read to be merely responsible for reading data.Every time
Library operation is write, synchronized update cache reads first read cache in reading DB every time.Library is write with regard to one, reading library can have multiple, adopt
It is synchronous with multiple reading data in library to be responsible for master library with dataguard.
The above are preferred embodiments of the present invention, all any changes made according to the technical solution of the present invention, and generated function is made
When with range without departing from technical solution of the present invention, all belong to the scope of protection of the present invention.
Claims (8)
1. a kind of realizing that the big data of read and write abruption manages disaster recovery method based on partition table technology, it is characterised in that：Including following
Step：
Step S1：According to three kinds of range partition, Hash subregions, compound subregion divisional types, respective partition table is established；
Step S2：Index is created to the partition table that step S1 is established；
Step S3：To database into being cut into primary database and from database；
Step S4：Primary database handles transactional operation, is operated from database processing SELECT query；
Step S5：It will be synchronized from database and primary database.
2. according to claim 1 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：The partition table includes following operation：Inquiry, insertion, update, deletion record, newly-increased, merging and deletion subregion.
3. according to claim 1 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：The index of partition table includes creating partial indexes and global index；When occurring many affairs in subregion and to ensure
Global index is used when the uniqueness of the data record in all subregions.
4. according to claim 1 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：The cutting of database is physics cutting, specifically includes following steps：Data are divided data by physics segmentation rules
On cloth to different DB servers, physics segmentation rules are including being business cutting and function cutting, and business cutting is according to database
Residing level carries out cutting in different service application scenes, and a system can include multiple business special topic, each business
Workshop possesses sub- special topic, and the distribution of DB servers is carried out according to different business special topics；Function cutting is being according to database
Effect played in system carries out cutting, and special topic has general utility functions in system, according to the different progress DB servers of function
Cutting；Specific database is accessed by routing rule routing, and the just not instead of single server of each access plane pair is more
Platform server.
5. according to claim 1 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：The cutting of database is data cutting, specifically includes following steps：Data segmentation rules, cutting rule are passed through to data
Include then type cutting and time cutting；Type cutting carries out cutting according to the different of data type, and homogeneous data is placed on same
It opens in table, inhomogeneity data are split in different tables；Period of the time cutting residing for data carries out cutting, according to data
Data are carried out data separating by amount according to different periods such as dates, so by different rules by data distribution a to number
According in the different tables in library.
6. according to claim 1 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：The database is oracle.
7. according to claim 6 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：Step S5 includes the following steps：When primary database writes library operation, synchronized update cache reads first read every time
Cache reads DB again；It is synchronous with the data from database that primary database is responsible for using dataguard.
8. according to claim 7 realize that the big data of read and write abruption manages disaster recovery method based on partition table technology, special
Sign is：There are one primary databases, has from data multiple.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810316136.1A CN108509636A (en) | 2018-04-10 | 2018-04-10 | It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810316136.1A CN108509636A (en) | 2018-04-10 | 2018-04-10 | It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108509636A true CN108509636A (en) | 2018-09-07 |
Family
ID=63381417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810316136.1A Pending CN108509636A (en) | 2018-04-10 | 2018-04-10 | It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108509636A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110471661A (en) * | 2019-08-20 | 2019-11-19 | 深圳科蓝金信科技发展有限公司 | The data description method of front-end interface and the generation method of front-end interface |
| CN111400309A (en) * | 2020-02-23 | 2020-07-10 | 中国平安财产保险股份有限公司 | Data updating method, electronic device and readable storage medium |
| CN112835887A (en) * | 2019-11-25 | 2021-05-25 | 航天信息股份有限公司 | Database management method, database management device, computing equipment and storage medium |
| CN114253770A (en) * | 2021-12-17 | 2022-03-29 | 苏州浪潮智能科技有限公司 | Master-slave backup system of database |
| CN115617898A (en) * | 2022-12-21 | 2023-01-17 | 中国科学院长春光学精密机械与物理研究所 | Shooting range measurement and control data processing system, method and computer equipment based on SOA architecture |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982186A (en) * | 2012-12-26 | 2013-03-20 | 厦门市美亚柏科信息股份有限公司 | Method and system for maintaining range partitioning table based on oracle database system |
| CN105930387A (en) * | 2016-04-14 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Data operation system and method based on data routing and sharding |
| US20170103094A1 (en) * | 2015-10-07 | 2017-04-13 | Oracle International Corporation | Request routing and query processing in a sharded database |
-
2018
- 2018-04-10 CN CN201810316136.1A patent/CN108509636A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982186A (en) * | 2012-12-26 | 2013-03-20 | 厦门市美亚柏科信息股份有限公司 | Method and system for maintaining range partitioning table based on oracle database system |
| US20170103094A1 (en) * | 2015-10-07 | 2017-04-13 | Oracle International Corporation | Request routing and query processing in a sharded database |
| CN105930387A (en) * | 2016-04-14 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Data operation system and method based on data routing and sharding |
Non-Patent Citations (2)
| Title |
|---|
| 金磐石: "分布式架构在银行核心业务系统的应用", 《计算机系统应用》 * |
| 项凯: "面向海量高并发数据库中间件的研究与应用", 《中国优秀硕士学位论文全文数据库，信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110471661A (en) * | 2019-08-20 | 2019-11-19 | 深圳科蓝金信科技发展有限公司 | The data description method of front-end interface and the generation method of front-end interface |
| CN112835887A (en) * | 2019-11-25 | 2021-05-25 | 航天信息股份有限公司 | Database management method, database management device, computing equipment and storage medium |
| CN111400309A (en) * | 2020-02-23 | 2020-07-10 | 中国平安财产保险股份有限公司 | Data updating method, electronic device and readable storage medium |
| CN111400309B (en) * | 2020-02-23 | 2023-09-15 | 中国平安财产保险股份有限公司 | Data updating method, electronic device and readable storage medium |
| CN114253770A (en) * | 2021-12-17 | 2022-03-29 | 苏州浪潮智能科技有限公司 | Master-slave backup system of database |
| CN114253770B (en) * | 2021-12-17 | 2023-12-26 | 苏州浪潮智能科技有限公司 | Master-slave backup system of database |
| CN115617898A (en) * | 2022-12-21 | 2023-01-17 | 中国科学院长春光学精密机械与物理研究所 | Shooting range measurement and control data processing system, method and computer equipment based on SOA architecture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108509636A (en) | It is a kind of to realize that the big data of read and write abruption manages disaster recovery method based on partition table technology | |
| CN104657459B (en) | A kind of mass data storage means based on file granularity | |
| US9501550B2 (en) | OLAP query processing method oriented to database and HADOOP hybrid platform | |
| Băzăr et al. | The Transition from RDBMS to NoSQL. A Comparative Analysis of Three Popular Non-Relational Solutions: Cassandra, MongoDB and Couchbase. | |
| AU2015369723B2 (en) | Identifying join relationships based on transactional access patterns | |
| CN103631924B (en) | A kind of application process and system of distributive database platform | |
| WO2015106711A1 (en) | Method and device for constructing nosql database index for semi-structured data | |
| US20100235606A1 (en) | Composite hash and list partitioning of database tables | |
| CN111917834A (en) | A data synchronization method, device, storage medium and computer equipment | |
| CN103793493B (en) | A kind of method and system for handling car-mounted terminal mass data | |
| WO2003009178A2 (en) | Database systems, methods and computer program products including primary key and super key indexes for use with partitioned tables | |
| CN108319645B (en) | Multi-version file view management method and device in heterogeneous storage environment | |
| US11151081B1 (en) | Data tiering service with cold tier indexing | |
| CN106611053B (en) | Data cleaning and indexing method | |
| CN111639082A (en) | Object storage management method and system of billion-level node scale knowledge graph based on Ceph | |
| CN108509437A (en) | A kind of ElasticSearch inquiries accelerated method | |
| CN105069109A (en) | Dilatation method and system of distributed database | |
| CN105426373A (en) | Database synchronization method and device | |
| KR20190063835A (en) | System for processing real-time data modification of in-memory database | |
| CN107590257A (en) | A kind of data base management method and device | |
| US7310719B2 (en) | Memory management tile optimization | |
| CN104598652B (en) | A kind of data base query method and device | |
| CN103365987A (en) | Clustered database system and data processing method based on shared-disk framework | |
| CN108628969A (en) | Spatial keyword indexing method and platform and storage medium | |
| CN107408239B (en) | Architecture for managing mass data in communication application through multiple mailboxes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180907 |