CN110222035A - A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery - Google Patents
A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery Download PDFInfo
- Publication number
- CN110222035A CN110222035A CN201910495162.XA CN201910495162A CN110222035A CN 110222035 A CN110222035 A CN 110222035A CN 201910495162 A CN201910495162 A CN 201910495162A CN 110222035 A CN110222035 A CN 110222035A
- Authority
- CN
- China
- Prior art keywords
- page
- data
- database
- damage
- check
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of efficient fault-tolerance approaches of the database page based on exclusive or check and journal recovery, the following steps are included: calculating the corresponding check code of page of data and being stored in the head of page of data；From when read data pages are into memory in storage medium, judge whether page of data is damaged by check code；During log-on data library carries out instance recovery, then the corrupted data page is skipped so that database normally starts, prompting user data page corruption occurs in which database object after starting；After log-on data Kucheng's function, determine that page corruption problem occurs in which table, user is handled by special SQL come the damage page to the table.Whether the present invention has rational design, damage in the XOR operation check code of the page of data store data inside page for the real-time detection page, can successfully restore the data damaged, reduce the data degradation of user.
Description
Technical field
The invention belongs to data storages and inquiring technology field, especially a kind of number based on exclusive or check and journal recovery
According to the efficient fault-tolerance approach of the library page.
Background technique
Usually, the data of large-scale general relevant database be stored in non-volatile memory medium (it is common as
Disk, disk array, network storage or solid state hard disk), these storage mediums are to reduce IO expense, usually with page of data
(512 integral multiples) are written and read for unit, that is to say, that even the data of 1 byte of access, it is also necessary to read and write entire page
Face is 8K size in magic data database storing page-size.
With the data explosion that information technology generates, cause the data volume stored in modern data library increasing, data
Frequent access lead to occur in storage medium that bad block frequency is higher and higher, and part bad block cannot be by the management of storage medium itself
System discovery (cost and performance factor).Once occurring bad block in database datafile, gently then lead to partial database object
It can not access, it is heavy then cause Database Systems that can not start.
Currently, the usual method to solve the above problems is to be examined in real time by the file system of operating system to block corruptions
It surveys, finds bad block in advance, but this method cannot very avoid bad block accessed, especially that class data can normally be read
It writes, but realizes that the bad block overturn occurs in the data content returned, cannot be found substantially.
Summary of the invention
It is an object of the invention to overcome the deficiencies in the prior art, propose that a kind of design is reasonable, guarantee data security access
And reduce the efficient fault-tolerance approach of the database page based on exclusive or check and journal recovery of user data loss.
The present invention solves its technical problem and adopts the following technical solutions to achieve:
A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery, comprising the following steps:
Step 1: before storage medium is written in the page of data in memory, the corresponding check code of page of data is calculated,
And the check code is stored in the head of page of data；
Step 2: from when read data pages are into memory in storage medium, calculating the school of the page of data read out
Code is tested, carry out numerical value comparison with the check code for being stored in page head: if numerical value is equal, this page does not have mistake, mark
It is denoted as normal page；If numerical value etc., mistake is had occurred in this page, labeled as the damage page, enters step 3；
Step 3: during log-on data library carries out instance recovery, then skipping the corrupted data page so that database
Normal starting reminds user data page corruption occurs in which database object after starting.
Step 4: after log-on data Kucheng's function, determining that page corruption problem occurs in which table, user passes through special
SQL is handled come the damage page to the table.
Step 1 check code is to carry out 64 XOR operation to the data in page of data to generate, and by check code
It charges in page head.
The check code is 8 byte lengths, and check code is stored in the head of page of data.
The concrete methods of realizing of the step 1 are as follows:
(1) prepare the page of data in memory storage medium is written；
(2) data page size is 8192 bytes, can include in memory 1024 64 as one by page of data
The array of integer；
(3) the 1st element in 64 integer arrays is assigned a value of 0；
(4) the 1st element in 64 integer arrays is successively subjected to exclusive or fortune with remaining 1023 elements in array
It calculates, the 1st element is written in each result；
(5) storage medium is written into page of data.
The implementation method of the step 2 the following steps are included:
(1) page of data is read in into memory from storage medium；
(2), if storage medium reports an error in read data pages, the page of data in memory is labeled as having damaged；
(3) data page size is 8192 bytes, can include in memory 1024 64 as one by page of data
The array of integer；
(4) successively carried out using the local variable x that initial value is 0 with 1023 elements in array in addition to first element
Local variable x is written in XOR operation, each result；
(5) do you judge that local variable x is equal with first element of 64 integer arrays? it, will be in memory if equal
Page of data labeled as effective；If unequal, by the page of data in memory labeled as damage.
The processing method of the step 3 the following steps are included:
(1) database starting starts to carry out instance recovery；
(2) initialization damage page of data records chained list；
(3) log is read from redoing log in file；
(4) log recording is parsed, page of data number is obtained；
(5) if page of data number skips the log in damage page of data record chained list；
(6) from storage medium read data pages；
(7), if data page corruption, damage page of data is added in page of data and its affiliated database object and records chain
In table, and the log is skipped, without reforming, otherwise, reforms the log；
(8) continue to read next log, repeat the above process, until completion is reformed in all logs；
(9) after the completion of instance recovery, check in damage page of data record chained list with the presence or absence of record, if it does, to using
Family gives a warning, and tells that the page of data of which database object of user is damaged；
(10) database starts.
The step 4 handled by SQL come the damage page to the table the following steps are included:
(1) exclusive lock is added to table, to prevent other session access table；
(2) the page of data of unmarked damage is successively obtained from segment management module: identifying whether the page of data is damaged, such as
Fruit page of data is damaged, and is marked in tablespace management structure, does not use the page of data later, and in section pipe
Reason module is also marked, and when carrying out the page search of section later, needs to skip the page；
(3) continue to obtain page of data, until all page of data handled section；
(4) the exclusive lock of free list, return run succeeded.
The advantages and positive effects of the present invention are:
The present invention has rational design, 8 bytes is distributed on database page head, to the different of this page store data inside page
Or arithmetic check code, it can be used for whether the real-time detection page damages.In the case where detecting data page corruption, pass through
The mode for skipping the access damage page, enables database normally to start success, and be not loaded with the data for having already appeared damage.
After database starting, it can be damaged by database administrator user using the means such as data backup, log, mirror image, recovery
Data, and write in unspoiled page of data, restore data.To these means are not configured so as to cause restoring data invariably
The case where, user still can handle the database object comprising damage page of data by sql command, make in database object not
Data in the page of data of damage can be accessed normally, reduce the data degradation of user.
Detailed description of the invention
Fig. 1 is page of data write-in flow chart of the invention；
Fig. 2 is that page of data of the invention reads flow chart；
Fig. 3 is fault-tolerant damage page work flow diagram of the invention；
Fig. 4 is processing damage page flow chart of the invention.
Specific embodiment
The embodiment of the present invention is further described below in conjunction with attached drawing.
A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery, comprising the following steps:
Step 1: before storage medium is written in the page of data in memory, first calculating corresponding 8 byte of page of data
Check code, and check code is stored in 8 byte of head of page of data.Purpose in this way is can to identify and locate
Manage the page of damage.
The present invention is in order to identify the damage page, before storage medium is written in page of data, to the data in page of data
64 XOR operation are carried out, operation result is charged in 8 bytes on page head.By page of data from storage medium reading in
When depositing, 64 XOR operation are equally carried out, result and 8 byte of page head are compared.
As shown in Figure 1, page of data writing process the following steps are included:
(1) prepare the page of data in memory storage medium is written
(2) data page size is 8192 bytes, can include in memory 1024 64 as one by page of data
The array of integer
(3) the 1st element in 64 integer arrays is assigned a value of 0
(4) the 1st element in 64 integer arrays is successively subjected to exclusive or fortune with remaining 1023 elements in array
It calculates, the 1st element is written in each result
(5) storage medium is written into page of data.
Step 2: from when read data pages are into memory in storage medium, according to the identical calculating side of step 1
Formula calculates 8 byte check codes of the page of data read out, and 8 bytes for being stored in page head with step 1 carry out numerical value
Comparison: if numerical value is equal, this page does not have mistake, is labeled as normal page；If numerical value etc., this page hair
Mistake has been given birth to, labeled as the damage page, this page of specially treated is required in step 3 and step 4.
As shown in Fig. 2, page of data reading process the following steps are included:
(1) page of data is read in into memory from storage medium；
(2), if storage medium reports an error in read data pages, the page of data in memory is labeled as having damaged；
(3) data page size is 8192 bytes, can include in memory 1024 64 as one by page of data
The array of integer；
(4) successively carried out using the local variable x that initial value is 0 with 1023 elements in array in addition to first element
Local variable x is written in XOR operation, each result；
(5) do you judge that local variable x is equal with first element of 64 integer arrays? it, will be in memory if equal
Page of data labeled as effective；If unequal, by the page of data in memory labeled as damage.
Step 3: during log-on data library, need to redo log, when reading the content of the relevant page of data of log,
If page corruption is marked in step 2, need to carry out fault-tolerant processing: the page of all damages needs to be saved in special chained list
In；Since the page has damaged, log relevant to the page, which is reformed, to have to skip；It is alerted to user, prompts to damage
Table where the bad page.
Data page corruption is found during log-on data library carries out instance recovery, then is skipped the page, is guaranteed
Database can normally start, and remind user data page corruption occurs in which database object after starting.
As shown in figure 3, fault-tolerant damage page processing method the following steps are included:
(1) database starting starts to carry out instance recovery
(2) initialization damage page of data records chained list
(3) log is read from redoing log in file
(4) log recording is parsed, page of data number is obtained
(5) if page of data number skips the log, without reforming in damage page of data record chained list
(6) from storage medium read data pages
(7), if data page corruption, damage page of data is added in page of data and its affiliated database object and records chain
In table, and the log is skipped, without reforming.Otherwise, the log is reformed
(8) continue to read next log, repeat the above process, until completion is reformed in all logs.
(9) after the completion of instance recovery, check in damage page of data record chained list with the presence or absence of record, if it does, to using
Family gives a warning, and tells that the page of data of which database object of user is damaged.
(10) database starts successfully.
Step 4: after log-on data Kucheng's function, can specify that page damage occurs in which table by step 3 warning information
Bad problem, user can be handled by special SQL come the damage page to the table, mainly include the following contents:
In the segment management information of table, traverses all page of data of section and carry out the page check of step 2, if the page
Damage, needs for the page to be identified, requires to ignore the page to the access of table later；
The damage page found during section traversal simultaneously, is also required to be identified damage in tablespace management structure
Bad, the later page should not be allocated again.
The present invention also provides a kind of mechanism, and user is allowed to handle by sql command " ALTER TABLE table name REPAIR "
Database object comprising damaging page of data, allows the positive frequentation of data in database object in unspoiled page of data
It asks, reduces the data degradation of user.The page of data of damage can be extractd from database object during processing, and in table
It is that each page increase marker bit is marked in space management structure, does not use the page of data later.
In magic database, the page of data of table is managed by segment management module: each table has unique correspondence
A data segment；Each data segment has unique segment number；Each page of data at most belongs to a section, and can be on page head
Data segment belonging to recording；Segment management module is also required to increase marker bit for each page of data of this section, marks whether to damage.
As shown in figure 4, in this step processing damage the page method the following steps are included:
(1) exclusive lock is added to table, to prevent other session access table
(2) the page of data of unmarked damage is successively obtained from segment management module: identifying whether the page of data is damaged, such as
Fruit page of data is damaged, and is marked in tablespace management structure, does not use the page of data later, and in section pipe
Reason module is also marked, and when carrying out the page search of section later, needs to skip the page
(3) continue to obtain page of data, until all page of data handled section
(4) the exclusive lock of free list, return run succeeded.
It is emphasized that embodiment of the present invention be it is illustrative, without being restrictive, therefore the present invention includes
It is not limited to embodiment described in specific embodiment, it is all to be obtained according to the technique and scheme of the present invention by those skilled in the art
Other embodiments out, also belong to the scope of protection of the invention.
Claims (7)
1. a kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery, it is characterised in that including following step
It is rapid:
Step 1: before storage medium is written in the page of data in memory, calculating the corresponding check code of page of data, and will
The check code is stored in the head of page of data；
Step 2: from when read data pages are into memory in storage medium, calculating the verification of the page of data read out
Code, carry out numerical value comparison with the check code for being stored in page head: if numerical value is equal, this page does not have mistake, label
For normal page；If numerical value etc., mistake is had occurred in this page, labeled as the damage page, enters step 3；
Step 3: during log-on data library carries out instance recovery, then skipping the corrupted data page so that database is normal
Starting reminds user data page corruption occurs in which database object after starting.
Step 4: after log-on data Kucheng's function, determine that page corruption problem occurs in which table, user by special SQL come
The damage page of the table is handled.
2. the efficient fault-tolerance approach of a kind of database page based on exclusive or check and journal recovery according to claim 1,
It is characterized by: step 1 check code is to carry out 64 XOR operation to the data in page of data to generate, and will verify
Code is charged in page head.
3. the efficiently fault-tolerant side of a kind of database page based on exclusive or check and journal recovery according to claim 1 or 2
Method, it is characterised in that: the check code is 8 byte lengths, and check code is stored in the head of page of data.
4. the efficiently fault-tolerant side of a kind of database page based on exclusive or check and journal recovery according to claim 1 or 2
Method, it is characterised in that: the concrete methods of realizing of the step 1 are as follows:
(1) prepare the page of data in memory storage medium is written；
(2) data page size is 8192 bytes, can include in memory 1024 64 integers as one by page of data
Array；
(3) the 1st element in 64 integer arrays is assigned a value of 0；
(4) the 1st element in 64 integer arrays is successively subjected to XOR operation with remaining 1023 elements in array, often
The 1st element is written in secondary result；
(5) storage medium is written into page of data.
5. the efficient fault-tolerance approach of a kind of database page based on exclusive or check and journal recovery according to claim 1,
It is characterized by: the implementation method of the step 2 the following steps are included:
(1) page of data is read in into memory from storage medium；
(2), if storage medium reports an error in read data pages, the page of data in memory is labeled as having damaged；
(3) data page size is 8192 bytes, can include in memory 1024 64 integers as one by page of data
Array；
(4) 1023 elements in the local variable x and array that the use of initial value are 0 in addition to first element successively carry out exclusive or
Local variable x is written in operation, each result；
(5) do you judge that local variable x is equal with first element of 64 integer arrays? if equal, by the number in memory
It is effective according to page marks；If unequal, by the page of data in memory labeled as damage.
6. the efficient fault-tolerance approach of a kind of database page based on exclusive or check and journal recovery according to claim 1,
It is characterized by: the processing method of the step 3 the following steps are included:
(1) database starting starts to carry out instance recovery；
(2) initialization damage page of data records chained list；
(3) log is read from redoing log in file；
(4) log recording is parsed, page of data number is obtained；
(5) if page of data number skips the log in damage page of data record chained list；
(6) from storage medium read data pages；
(7), if data page corruption, damage page of data is added in page of data and its affiliated database object and records chained list
In, and the log is skipped, without reforming, otherwise, reform the log；
(8) continue to read next log, repeat the above process, until completion is reformed in all logs；
(9) after the completion of instance recovery, check in damage page of data record chained list with the presence or absence of record, if it does, being sent out to user
It alerts out, tells that the page of data of which database object of user is damaged；
(10) database starts.
7. the efficient fault-tolerance approach of a kind of database page based on exclusive or check and journal recovery according to claim 1,
It is characterized by: the step 4 handled by SQL come the damage page to the table the following steps are included:
(1) exclusive lock is added to table, to prevent other session access table；
(2) the page of data of unmarked damage is successively obtained from segment management module: identifying whether the page of data is damaged, if number
It is damaged according to the page, is marked in tablespace management structure, do not use the page of data later, and in segment management mould
Block is also marked, and when carrying out the page search of section later, needs to skip the page；
(3) continue to obtain page of data, until all page of data handled section；
(4) the exclusive lock of free list, return run succeeded.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910495162.XA CN110222035A (en) | 2019-06-10 | 2019-06-10 | A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910495162.XA CN110222035A (en) | 2019-06-10 | 2019-06-10 | A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110222035A true CN110222035A (en) | 2019-09-10 |
Family
ID=67816054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910495162.XA Pending CN110222035A (en) | 2019-06-10 | 2019-06-10 | A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110222035A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110764943A (en) * | 2019-10-21 | 2020-02-07 | 中国民航信息网络股份有限公司 | Data processing method and device for Oracle database |
| CN112540799A (en) * | 2020-12-02 | 2021-03-23 | 深圳市硅格半导体有限公司 | Management method, system, terminal device and storage medium of startup data |
| CN112905570A (en) * | 2021-04-08 | 2021-06-04 | 平安科技（深圳）有限公司 | Database page fault tolerance method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1784682A (en) * | 2004-05-03 | 2006-06-07 | 微软公司 | Systems and methods for automatic database or file system maintenance and repair |
| CN102760161A (en) * | 2012-06-12 | 2012-10-31 | 天津神舟通用数据技术有限公司 | Log organization structure clustered based on transaction aggregation and method for realizing corresponding recovery protocol thereof |
| CN108280097A (en) * | 2017-01-06 | 2018-07-13 | 阿里巴巴集团控股有限公司 | A kind of fault handling method and device of Database Systems |
| CN108874588A (en) * | 2018-06-08 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of database instance restoration methods and device |
-
2019
- 2019-06-10 CN CN201910495162.XA patent/CN110222035A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1784682A (en) * | 2004-05-03 | 2006-06-07 | 微软公司 | Systems and methods for automatic database or file system maintenance and repair |
| CN102760161A (en) * | 2012-06-12 | 2012-10-31 | 天津神舟通用数据技术有限公司 | Log organization structure clustered based on transaction aggregation and method for realizing corresponding recovery protocol thereof |
| CN108280097A (en) * | 2017-01-06 | 2018-07-13 | 阿里巴巴集团控股有限公司 | A kind of fault handling method and device of Database Systems |
| CN108874588A (en) * | 2018-06-08 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of database instance restoration methods and device |
Non-Patent Citations (1)
| Title |
|---|
| PDSERVICE: ""MySQL InnoDB存储引擎表损坏恢复指南"", 《HTTPS://WWW.PARNASSUSDATA.COM/EN/NODE/329?LANGUAGE=EN》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110764943A (en) * | 2019-10-21 | 2020-02-07 | 中国民航信息网络股份有限公司 | Data processing method and device for Oracle database |
| CN110764943B (en) * | 2019-10-21 | 2023-05-05 | 中国民航信息网络股份有限公司 | Data processing method and device of Oracle database |
| CN112540799A (en) * | 2020-12-02 | 2021-03-23 | 深圳市硅格半导体有限公司 | Management method, system, terminal device and storage medium of startup data |
| CN112905570A (en) * | 2021-04-08 | 2021-06-04 | 平安科技（深圳）有限公司 | Database page fault tolerance method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7103811B2 (en) | Mechanisms for detecting silent errors in streaming media devices | |
| US6195761B1 (en) | Method and apparatus for identifying and repairing mismatched data | |
| US8418030B2 (en) | Storage system with data recovery function and method thereof | |
| US6233696B1 (en) | Data verification and repair in redundant storage systems | |
| US11531590B2 (en) | Method and system for host-assisted data recovery assurance for data center storage device architectures | |
| US5860090A (en) | Append-only storage in a disk array using striping and parity caching | |
| CN101916173B (en) | RAID (Redundant Array of Independent Disks) based data reading and writing method and system thereof | |
| US7908512B2 (en) | Method and system for cache-based dropped write protection in data storage systems | |
| KR100226614B1 (en) | Built-in directory record for compressed data on a direct access storage device and method for maintaining directory information on compressed data | |
| US7020805B2 (en) | Efficient mechanisms for detecting phantom write errors | |
| JPH05505264A (en) | Non-volatile memory storage of write operation identifiers in data storage devices | |
| CN115292266B (en) | High-reliability log storage method based on memory | |
| US6389511B1 (en) | On-line data verification and repair in redundant storage system | |
| CN101377748B (en) | Method for verifying read-write function of storage device | |
| CN110222035A (en) | A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery | |
| CN101599305A (en) | Storage system with data restoration function and data restoration method thereof | |
| CN108141229A (en) | Damage the efficient detection of data | |
| CN110555055A (en) | data mining method for redo log file of Oracle database | |
| US10642508B2 (en) | Method to limit impact of partial media failure of disk drive and detect/report the loss of data for objects due to partial failure of media | |
| US9170747B2 (en) | Storage device, control device, and control method | |
| CN111666046B (en) | Data storage method, device and equipment | |
| US20060075287A1 (en) | Detecting data integrity | |
| CN104205097A (en) | De-duplicate method device and system | |
| KR100501414B1 (en) | Method of and apparatus for logging and restoring the meta data in file system | |
| CN101739308B (en) | Image file generation method and its storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190910 |