CN109491861B - Database repair method and device for overflow page exception and storage medium - Google Patents
Database repair method and device for overflow page exception and storage medium Download PDFInfo
- Publication number
- CN109491861B CN109491861B CN201811226067.1A CN201811226067A CN109491861B CN 109491861 B CN109491861 B CN 109491861B CN 201811226067 A CN201811226067 A CN 201811226067A CN 109491861 B CN109491861 B CN 109491861B
- Authority
- CN
- China
- Prior art keywords
- data
- index
- page
- abnormal
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000008439 repair process Effects 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000002159 abnormal effect Effects 0.000 claims abstract description 78
- 230000004048 modification Effects 0.000 claims description 11
- 238000012986 modification Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 2
- 239000012634 fragment Substances 0.000 abstract description 3
- 238000011084 recovery Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000015654 memory Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 239000002253 acid Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database repair method, a device and a storage medium for overflow page abnormity, wherein the method comprises the following steps: the method comprises the steps of obtaining abnormal information of an abnormal data node of a database overflow page; and repairing the database according to the repairing mode and the abnormal information. The restoration under the simple mode of the invention can realize the bypass restoration of abnormal data, and the overflow page data fragments before the abnormal point can be automatically restored under the standard restoration mode, i.e. the data before the abnormal point in one data record is restored, so as to restore the available data as much as possible, improve the integrity of the restored data as much as possible and improve the user experience.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a database repair method and device for overflow page abnormity and a storage medium.
Background
SQLite, a lightweight database, is an ACID compliant relational database management system that is contained in a relatively small C library. It is a public domain item established by d.richardhipp. The design target is embedded, and the embedded type memory is used in a plurality of embedded products at present, the occupied resource is very low, and only a few hundred K of memories are enough in the embedded type equipment. The system can support a mainstream operating system such as Windows/Linux/Unix and the like, can be combined with a plurality of programming languages such as Tcl, C #, PHP, Java and the like, has an ODBC interface, and has a processing speed higher than that of a world-known database management system with two types of sources such as Mysql and PostgreSQL. Consequently, the use of SQLite databases is becoming more widespread, particularly in portable devices.
However, because the stability of the portable device is relatively poor, for example, when data writing is performed, power is suddenly turned off, so that data abnormality of SQLite is caused, at present, in the aspect of abnormal repair of the SQLite database, at present, research at home and abroad is few, some database repair tools are attached to SQLite general analysis products in the market at present, but the limitation is large, data in the repaired database generally has some defects, that is, the database repair process is incomplete, and the condition that the records of the repaired database have great differences often occurs, so that the accuracy and the comprehensiveness of the database are affected. Therefore, how to accurately repair the data abnormality of the SQLite is a technical problem in the prior art.
Disclosure of Invention
The present invention provides the following technical solutions to overcome the above-mentioned drawbacks in the prior art.
A method of database repair of an overflow page exception, the method comprising:
the method comprises the steps of obtaining abnormal information of an abnormal data node of a database overflow page unit;
and repairing the database according to the repairing mode and the abnormal information.
Further, the exception information includes a page number start _ pgno of a first node of a linked list of the database overflow page unit, a unit start address start _ offset, a next unit start address next _ offset, and a page number abnormal _ pgno of a node where an exception occurs in the database overflow page, and a node number abnormal _ index, and the exception information is expressed as:
[start_pgno，start_offset，next_offset，abnormal_pgno，abnormal_index]，
if the first node is the last unit of the SQLite page, the next _ offset is equal to the usable _ size, which is the available size of the SQLite page.
Furthermore, the database is an SQLite database, the database comprises a plurality of SQLite pages, the SQLite pages are managed by adopting a B + tree, and the SQLite pages storing data in the B + tree are called leaf pages.
Furthermore, when the repair mode is simple repair, all the data records where the abnormal data nodes are located are modified to be null.
Further, when the repair mode is a standard repair, repairing the database includes:
truncating the abnormal data nodes, marking the overflow page number of the last data node before truncation as 0, and indicating the end of the overflow page linked list; modifying the head data of the data record where the abnormal data node is located; and calculating whether the space occupied by the head data of the data record is reduced or not, if so, moving the head data of the data record backwards, and correspondingly modifying the unit index in the page head corresponding to the data record.
Further, the last data node before truncation is denoted as [ pre _ pgno, pre _ index ], where pre _ pgno is the page number of the previous data node before the abnormal data node, and pre _ index is abrormal _ index-1; marking the overflow page number of the last data node before truncation as 0 means padding the overflow page number of 4 bytes before the pre _ pgno page to 0.
Furthermore, the operation of modifying the header data of the data record in which the abnormal data node is located is as follows:
reading original header data of a data record, the original header data including a total data record length TL 1 Total head length HL 1 And field INDEX INDEX i Wherein i is more than or equal to 0 and less than n, and n is the number of fields of the data record and is an integer;
calculating the size of the data domain before and after modification, wherein the usable size of the page of the database is recorded as usable _ size, and the total length of the data after truncation is recorded as DL 2 Total length of original data is DL 1 And then:
DL 1 ＝TL 1 -HL 1 ；
DL 2 ＝pre_index*(usable_size-4)+(next_offset-start_offset-4)-HL 1 ；
modifying header data of the truncated data record, the header data including a total record length TL 2 Total head length HL 2 And field INDEX NEW _ INDEX i ；
Assuming truncation from field m, f 1 (index i ) Is the length occupied by field i, f 2 (index i ) Index the length occupied by itself for field i, then:
f 1 (index i )＝0，(i＝m+1..n-1)；
further, the number of bytes cnt that need to be moved backward in calculating the modified data recording header data, wherein,
cnt＝f 2 (TL 1 )+f 1 (HL 1 )-f 2 (TL 2 )+f 1 (HL 2 )，
if the cnt is 0, the data recording head data does not need to be moved, the unit index does not need to be modified, and if the cnt is not 0, the modified data recording head data is moved backwards by cnt bytes so as to keep the close connection of the head field and the data field; and modifying the unit index of the page head corresponding to the data record according to the offset position of the modified data record head data, namely increasing the cnt bytes by the unit index value.
The invention also provides a database repair device for overflow page abnormity, which comprises:
the acquiring unit is used for acquiring the abnormal information of one abnormal data node of the database overflow page unit;
and the repairing unit is used for repairing the database according to the repairing mode and the abnormal information.
Further, the exception information includes a page number start _ pgno of a first node of a linked list of the database overflow page unit, a unit start address start _ offset, a next unit start address next _ offset, and a page number abnormal _ pgno of a node where an exception occurs in the database overflow page, and a node number abnormal _ index, and the exception information is expressed as:
[start_pgno，start_offset，next_offset，abnormal_pgno，abnormal_index]，
if the first node is the last unit of the SQLite page, the next _ offset is equal to the usable _ size, which is the available size of the SQLite page.
Furthermore, the database is an SQLite database, the database comprises a plurality of SQLite pages, the SQLite pages are managed by adopting a B + tree, and the SQLite pages storing data in the B + tree are called leaf pages.
Furthermore, when the repair mode is simple repair, all the data records where the abnormal data nodes are located are modified to be null.
Further, when the repair mode is a standard repair, repairing the database includes:
truncating the abnormal data nodes, marking the overflow page number of the last data node before truncation as 0, and indicating the end of the overflow page linked list; modifying the head data of the data record where the abnormal data node is located; and calculating whether the space occupied by the head data of the data record is reduced or not, if so, moving the head data of the data record backwards, and correspondingly modifying the unit index in the page head corresponding to the data record.
Further, the last data node before truncation is denoted as [ pre _ pgno, pre _ index ], where pre _ pgno is the page number of the previous data node before the abnormal data node, and pre _ index is abrormal _ index-1; marking the overflow page number of the last data node before truncation as 0 means padding the overflow page number of 4 bytes before the pre _ pgno page to 0.
Furthermore, the operation of modifying the header data of the data record in which the abnormal data node is located is as follows:
reading original header data of a data record, the original header data including a total data record length TL i Total head length HL 1 And field INDEX INDEX i Wherein i is more than or equal to 0 and less than n, and n is the number of fields of the data record and is an integer;
calculating the size of the data domain before and after modification, wherein the usable size of the page of the database is recorded as usable _ size, and the total length of the data after truncation is recorded as DL 2 Total length of original data is DL 1 And then:
DL 1 ＝TL 1 -HL 1 ；
DL 2 ＝pre_index*(usable_size-4)+(next_offset-start_offset-4)-HL 1 ；
modifying header data of the truncated data record, the header data including a total record length TL 2 Total head length HL 2 And field INDEX NEW _ INDEX i ；
Assuming truncation from field m, f 1 (index i ) Is the length occupied by field i, f 2 (index i ) Index the length occupied by itself for field i, then:
f 1 (index i )＝0，(i＝m+1..n-1)；
further, the number of bytes cnt that need to be moved backward in calculating the modified data recording header data, wherein,
cnt＝f 2 (TL 1 )+f 1 (HL 1 )-f 2 (TL 2 )+f 1 (HL 2 )，
if the cnt is 0, the data recording head data does not need to be moved, the unit index does not need to be modified, and if the cnt is not 0, the modified data recording head data is moved backwards by cnt bytes so as to keep the close connection of the head field and the data field; and modifying the unit index of the page head corresponding to the data record according to the offset position of the modified data record head data, namely increasing the cnt bytes by the unit index value.
The invention also proposes a computer-readable storage medium having stored thereon computer program code which, when executed by a computer, performs any of the methods described above.
The invention has the technical effects that: according to the method and the device, the abnormal information of one abnormal data node of the overflow page of the database is obtained, the database is repaired according to the repair mode and the abnormal information, the abnormal data can be repaired in the simple mode to bypass the repair, the data fragments of the overflow page before the abnormal point can be automatically repaired in the standard repair mode, namely the data in one data record before the abnormal node is repaired, the available data is recovered as much as possible, the integrity of the repaired data is improved as much as possible, and the user experience is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a flow diagram of a method of database repair of an overflow page exception, according to an embodiment of the present invention.
Figure 2 is a block diagram of a SQLite data record according to an embodiment of the present invention.
FIG. 3 is a block diagram of a database overflow page according to an embodiment of the present invention.
Fig. 4 is a block diagram of a database repair apparatus for an overflow page exception according to an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
FIG. 1 illustrates a database repair method for an overflow page exception according to the present invention, which includes:
an obtaining step S101, obtaining abnormal information of an abnormal data node of a database overflow page unit;
and a repairing step S102, repairing the database according to the repairing mode and the abnormal information.
In one embodiment of the method, the database adopted by the method is an SQLite database, the database comprises a plurality of SQLite pages, the SQLite pages are managed by adopting a B + tree, and the SQLite pages storing data in the B + tree are called leaf pages. The method is not limited to a certain database, is suitable for the abnormal repair analysis of the overflow pages of all SQLite databases, and is also suitable for the abnormal repair of the overflow pages of other databases similar to the SQLite databases.
The SQLite database file is composed of a series of data tables, each data table is composed of a series of pages, and the page organization is managed by a B + tree, each B + tree node corresponds to a SQLite page, the size of the page, page _ size, is the power of n of 2, and is commonly 1024, 2048, 4096 bytes, etc., the reserved part of the tail of the page, page _ size, which is deducted, is the available size of the page, usable _ size (in general, page _ size and usable _ size are equal). The leaf nodes where data is stored are referred to as leaf pages, and the internal nodes where indexes are stored are referred to as internal pages.
The SQLite Data record structure is shown in fig. 2, where the total record length (denoted as TL) and the ROWID (the ROWID is an id uniquely identifying the physical location of the record) are integers of variable length, the occupied space size is 1-9 bytes, the header Index length is the space size occupied by N indexes (denoted as Index) and the header Index length itself, the N Data (denoted as Data) fields are in one-to-one correspondence with the N Index fields, Index i Record DATA i The amount of space occupied.
The B + tree leaf pages internally organize data in units of cells (cells), one cell containing one (or part, when an overflow page is used) payload (also referred to as B + tree record data). Cells (cells) are of variable size and the page size is fixed, which makes it possible for a cell to be larger than a full page, and such a cell overflows onto a linked list of overflowing pages, as shown in fig. 3. In FIG. 3, the last cell of the B + tree page is oversized, requiring the use of overflow pages. At this time, the last 4 bytes of the cell are the page number of the 1 st overflow page in the overflow page chain table. For each overflow page, the first 4 bytes are the page number of the next overflow page, and when the value is 0, the page is the tail page of the table of the overflow page linked list. Each overflow page is filled with data (except for the first 4 bytes) in its entirety except for the last overflow page, and the last overflow page may have little data, even only one byte of data, but an overflow page will not store data from both cells.
In order to repair the overflow page exception, exception information needs to be obtained, where the exception information includes a page number start _ pgno, a unit start address start _ offset, a next unit start address next _ offset, and a page number abnormal _ pgno and a node sequence number abnormal _ index of a node where the database overflow page exception occurs, where the exception information is expressed as:
[ start _ pgno, start _ offset, next _ offset, absolute _ pgno, absolute _ index ], where if the first node is the last element of the SQLite page, next _ offset is equal to usable _ size, which is the available size of the SQLite page. One method for acquiring exception information is to perform traversal acquisition on the B + tree and the linked list shown in fig. 3, but those skilled in the art may also acquire exception information by other methods.
In an embodiment of the method, when the repair mode is simple repair, all data records where the abnormal data node is located are modified to be null, which is also called bypass mode, that is, all the data records are initialized to be null, that is, the data records can be accessed but have no data, so that it is possible to prevent that a report error occurs during database operation (such as query) and the query of data is interrupted, which is an inventive point of the present invention.
The more important invention point of the present invention is to recover the available data in the data record as much as possible, that is, recover the available data as much as possible, and improve the integrity of the recovered data to improve the user experience, when the recovery mode is a standard recovery, the standard recovery mode may also be called a 'minimally invasive' recovery, that is, the internal of the data node is recovered, and the recovering the database includes: truncating the abnormal data nodes, marking the overflow page number of the last data node before truncation as 0, and indicating the end of the overflow page linked list; modifying the head data of the data record where the abnormal data node is located; and calculating whether the space occupied by the head data of the data record is reduced or not, if so, moving the head data of the data record backwards, and correspondingly modifying the unit index in the page head corresponding to the data record.
In one embodiment, the last data node before truncation is denoted as [ pre _ pgno, pre _ index ], where pre _ pgno is the page number of the previous data node before the abnormal data node, and pre _ index is abrormal _ index-1; marking the overflow page number of the last data node before truncation as 0 means padding the overflow page number of 4 bytes before the pre _ pgno page to 0.
In one embodiment, the operation of modifying the header data of the data record in which the abnormal data node is located is: reading original header data of a data record, the original header data including a total data record length TL 1 Total head length HL 1 And field INDEX INDEX i Wherein i is more than or equal to 0 and less than n, and n is the number of fields of the data record and is an integer;
calculating the size of the data domain before and after modification, wherein the page available size of the database is recorded as usable _ size, the usable _ size and the page size are equal in most cases, and the total length of the data after truncation is recorded as DL 2 Total length of original data is DL 1 And then:
DL 1 ＝TL 1 -HL 1 ；
DL 2 ＝pre_index*(usable_size-4)+(next_offset-start_offset-4)-HL 1 ；
modifying header data of the truncated data record, the header data including a total record length TL 2 Total head length HL 2 And field INDEX NEW _ INDEX i ；
Assuming truncation from field m, f 1 (index i ) Is the length occupied by field i, f 2 (index i ) Index the length occupied by itself for field i, then:
f 1 (index i )＝0，(i＝m+1..n-1)；
in one embodiment, the number of bytes cnt that need to be moved back to calculate the modified data record header data, wherein,
cnt＝f 2 (TL 1 )+f 1 (HL 1 )-f 2 (TL 2 )+f 1 (HL 2 )，
if the cnt is 0, the data recording head data does not need to be moved, the unit index does not need to be modified, and if the cnt is not 0, the modified data recording head data is moved backwards by cnt bytes so as to keep the close connection of the head field and the data field; and according to the offset position of the modified data record head data, modifying the unit index of the page head corresponding to the data record, namely, increasing the cnt bytes by the unit index value.
With further reference to fig. 4, as an implementation of the method shown in fig. 1, the present application provides an embodiment of an apparatus for detecting an overflow page exception of a database, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 1, and the apparatus may be specifically included in various electronic devices.
FIG. 4 shows an apparatus for repairing a database with an overflow page exception according to the present invention, which includes:
an obtaining unit 401, configured to obtain exception information of an exception data node of the database overflow page unit;
a repairing unit 402, configured to repair the database according to a repairing mode and the exception information.
In an embodiment of the present invention, the database adopted in the present invention is an SQLite database, the database includes a plurality of SQLite pages, the SQLite pages are managed by a B + tree, and the SQLite pages storing data in the B + tree are called leaf pages. The device is not limited to a certain database, is suitable for the abnormal repair analysis of the overflow pages of all SQLite databases, and is also suitable for the abnormal repair of the overflow pages of other databases similar to the SQLite databases.
The SQLite database file is composed of a series of data tables, each data table is composed of a series of pages, and the page organization is managed by a B + tree, each B + tree node corresponds to a SQLite page, the size of the page, page _ size, is the power of n of 2, and is commonly 1024, 2048, 4096 bytes, etc., the reserved part of the tail of the page, page _ size, which is deducted, is the available size of the page, usable _ size (in general, page _ size and usable _ size are equal). The leaf nodes where data is stored are referred to as leaf pages, and the internal nodes where indexes are stored are referred to as internal pages.
The SQLite Data record structure is shown in fig. 2, where the total record length (denoted as TL) and the ROWID (the ROWID is an id uniquely identifying the physical location of the record) are integers of variable length, the occupied space size is 1-9 bytes, the header Index length is the space size occupied by N indexes (denoted as Index) and the header Index length itself, the N Data (denoted as Data) fields are in one-to-one correspondence with the N Index fields, Index i Record DATA i The amount of space occupied.
The B + tree leaf pages internally organize data in units of cells (cells), one cell containing one (or part, when an overflow page is used) payload (also referred to as B + tree record data). Cells (cells) are of variable size and the page size is fixed, which makes it possible for a cell to be larger than a full page, and such a cell overflows onto a linked list of overflowing pages, as shown in fig. 3. In FIG. 3, the last cell of the B + tree page is oversized, requiring the use of overflow pages. At this time, the last 4 bytes of the cell are the page number of the 1 st overflow page in the overflow page chain table. For each overflow page, the first 4 bytes are the page number of the next overflow page, and when the value is 0, the page is the tail page of the table of the overflow page linked list. Each overflow page is filled with data (except for the first 4 bytes) in its entirety except for the last overflow page, and the last overflow page may have little data, even only one byte of data, but an overflow page will not store data from both cells.
In order to repair an overflow page exception, exception information needs to be acquired, where the exception information includes a page number start _ pgno, a unit start address start _ offset, a next unit start address next _ offset of a first node in a linked list of a database overflow page unit, and a page number abnormal _ pgno and a node number abnormal _ index of a node where an exception occurs in the database overflow page, and the exception information is expressed as:
[ start _ pgno, start _ offset, next _ offset, absolute _ pgno, absolute _ index ], where if the first node is the last element of the SQLite page, next _ offset is equal to usable _ size, which is the available size of the SQLite page. One device for acquiring exception information is to perform traversal acquisition on a B + tree and the linked list shown in fig. 3, but those skilled in the art may acquire the exception information by other devices.
In an embodiment of the present invention, when the repair mode is simple repair, all data records where the abnormal data node is located are modified to be null, which is also referred to as bypass mode, that is, all the data records are initialized to be null, that is, the data records can be accessed but do not have any data, so that it is possible to prevent that a fault is reported during database operation (such as query) and the query of data is interrupted, which is an inventive point of the present invention.
The more important invention point of the present invention is to recover the available data in the data record as much as possible, that is, recover the available data as much as possible, and improve the integrity of the recovered data to improve the user experience, when the recovery mode is a standard recovery, the standard recovery mode may also be called a 'minimally invasive' recovery, that is, the internal of the data node is recovered, and the recovering the database includes: truncating the abnormal data nodes, marking the overflow page number of the last data node before truncation as 0, and indicating the end of the overflow page linked list; modifying the head data of the data record where the abnormal data node is located; and calculating whether the space occupied by the head data of the data record is reduced or not, if so, moving the head data of the data record backwards, and correspondingly modifying the unit index in the page head corresponding to the data record.
In one embodiment, the last data node before truncation is denoted as [ pre _ pgno, pre _ index ], where pre _ pgno is the page number of the previous data node before the abnormal data node, and pre _ index is abrormal _ index-1; marking the overflow page number of the last data node before truncation as 0 means padding the overflow page number of 4 bytes before the pre _ pgno page to 0.
In one embodiment, the operation of modifying the header data of the data record in which the abnormal data node is located is: reading original header data of a data record, the original header data comprisingTotal data recording length TL 1 Total head length HL 1 And field INDEX INDEX i Wherein i is more than or equal to 0 and less than n, and n is the number of fields of the data record and is an integer;
calculating the size of the data domain before and after modification, wherein the page available size of the database is recorded as usable _ size, the usable _ size and the page size are equal in most cases, and the total length of the data after truncation is recorded as DL 2 Total length of original data is DL 1 And then:
DL 1 ＝TL 1 -HL 1 ；
DL 2 ＝pre_index*(usable_size-4)+(next_offset-start_offset-4)-HL 1 ；
modifying header data of the truncated data record, the header data including a total record length TL 2 Total head length HL 2 And field INDEX NEW _ INDEX i ；
Assuming truncation from field m, f 1 (index i ) Is the length occupied by field i, f 2 (index i ) Index the length occupied by itself for field i, then:
f 1 (index i )＝0，(i＝m+1..n-1)；
in one embodiment, the number of bytes cnt that need to be moved back to calculate the modified data record header data, wherein,
cnt＝f 2 (TL 1 )+f 1 (HL 1 )-f 2 (TL 2 )+f 1 (HL 2 )，
if the cnt is 0, the data recording head data does not need to be moved, the unit index does not need to be modified, and if the cnt is not 0, the modified data recording head data is moved backwards by cnt bytes so as to keep the close connection of the head field and the data field; and modifying the unit index of the page head corresponding to the data record according to the offset position of the modified data record head data, namely increasing the cnt bytes by the unit index value.
The method has the technical effects that the abnormal data can be repaired in a simple mode to bypass the repair, the overflowing page data fragments before the abnormal point can be automatically repaired in a standard repair mode, namely, the data before the abnormal point in one data record is repaired, the available data is recovered as much as possible, the integrity of the repaired data is improved as much as possible, and the user experience is improved.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
Finally, it should be noted that: although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that: modifications and equivalents may be made thereto without departing from the spirit and scope of the invention and it is intended to cover in the claims the invention as defined in the appended claims.
Claims (13)
1. A method for database repair of an overflow page exception, the method comprising:
the method comprises the steps of obtaining abnormal information of an abnormal data node of a database overflow page unit;
a repairing step, repairing the database according to a repairing mode and the abnormal information;
when the repair mode is simple repair, all data records where the abnormal data nodes are located are modified to be null;
when the repair mode is a standard repair, repairing the database comprises:
truncating the abnormal data nodes, marking the overflow page number of the last data node before truncation as 0, and indicating the end of the overflow page linked list; modifying the head data of the data record where the abnormal data node is located; calculating the total head length of the modified data record, comparing the total head length of the data record before modification, if the total head length of the data record after modification is reduced, moving the head data of the data record backwards to the previous byte of the data field, and correspondingly modifying the unit index in the page head corresponding to the data record into a new offset address of the data record, so that the modified data record becomes the data record conforming to the SQLite coding rule, otherwise, the head data does not need to be moved backwards and the page head unit index does not need to be modified.
2. The method of claim 1, wherein the exception information includes a page number start _ pgno, a unit start address start _ offset, a next unit start address next _ offset of a first node of a linked list of database overflow page units, and a page number abnormal _ pgno, a node sequence number abnormal _ index of a node where an exception occurs in a database overflow page, and wherein the exception information is expressed as:
[start_pgno,start_offset,next_offset,abnormal_pgno,abno rmal_index]，
if the first node is the last unit of the SQLite page, the next _ offset is equal to the usable _ size, which is the available size of the SQLite page.
3. The method according to claim 2, wherein the database is an SQLite database, the database comprises a plurality of SQLite pages, the SQLite pages are managed by adopting a B + tree, and the SQLite pages storing data in the B + tree are called leaf pages.
4. The method of claim 1, wherein the last data node before truncation is denoted as [ pre _ pgno, pre _ index ], where pre _ pgno is the page number of the previous data node before the abnormal data node, and pre _ index is abnormal _ index-1; marking the overflow page number of the last data node before truncation as 0 means padding the overflow page number of 4 bytes before the pre _ pgno page to 0.
5. The method of claim 4, wherein the operation of modifying the header data of the data record in which the abnormal data node is located is:
reading original header data of a data record, the original header data including a total data record length TL 1 Total head length HL 1 And field INDEX INDEX i Wherein i is more than or equal to 0 and less than n, and n is the number of fields of the data record and is an integer;
calculating the size of the data domain before and after modification, wherein the usable size of the page of the database is recorded as usable _ size, and the total length of the data after truncation is recorded as DL 2 Total length of original data is DL 1 And then:
DL 1 ＝TL 1 -HL 1 ；
DL 2 ＝pre_index*(usable_size-4)+(next_offset-start_offset-4)-HL 1 ；
modifying header data of the truncated data record, the header data including a total record length TL 2 Head assemblyLength HL 2 And field INDEX NEW _ INDEX i ；
Assuming truncation from field m, f 1 (index i ) Is the length occupied by field i, f 2 (index i ) Index the length occupied by itself for field i, then:
f 1 (index i )＝0,(i＝m+1..n-1)；
6. the method of claim 5,
and calculating the byte count cnt which needs to be moved backwards by the modified data recording head data, wherein,
cnt＝f 2 (TL 1 )+f 1 (HL 1 )-f 2 (TL 2 )+f 1 (HL 2 )，
if the cnt is 0, the data recording head data does not need to be moved, the unit index does not need to be modified, and if the cnt is not 0, the modified data recording head data is moved backwards by cnt bytes so as to keep the close connection of the head field and the data field; and modifying the unit index of the page head corresponding to the data record according to the offset position of the modified data record head data, namely increasing the cnt bytes by the unit index value.
7. An apparatus for database repair of overflow page exceptions, the apparatus comprising:
the acquiring unit is used for acquiring the abnormal information of an abnormal data node of the database overflow page unit;
the repairing unit is used for repairing the database according to a repairing mode and the abnormal information;
when the repair mode is simple repair, all data records where the abnormal data nodes are located are modified to be empty;
when the repair mode is a standard repair, repairing the database comprises:
truncating the abnormal data nodes, marking the overflow page number of the last data node before truncation as 0, and indicating the end of the overflow page linked list; modifying the head data of the data record where the abnormal data node is located; calculating the total head length of the modified data record, comparing the total head length of the data record before modification, if the total head length of the data record after modification is reduced, moving the head data of the data record backwards to the previous byte of the data field, and correspondingly modifying the unit index in the page head corresponding to the data record into a new offset address of the data record, so that the modified data record becomes the data record conforming to the SQLite coding rule, otherwise, the head data does not need to be moved backwards and the page head unit index does not need to be modified.
8. The apparatus of claim 7, wherein the exception information comprises a page number start _ pgno, a unit start address start _ offset, a next unit start address next _ offset of a first node of a linked list of database overflow page units, and a page number abnormal _ pgno, a node sequence number abnormal _ index of a node where the database overflow page occurs an exception, and wherein the exception information is expressed as:
[start_pgno,start_offset,next_offset,abnormal_pgno,abnormal_index]，
if the first node is the last unit of the SQLite page, the next _ offset is equal to the usable _ size, which is the available size of the SQLite page.
9. The device of claim 8, wherein the database is an SQLite database, and the database comprises a plurality of SQLite pages, and the SQLite pages are managed by adopting a B + tree, wherein the SQLite pages storing data in the B + tree are called leaf pages.
10. The apparatus of claim 7, wherein the last data node before truncation is denoted as [ pre _ pgno, pre _ index ], where pre _ pgno is the page number of the previous data node before the abnormal data node, and pre _ index is abrormal _ index-1; marking the overflow page number of the last data node before truncation as 0 means padding the overflow page number of 4 bytes before the pre _ pgno page to 0.
11. The apparatus according to claim 10, wherein the operation of modifying the header data of the data record in which the abnormal data node is located is:
reading original header data of a data record, the original header data including a total data record length TL 1 Total head length HL 1 And field INDEX INDEX i Wherein i is more than or equal to 0 and less than n, and n is the number of fields of the data record and is an integer;
calculating the size of the data domain before and after modification, wherein the usable size of the page of the database is recorded as usable _ size, and the total length of the data after truncation is recorded as DL 2 Total length of original data is DL 1 And then:
DL 1 ＝TL 1 -HL 1 ；
DL 2 ＝pre_index*(usable_size-4)+(next_offset-start_offset-4)-HL 1 ；
modifying header data of the truncated data record, the header data including a total record length TL 2 Total head length HL 2 And field INDEX NEW _ INDEX i ；
Assuming truncation from field m, f 1 (index i ) Is the length occupied by field i, f 2 (index i ) Index the length occupied by itself for field i, then:
f 1 (index i )＝0,(i＝m+1..n-1)；
12. the apparatus of claim 11,
and calculating the byte count cnt which needs to be moved backwards by the modified data recording head data, wherein,
cnt＝f 2 (TL 1 )+f 1 (HL 1 )-f 2 (TL 2 )+f 1 (HL 2 )，
if the cnt is 0, the data recording head data does not need to be moved, the unit index does not need to be modified, and if the cnt is not 0, the modified data recording head data is moved backwards by cnt bytes so as to keep the close connection of the head field and the data field; and modifying the unit index of the page head corresponding to the data record according to the offset position of the modified data record head data, namely increasing the cnt bytes by the unit index value.
13. A computer-readable storage medium, characterized in that the storage medium has stored thereon computer program code which, when executed by a computer, performs the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811226067.1A CN109491861B (en) | 2018-10-19 | 2018-10-19 | Database repair method and device for overflow page exception and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811226067.1A CN109491861B (en) | 2018-10-19 | 2018-10-19 | Database repair method and device for overflow page exception and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109491861A CN109491861A (en) | 2019-03-19 |
| CN109491861B true CN109491861B (en) | 2022-08-12 |
Family
ID=65692279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811226067.1A Active CN109491861B (en) | 2018-10-19 | 2018-10-19 | Database repair method and device for overflow page exception and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109491861B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112579336B (en) * | 2020-12-31 | 2022-04-22 | 厦门市美亚柏科信息股份有限公司 | Database exception recovery method and device, computing device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH04257037A (en) * | 1991-02-08 | 1992-09-11 | Nec Corp | Data base stored failure page erase/restore system |
| EP1209569A1 (en) * | 2000-04-12 | 2002-05-29 | Annex Systems Incorporated | Data backup/recovery system |
| AU2007202588A1 (en) * | 1998-02-13 | 2007-06-28 | Oracle International Corporation | Method and apparatus for transferring data from the cache of one node to the cache of another node |
| CN106599048A (en) * | 2016-11-10 | 2017-04-26 | 厦门安胜网络科技有限公司 | Method and device for recovering deleted records of SQLite database file |
| CN107193871A (en) * | 2017-04-12 | 2017-09-22 | 腾讯科技（深圳）有限公司 | Database recovering method and device |
| CN108062358A (en) * | 2017-11-28 | 2018-05-22 | 厦门市美亚柏科信息股份有限公司 | The offline restoration methods of innodb engine deletion records, storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2302981C (en) * | 2000-03-29 | 2004-06-01 | Ibm Canada Limited-Ibm Canada Limitee | Online database table reorganization |
-
2018
- 2018-10-19 CN CN201811226067.1A patent/CN109491861B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH04257037A (en) * | 1991-02-08 | 1992-09-11 | Nec Corp | Data base stored failure page erase/restore system |
| AU2007202588A1 (en) * | 1998-02-13 | 2007-06-28 | Oracle International Corporation | Method and apparatus for transferring data from the cache of one node to the cache of another node |
| EP1209569A1 (en) * | 2000-04-12 | 2002-05-29 | Annex Systems Incorporated | Data backup/recovery system |
| CN106599048A (en) * | 2016-11-10 | 2017-04-26 | 厦门安胜网络科技有限公司 | Method and device for recovering deleted records of SQLite database file |
| CN107193871A (en) * | 2017-04-12 | 2017-09-22 | 腾讯科技（深圳）有限公司 | Database recovering method and device |
| CN108062358A (en) * | 2017-11-28 | 2018-05-22 | 厦门市美亚柏科信息股份有限公司 | The offline restoration methods of innodb engine deletion records, storage medium |
Non-Patent Citations (2)
| Title |
|---|
| A Recovery Scheme for the Deleted Overflow Data in SQLite Database;Lee, Gyu-Won 等;《Journal of Korean Institute of Information Technology》;20120101;第10卷(第11期);全文 * |
| iOS平台下微信删除恢复的原理研究及技术实验;马梦笔;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20150915(第09期);论文第26-38页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109491861A (en) | 2019-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8396839B1 (en) | Representing de-duplicated file data | |
| US10817469B2 (en) | Method and apparatus for repairing file system directory tree | |
| US10191693B2 (en) | Performing updates on variable-length data sequentially stored and indexed to facilitate reverse reading | |
| US20170371551A1 (en) | Capturing snapshots of variable-length data sequentially stored and indexed to facilitate reverse reading | |
| US8595248B2 (en) | Querying a cascading index that avoids disk accesses | |
| US8290915B2 (en) | Retrieval and recovery of data chunks from alternate data stores in a deduplicating system | |
| CN111104377B (en) | File management method, electronic device and computer readable storage medium | |
| CN110532136B (en) | Raid data recovery method based on metadata | |
| CN113360322B (en) | Method and equipment for recovering data based on backup system | |
| CN107111460A (en) | Deduplication Using Chunk Files | |
| CN111125298A (en) | Method, equipment and storage medium for reconstructing NTFS file directory tree | |
| WO2020119143A1 (en) | Database deleted record recovery method and system | |
| CN112328631A (en) | Production fault analysis method and device, electronic equipment and storage medium | |
| CN113419897B (en) | File processing method and device, electronic equipment and storage medium thereof | |
| Li et al. | Database management strategy and recovery methods of Android | |
| US11977460B1 (en) | File system namespace reconstruction | |
| US9483485B1 (en) | Restore database from backup system where full and incremental backups are performed simultaneously | |
| CN109491861B (en) | Database repair method and device for overflow page exception and storage medium | |
| CN107329966B (en) | Machine data storage method and system | |
| CN112698866B (en) | A Git-based code line life cycle tracing method and electronic device | |
| CN113360495B (en) | A recovery method, device, equipment and readable medium for database query interruption | |
| KR101670473B1 (en) | Method for recovering deleted data by mysql innodb database | |
| CN118012656A (en) | Damaged PDF document repairing method, device, equipment and storage medium | |
| US20180039663A1 (en) | Performing set operations on variable-length data sequentially stored and indexed to facilitate reverse reading | |
| CN112579612A (en) | Database index table record analysis method and device, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |