CN111221801A

CN111221801A - A database migration method, system and related device

Info

Publication number: CN111221801A
Application number: CN201911332547.0A
Authority: CN
Inventors: 马国良
Original assignee: Suzhou Inspur Intelligent Technology Co Ltd
Current assignee: Suzhou Inspur Intelligent Technology Co Ltd
Priority date: 2020-03-02
Filing date: 2020-03-02
Publication date: 2020-06-02

Abstract

The application provides a database migration method, which comprises the following steps: when a migration instruction is received, determining SCN parameters; exporting the database full database data consistency logic of the time corresponding to the SCN parameter so as to enable a target database to recover the database full database data; determining data increment records of UNDO table space after the time corresponding to the SCN parameters; and exporting the data increment record, and stopping database business after the data increment record is exported, so that the target database utilizes the data increment record to perform data increment operation on the data of the whole database in time sequence. According to the method and the device, the time for the whole database data migration of the whole database is not required to be waited, and the downtime of the service system during the logical migration of the database is greatly reduced. The application also provides a database migration system, a computer readable storage medium and a database, which have the beneficial effects.

Description

Database migration method, system and related device

Technical Field

The present application relates to the field of databases, and in particular, to a method, a system, and a related device for database migration.

Background

The database migration is generally divided into a physical migration mode and a logical migration mode, the physical mode is generally to copy data files or data blocks of an original database, a physical structure of the original database is reserved after the migration, online migration can be realized, namely, the use of the original database is not affected or only shorter downtime is needed, but the migration requires that an original library host and a target library host must belong to the same hardware architecture and have the same operating system type, for example, the original library host and the target library host are required to be both in an X86 architecture, and the operating systems are both LINUX or WINDOWS, and the physical migration mode is not suitable for database migration across hardware architectures and different operating system types. Database migration across hardware architectures and different operating system types generally uses a logical migration mode to perform database migration, that is, data is exported from a source library and imported into a target library.

Therefore, how to reduce the influence on the business when the database is migrated is a technical problem that needs to be solved urgently by those skilled in the art.

Disclosure of Invention

The application aims to provide a database migration method, a database migration system, a computer readable storage medium and a database, which can reduce the influence on the service during the migration of the database.

In order to solve the technical problem, the application provides a database migration method, which has the following specific technical scheme:

when a migration instruction is received, determining SCN parameters;

exporting the database full database data consistency logic of the time corresponding to the SCN parameter so as to enable a target database to recover the database full database data;

determining data increment records of UNDO table space after the time corresponding to the SCN parameters;

and exporting the data increment record, and stopping database business after the data increment record is exported, so that the target database utilizes the data increment record to perform data increment operation on the data of the whole database in time sequence.

Wherein deriving the data delta record comprises:

exporting the data increment record in SCN intervals;

wherein, the starting point of the first SCN interval is the time corresponding to the SCN parameter, and the starting points of the rest SCN intervals are the time end points of the last SCN interval.

Wherein, when exporting the data incremental record, the method further comprises:

judging whether the whole database data of the database are completely exported or not;

if yes, stopping exporting the data increment record by taking the time cut-off point when all the database data of the database are exported.

Wherein the SCN parameters comprise a system check point SCN, a data file check point SCN, a start SCN and a stop SCN.

The present application further provides a database migration system, including:

the SCN determining module is used for determining SCN parameters when the migration instruction is received;

the data export module is used for exporting the database full database data consistency logic of the time corresponding to the SCN parameter so as to enable the target database to recover the database full database data;

the increment determining module is used for determining data increment records of the UNDO table space after the time corresponding to the SCN parameter;

and the increment export module is used for exporting the data increment record and stopping database business after the data increment record is exported so that the target database utilizes the data increment record to carry out data increment operation on the data of the whole database according to the time sequence.

Wherein the increment derivation module comprises:

the increment exporting unit is used for exporting the data increment record in SCN intervals;

Wherein, still include:

the judging module is used for judging whether all the database data of the database are exported; if yes, stopping exporting the data increment record by taking the time cut-off point when all the database data of the database are exported.

The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method as set forth above.

The present application also provides a database comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when calling the computer program in the memory.

The application provides a database migration method, which comprises the following steps: when a migration instruction is received, determining SCN parameters; exporting the database full database data consistency logic of the time corresponding to the SCN parameter so as to enable a target database to recover the database full database data; determining data increment records of UNDO table space after the time corresponding to the SCN parameters; and exporting the data increment record, and stopping database business after the data increment record is exported, so that the target database utilizes the data increment record to perform data increment operation on the data of the whole database in time sequence.

According to the method and the device, the SCN parameter of the database migration is firstly determined so as to ensure that all data are in the same SCN parameter consistent state when the database is subjected to full-database export, and the database can still be normally used outwards when the full-database export is carried out, so that data increment operation can be normally carried out. And then, incremental exporting is carried out on the database, and data incremental records after the time corresponding to the SCN parameters are exported. The database service is stopped after the data delta record is exported. And then, the data of the whole database is firstly recovered in the target database, and then the data increment operation is carried out by utilizing the data increment record. The stopping time of the data service in the whole process is equivalent to the time of data increment operation, the time for waiting the whole database data migration is not needed, the downtime of a service system during the logical migration of the database is greatly reduced, and the effective utilization rate of the database can be improved. The application also provides a database migration system, a computer readable storage medium and a database, which have the above beneficial effects and are not described herein again.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a flowchart of a database migration method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a database full database data consistency logic derivation process according to an embodiment of the present application;

FIG. 3 is a schematic diagram of data export results provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an incremental data record export process provided by an embodiment of the present application;

FIG. 5 is a diagram illustrating a process for recovering destination library data according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a database migration system according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, fig. 1 is a flowchart of a database migration method according to an embodiment of the present application, where the method includes:

s101: when a migration instruction is received, determining SCN parameters;

when a migration instruction is received, the database migration needs to be carried out. At this point the SCN parameter is determined. The SCN parameter is a number that is automatically maintained by the DBMS to accumulate increments when the Oracle database is updated. In Oracle, there are four SCN parameters, respectively: the method comprises the steps of checking a system check point SCN, checking a data file SCN, starting the SCN and stopping the SCN. Each SCN corresponds to a point in time. Therefore, the SCN parameter is determined, and it is the actual migration time point of the migration instruction.

It should be noted that the actual time cannot be selected as the migration time in this step. In a database, several data increment operations may occur even every second. It is difficult to directly use time as the migration time. For this purpose, the present application utilizes SCN as the migration time point.

S102: exporting the consistency logic of the database full database data at the time corresponding to the SCN parameter so as to enable the target database to recover the database full database data;

after the SCN parameter is determined, the database full database data consistency logic at the time corresponding to the SCN parameter is derived, which refers to the state of the database at the time corresponding to the SCN parameter, including all data before the time. And after all the data in the whole database are exported, the data can be recovered in the target database.

Therefore, when the source library is exported in a full library, scn parameters are designated, so that all data are in the same scn consistent state when the full library is exported. It should be noted that, since the whole-library export is actually the whole-library data of the database at the past time point and does not involve any data increment operation after the time point, the source library of the database can still be used normally during the whole-library export process.

S103: determining a data increment record of the UNDO table space after the time corresponding to the SCN parameter;

the UNDO table space is configured by the database, when the database performs table data addition and deletion, the data before modification is firstly reserved, and then the data before modification is modified in an addition and deletion manner, so that the purpose of the modification is to facilitate rollback operation and reading consistency, the UNDO table space is used for storing the data before modification, the storage time of the data is influenced by the storage time length parameter, the size of the UNDO table space and the like, and the changed data can be reserved for a longer time according to needs.

Since only the database full database data corresponding to the SCN parameter time is derived in S102, in practice, many data increment operations will occur after the event, so this step needs to determine the data increment record after the time.

It should be noted that the normal use of the database is not affected in the step of obtaining the data increment record.

S104: and exporting the data increment record, and stopping database business after the data increment record is exported so that the target database carries out data increment operation on the data of the whole database by using the data increment record according to the time sequence.

This step requires that incremental records of data be exported. The method for deriving the data increment record is not limited, for example, the data increment record may be derived in SCN intervals, such that the starting point of the first SCN interval is the time corresponding to the SCN parameter, the starting points of the rest SCN intervals are the time end points of the previous SCN interval, and the time end point of the last SCN interval is the time required to stop obtaining the data increment record. In other words, incremental data record derivation can be performed in several SCN intervals.

There is no limitation on when incremental recording of the lead-out data is stopped. In fact, since the whole database data of the database is already migrated to the destination database, the data increment operation only needs to be synchronized after the time corresponding to the SCN parameter, and therefore, when the export of the data increment record is stopped, the migration process of the database is not affected. Of course, in order to ensure the integrity of the data volume, the time for exporting the incremental data records should be stopped after all the database data in the database has been migrated to the destination library in step S102.

It is easy to understand that, because of the huge database data, it takes a long time to perform data migration, that is, step S102 actually takes a long time. However, the database can still be normally used for external use when S102 is executed, and in order to reduce the influence of database migration on data use, it may be determined whether all the database full database data is exported when the incremental record is exported, and if all the database full database data is exported, the export of the incremental record of data is stopped at the time cut-off point when all the database full database data is exported.

In other words, once it is detected that the migration of the database full library data is completed, the data increment record of the content of the time interval from the time corresponding to the SCN parameter to the time when the migration of the database full library data is completed may be derived, and of course, any other time after the migration of the database full library data is completed may be selected, which is not limited herein.

And stopping database business after the data increment record is exported, wherein the target database needs to perform one-to-one operation on the database data in the whole database according to the data increment probability to complete the migration of the database.

Therefore, the database migration method provided by the application substantially comprises three processes, namely database full-database data consistency logic migration, database increment consistency logic migration and database logic import. Firstly, determining the SCN parameter of the database migration to ensure that all data are in the consistent state of the same SCN parameter when the database is subjected to full-library export, and at the moment, the database can still be normally used outwards when the full-library export is performed, namely, the data increment operation can be normally performed. And then, incremental exporting is carried out on the database, and data incremental records after the time corresponding to the SCN parameters are exported. The database service is stopped after the data delta record is exported. And then, the data of the whole database is firstly recovered in the target database, and then the data increment operation is carried out by utilizing the data increment record. The stopping time of the data service in the whole process is equivalent to the time of data increment operation, the time for waiting the whole database data migration is not needed, the downtime of a service system during the logical migration of the database is greatly reduced, and the effective utilization rate of the database can be improved.

The following is a description of a specific application of a database migration method provided by the present application, which is described with reference to three processes of database full database data consistency logic migration, database incremental data consistency logic migration, and database logic import, specifically referring to fig. 2 to 5, fig. 2 is a schematic diagram of a process for exporting database full database data consistency logic provided by an embodiment of the present application, fig. 3 is a schematic diagram of a data export result provided by an embodiment of the present application, fig. 4 is a schematic diagram of a process for exporting incremental data records provided by an embodiment of the present application, and fig. 5 is a schematic diagram of a process for restoring target database data provided by an embodiment of the present application:

1) database full library consistency logic derivation

As shown in FIG. 2, there are two tables tab1 and tab2, each having three rows of data. At 11:17:10, the data in both tables is as shown in the uppermost part of FIG. 2, with an update operation at 11:17:20 modifying the data in the first row col1 column of table tab1 to 'am', and at 11:17:22, an update operation modifying the data in the second row col2 column of table tab2 to 25.

If parameter SCN is set to 11113 at 11:17:30 (corresponding to time 11:17:21), the full library logical export of the database is performed. 11:17:30 to 11:17:21, whether the table is modified or not is searched in the UNDO table space, and if the table is modified, the data originally recorded in the UNDO is reserved. After searching, it is found that table tab1 is not modified between 11:17:30 and 11:17:21, table tab2 is modified, and the export program will take out the recorded data before modification of table tab2 in the UNDO and return, so that tab1 is the modified data when the SCN is set to 11113 export, and tab2 is the data before modification.

2) Database incremental consistency logic exports

The data derived when the full library logic derived parameter SCN is set to 11113 is shown in fig. 3.

The SCN corresponding to 11:17:33 is 11118, the SCN during the last backup is 11113, and the increment export SCN parameter is set to 11118-.

Referring to fig. 4, the data for tab1 and tab2 has been changed between 11:17:21 and 11:17:33, looking up the UNDO table space in chronological order starting from 11:17:21 to 11:17:33, looking up the modified records for tab1 and tab2, finding three records in total: 26 before the second row col2 column of tab2 is modified, 25 after the second row col2 column is modified, 25 before the third row col3 column is modified, a male is modified, a female is modified, bb before the second row col2 column of tab1 is modified, bm after the third row col2 column is modified, the three modified records are exported in increment export, and multiple times of increment export can be carried out according to needs, wherein the starting point of each increment export is the end point of the last increment export, so that the continuity of increment export is ensured.

3) Database logical import

Referring to fig. 5, the database logic recovery firstly imports data exported by the database full-library consistency logic, and then imports data exported in increments in sequence.

And after the increment is imported, modifying the basic data according to the data change condition recorded in the increment in sequence according to time sequence to finally obtain the desired data.

According to the database migration method, scn parameters are designated when the source library is exported in a full library mode, all data are in the same scn consistent state when the full library is exported, the source library can be normally used outwards in the whole library exporting process, and then data incremental record exporting is carried out. All incremental record exports before the last increment do not influence the normal operation of the database service system, and the service system is required to be stopped only after the last incremental export to ensure the data of the database to be static, so the method and the system can realize the database migration and reduce the shutdown time of the service system during the logical migration as much as possible.

In the following, a database migration system provided in the embodiments of the present application is introduced, and the database migration system described below and the database migration method described above may be referred to correspondingly.

Referring to fig. 6, the present application further provides a database migration system, including:

the SCN determining module 100 is configured to determine an SCN parameter when the migration instruction is received;

the data export module 200 is configured to export the database full database data consistency logic of the time corresponding to the SCN parameter, so that the target database recovers the database full database data;

an increment determining module 300, configured to determine a data increment record of the UNDO table space after the time corresponding to the SCN parameter;

and an increment export module 400, configured to export the data increment record, and stop database service after the data increment record is exported, so that the target database performs data increment operation on the full database data by using the data increment record in time sequence.

Based on the above embodiment, as a preferred embodiment, the increment derivation module 400 includes:

Based on the above embodiment, as a preferred embodiment, the method further includes:

The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed, may implement the steps provided by the above-described embodiments. The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The application also provides a database, which may include a memory and a processor, wherein the memory stores a computer program, and the processor may implement the steps provided by the above embodiments when calling the computer program in the memory. Of course, the database may also include various network interfaces, power supplies, and the like.

The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system provided by the embodiment, the description is relatively simple because the system corresponds to the method provided by the embodiment, and the relevant points can be referred to the method part for description.

The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.

It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Claims

1. A method of database migration, comprising:

when a migration instruction is received, determining SCN parameters;

2. The database migration method according to claim 1, wherein exporting the data delta record comprises:

exporting the data increment record in SCN intervals;

3. The database migration method according to claim 1, wherein the exporting of the data incremental record further comprises:

4. The database migration method according to claim 1, wherein said SCN parameters include system checkpoint SCN, data file checkpoint SCN, start SCN and stop SCN.

5. A database migration system, comprising:

6. The database migration system according to claim 5, wherein said delta exporting module comprises:

7. The database migration system according to claim 5, further comprising:

8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.

9. A database comprising a memory in which a computer program is stored and a processor which, when called upon in the memory, implements the steps of the method according to any one of claims 1 to 4.