CN111475334B

CN111475334B - TiDB database maintenance method, device, computer equipment and storage medium

Info

Publication number: CN111475334B
Application number: CN202010150801.1A
Authority: CN
Inventors: 汤盛华
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2020-03-05
Filing date: 2020-03-05
Publication date: 2025-06-24
Anticipated expiration: 2040-03-05
Also published as: CN111475334A

Abstract

The invention discloses a maintenance method, a device, computer equipment and a storage medium of a TiDB database, wherein the method comprises the steps of collecting data in a TiDB production library through a TiDB-Binlog cluster component, obtaining a Binlog file corresponding to the data in the TiDB production library, synchronizing the Binlog file to a TiDB same-city disaster backup library, synchronizing the Binlog file in the TiDB same-city disaster backup library to a flash memory NAS device by adopting a preset timing backup script, carrying out data reduction processing on the TiDB production library based on the Binlog file stored in the flash memory NAS device when monitoring that the TiDB production library is abnormal, avoiding the problem of slow backup and reduction speed caused by directly storing the data in the production library in an archiving tape in a traditional mode, and improving the maintenance efficiency of the TiDB database.

Description

TiDB database maintenance method, tiDB database maintenance device, computer equipment and storage medium

Technical Field

The present invention relates to the field of data processing, and in particular, to a method and apparatus for maintaining TiDB database, a computer device, and a storage medium.

Background

Along with the rapid development of economy, the service range of more and more enterprises and institutions is wider and more, the user groups are wider and wider, and a plurality of enterprises begin to adopt distributed databases for data storage so as to improve the service processing capacity, tiDB is used as a database with an open source distributed relation, and the database has higher safety, is compatible with MySQL protocol and ecology, is convenient to migrate and has extremely low operation and maintenance cost, thus becoming a popular distributed database.

When the database performs transaction processing, data backup is needed, so that when faults occur, data recovery and restoration processing is performed according to backup files, the existing TiDB database performs data backup through an archiving tape, the IO transmission capacity of the archiving tape is weak, and the TiDB database is large in data size, so that the data backup and restoration time is long, and the maintenance efficiency of the database is low.

Disclosure of Invention

The embodiment of the invention provides a method, a device, computer equipment and a storage medium for maintaining TiDB databases, so as to improve the maintenance efficiency of TiDB databases.

In order to solve the above technical problems, an embodiment of the present application provides a method for maintaining TiDB databases, including:

collecting data in a TiDB production library through TiDB-Binlog cluster components, obtaining a Binlog file corresponding to the data in the TiDB production library, and synchronizing the Binlog file to a TiDB same-city disaster backup library;

Synchronizing the binlog files in the TiDB same-city disaster recovery library to a flash NAS device by adopting a preset timing backup script;

if TiDB production library abnormality is monitored, carrying out data reduction processing on the TiDB production library based on the binlog file stored in the flash NAS device.

Optionally, the TiDB-Binlog cluster component includes a Drainer component and a pump component, the collecting TiDB data in the production library by the TiDB-Binlog cluster component, obtaining a Binlog file corresponding to the data in the TiDB production library, and synchronizing the Binlog file to the TiDB same-city disaster backup library includes:

Collecting binglog files of a TiDB node server corresponding to the pump component through the pump component, taking the binglog files as basic files, and sending the basic files to the Drainer component;

receiving the base file through the Drainer component and merging the base file to obtain a merged binglog file;

and synchronizing the merged binglog files to the TiDB same-city disaster backup library.

Optionally, the flash NAS device includes a first flash device and a second flash device.

Optionally, the synchronizing the binlog file in the TiDB same-city disaster recovery library to the flash NAS device using a preset timing backup script includes:

moving and storing the binlog file in the TiDB same-city disaster recovery library into the first flash memory device;

Performing incremental and full difference verification on the first flash memory device and the second flash memory device at regular time to obtain a verification result;

And if the verification result is that the difference exists, updating the data in the second flash memory device by adopting the data in the first flash memory device according to the difference in the verification result.

Optionally, the moving and saving the binlog file in the TiDB co-city disaster recovery repository to the first flash memory device includes:

taking the binlog file in the TiDB same-city disaster backup library as data to be imported, and acquiring a storage path of the first flash memory device corresponding to the data to be imported;

Determining the number of target threads based on the data to be imported and the first flash memory device;

and constructing multiple threads based on the target thread number, and synchronizing the data to be imported to a storage path of the first flash memory device by adopting the multiple threads.

Optionally, if the abnormality of the production library is monitored TiDB, performing data reduction processing on the TiDB production library based on the binlog file stored in the flash NAS device includes:

Stopping the application service from writing the table of the TiDB production library, and clearing the data in the data table of the TiDB production library;

Acquiring a full backup file and an incremental backup file from the flash NAS device as files to be restored;

loading the full backup file into a data table in a TiDB production library, and loading the backup file into a data table in the TiDB production library;

And verifying the data in the data table in the TiDB production library, confirming that the data recovery is successful after the verification is passed, and starting the application service.

In order to solve the above technical problem, an embodiment of the present application further provides a TiDB database maintenance device, including:

The data acquisition module is used for acquiring data in a TiDB production library through TiDB-Binlog cluster components, obtaining a Binlog file corresponding to the data in the TiDB production library, and synchronizing the Binlog file to a TiDB same-city disaster backup library;

The file synchronization module is used for synchronizing the binlog files in the TiDB same-city disaster backup library to the flash NAS equipment by adopting a preset timing backup script;

and the data recovery module is used for carrying out data recovery processing on the TiDB production library based on the binlog file stored in the flash NAS device if the TiDB production library is monitored to be abnormal.

Optionally, the data acquisition module includes:

The collecting unit is used for collecting binglog files of the TiDB node server corresponding to the pump component through the pump component, taking the binglog files as basic files and sending the basic files to the Drainer component;

the merging unit is used for receiving the basic file through the Drainer component and merging the basic file to obtain a merged binglog file;

and the synchronization unit is used for synchronizing the binglog files after combination to the TiDB same-city disaster backup library.

Optionally, the flash NAS device includes a first flash device and a second flash device, and the file synchronization module includes:

the synchronization unit is used for moving the binlog files in the TiDB same-city disaster backup library and storing the binlog files into the first flash memory device;

The verification unit is used for carrying out incremental and full difference verification on the first flash memory device and the second flash memory device at regular time to obtain a verification result;

And the updating unit is used for updating the data in the second flash memory device by adopting the data in the first flash memory device according to the difference in the verification result if the verification result is that the difference exists.

Optionally, the synchronization unit includes:

a path obtaining subunit, configured to use a binlog file in the TiDB same-city disaster backup repository as data to be imported, and obtain a storage path of the first flash memory device corresponding to the data to be imported;

A thread number determining subunit, configured to determine the number of target threads based on the data to be imported and the first flash device;

and the multithreading synchronization subunit is used for constructing multithreading based on the target thread number and synchronizing the data to be imported to a storage path of the first flash memory device by adopting the multithreading.

Optionally, the data recovery module includes:

the data cleaning unit is used for stopping the writing list processing of the TiDB production library by the application service and cleaning the data in the data list of the TiDB production library;

The file to be restored determining unit is used for acquiring the full backup file and the backup adding file from the flash NAS equipment to serve as files to be restored;

the file restoring unit is used for loading the full backup file into a data table in the TiDB production library, and then loading the backup file into a data table in the TiDB production library;

and the data verification unit is used for verifying the data in the data table in the TiDB production library, confirming that the data recovery is successful after the verification is passed, and starting the application service.

In order to solve the above technical problem, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the steps of the method for maintaining a TiDB database described above when executing the computer program.

To solve the above technical problem, an embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program implements the steps of the method for maintaining a TiDB database described above when executed by a processor.

According to the method, the device, the computer equipment and the storage medium for maintaining the TiDB database, the data in the TiDB production library is collected through the TiDB-Binlog cluster component, the Binlog file corresponding to the data in the TiDB production library is obtained, the Binlog file is synchronized to the TiDB same-city disaster backup library, the Binlog file in the TiDB same-city disaster backup library is synchronized to the flash memory NAS equipment by adopting a preset timing backup script, the abnormality of the TiDB production library is monitored, the data reduction processing is carried out on the TiDB production library based on the Binlog file stored in the flash memory NAS equipment, the problem that the backup and reduction speed is slow due to the fact that the data in the production library are directly stored in an archiving tape in a traditional mode is avoided, and the maintenance efficiency of the database is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of one embodiment of a method of maintenance of TiDB databases of the present application;

FIG. 3 is a schematic diagram of one embodiment of a maintenance device for TiDB databases in accordance with the present application;

FIG. 4 is a schematic structural diagram of one embodiment of a computer device in accordance with the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs, the terms used in the description herein are used for the purpose of describing particular embodiments only and are not intended to limit the application, and the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the above description of the drawings are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1, as shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like.

The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture E interface display perts Group Audio Layer III, moving Picture expert compression standard audio plane 3), MP4 players (Moving Picture E interface display perts Group Audio Layer IV, moving Picture expert compression standard audio plane 4), laptop and desktop computers, and so on.

The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.

It should be noted that, the method for maintaining the TiDB database provided by the embodiment of the present application is executed by the server, and accordingly, the device for maintaining the TiDB database is disposed in the server.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. Any number of terminal devices, networks and servers may be provided according to implementation requirements, and the terminal devices 101, 102, 103 in the embodiment of the present application may specifically correspond to application systems in actual production.

Referring to fig. 2, fig. 2 shows a method for maintaining TiDB a database according to an embodiment of the present invention, and the method is applied to the server in fig. 1 for illustration, and is described in detail as follows:

S201, collecting data in a TiDB production library through TiDB-Binlog cluster components to obtain a Binlog file corresponding to the data in the TiDB production library, and synchronizing the Binlog file to a TiDB same-city disaster backup library.

The TiDB production library is used in production environment and based on TiDB open source distributed relational database, tiDB is a fusion type database product positioned in on-line transaction processing/on-line analysis processing (HTAP: hybrid Transactional/ANALYTICAL PROCESSING), and the important characteristics of one-key horizontal expansion, strong consistency, multi-copy data security, distributed transaction, real-time OLAP and the like are realized. Meanwhile, the method is compatible with MySQL protocol and ecology, is convenient to migrate and has extremely low operation and maintenance cost.

Specifically, in this embodiment, a plurality of TiDB node servers are included, and TiDB node servers perform data update (including but not limited to addition, deletion, modification, query, backup, etc.) during operation, obtain updated data in each TiDB node server through a TiDB-Binlog cluster component, generate corresponding Binlog files according to the updated data, and synchronize the Binlog files to TiDB same-city disaster backup libraries.

Wherein TiDB is the backup database of the same area based on TiDB.

The binlog file is a binary log file, and is used for recording information of SQL sentences updated by a user on a database, for example, SQL sentences for changing a database table and changing contents are recorded in binlog, but query on contents such as a database table is not recorded. By default, binlog files are in binary format and cannot be viewed using commands (e.g., cat, vi, etc.) to view text tools, but are viewed using mysqlbinlog parsing.

When data is written into the database, the updated SQL statement is written into the corresponding binlog file at the same time, and when dump backup is used, only data in a period of time is fully prepared.

S202, synchronizing a binlog file in TiDB same-city disaster recovery libraries to the flash NAS device by adopting a preset timing backup script.

The existing plug-in Storage is divided into Direct-Attached Storage (DAS) and networked Storage (FAS) according to a connection mode, and the networked Storage is further divided into Network-Attached Storage (NAS) and Storage area Network (Storage Area Network SAN) according to a transmission protocol.

Among them, the flash NAS device is a device connected to a network and having a data storage function, and is also called a "network storage". It is a special dedicated data storage server that includes storage devices (e.g., disk arrays, CD/DVD drives, tape drives, or removable storage media) and embedded system software that provides cross-platform file sharing functionality. In this configuration, the NAS centrally manages and processes all data on the network, and loads are offloaded from the application or enterprise server, thus effectively reducing the total ownership cost and protecting user investment. The NAS itself can support multiple protocols (e.g., NFS, CIFS, FTP, HTTP, etc.) and can support various operating systems. Through any workstation, the NAS device can be intuitively and conveniently managed by adopting an IE or Netscape browser. Its cost is far lower than using server storage, while its efficiency is far higher than the latter.

The NAS device is a true plug and play product, so the NAS device is also called a flash memory NAS device, a multi-computer platform is generally supported, users can enter the same document through a network supporting protocol, the NAS device can be used in a mixed Unix/Windows NT local area network without modification, meanwhile, the NAS device is very flexible to apply, and compared with other plug-in storage modes, the flash memory NAS device is a key problem, so that the bandwidth consumption in the backup process is realized. Unlike a Storage Area Network (SAN) in which backup data streams are transferred out of a LAN, the NAS still uses the network for backup and restoration, and thus, data backup and restoration can be performed in a multithreading manner, and data restoration and backup efficiency can be improved.

The preset timing backup script is used for timing synchronization of the binlog files, for example, in drainer points per day through the timing backup script, the binlog files with the creation time of the previous day are transferred to the flash memory NAS device, and the types of the preset timing backup script include but are not limited to shell script, javaScript script, lua script, python script and the like.

In the existing TiDB database backup method, when a production library fails, full-backup files and incremental-backup files are required to be restored to a local server through an archiving tape and then imported into a database, because the full-backup files of the database which are required to be restored from the archiving tape are generally larger (the size of the database is different, generally tens of G to several T), the tape IO capacity is weaker, the time for acquiring the restored files is longer, the application stop time is prolonged, and the continuous operation of a service system is seriously influenced.

It should be noted that, in this embodiment, after the binlog file is moved and saved to the flash NAS device, the data stored in the flash NAS device is further synchronized to the archive tape, so that the data can be saved and read offline.

The archiving tape is also called as tape memory (MAGNETIC TAPE storage), which is a storage device composed of tape drive and its controller, and is an auxiliary memory of computer. The tape drive is composed of tape drive mechanism and magnetic head, and can drive the tape to move relative to the magnetic head, and uses the magnetic head to make electromagnetic conversion, and can sequentially record or read data on the tape. Tape storage is one of the computer peripherals. The tape controller is a control circuit device for the central processing unit to access data on the tape drive. Tape storage accesses data in a sequential manner. The magnetic tape storing the data may be stored off-line and read interchangeably.

And S203, if the abnormality of the TiDB production library is monitored, carrying out data reduction processing on the TiDB production library based on the binlog file stored in the flash NAS device.

Specifically, the server side is provided with configuration application log monitoring and database performance index monitoring, the monitoring monitors various data in TiDB production libraries in real time, when partial data in TiDB production libraries exceeds a normal preset range, the abnormality of TiDB production libraries is confirmed, at the moment, alarm prompt is carried out through mails or telephones, and data reduction processing is carried out through binlog files stored in flash NAS equipment.

For the specific process of performing data reduction processing through the binlog file stored in the flash NAS device, reference may be made to the description of the subsequent embodiments, and for avoiding repetition, details are not repeated here.

In this embodiment, the TiDB-Binlog cluster component collects data in the TiDB production library to obtain a Binlog file corresponding to the data in the TiDB production library, synchronizes the Binlog file to the TiDB same-city disaster backup library, and then uses a preset timing backup script to synchronize the Binlog file in the TiDB same-city disaster backup library to the flash memory NAS device, and when the abnormality of the TiDB production library is monitored, based on the Binlog file stored in the flash memory NAS device, performs data reduction processing on the TiDB production library, so that the problem that the backup and reduction speed is slow due to the fact that the data in the production library are directly stored in an archive tape in the traditional mode is avoided, and the efficiency of database maintenance is improved.

In some optional implementations of the present embodiment, in step S201, the TiDB-Binlog cluster component includes a Drainer component and a pump component, collecting TiDB data in the production library through the TiDB-Binlog cluster component, obtaining a Binlog file corresponding to the data in the production library TiDB, and synchronizing the Binlog file to the TiDB same city disaster backup library includes:

Collecting binglog files of the TiDB node server corresponding to the pump component through the pump component, taking the binglog files as basic files, and sending the basic files to the Drainer component;

receiving the basic file through Drainer components and merging the basic file to obtain a merged binglog file;

and synchronizing the merged binglog files to TiDB same city disaster recovery library.

Specifically, the server side includes a plurality of TiDB node servers, each TiDB node server corresponds to one pump component, after the TiDB node server generates binglog files, the pump component collects the binglog files as basic files and sends the basic files to the Drainer component, the Drainer component merges all the received basic files, and synchronizes the merged binglog files to the TiDB same city disaster backup library.

Easily understood, the data format in binglog files is binary, and binglog files are used for synchronous backup, so that the backup efficiency can be greatly improved.

The pump component generates a daemon running in the background of each TiDB node server, and has the main functions of recording Binlog files generated by the TiDB node server when data processing is executed in real time and writing the Binlog files into corresponding storage spaces sequentially.

Wherein Drainer component interacts with pump component to collect Binlog file.

In this embodiment, the pump component collects binglog files of the TiDB node server corresponding to the pump component, and sends the basic files to the Drainer component, the Drainer component receives the basic files and merges the basic files to obtain merged binglog files, and then synchronizes the merged binglog files to the TiDB same-city disaster backup library, so as to realize binglog file fast synchronization to the TiDB same-city disaster backup library, and improve the efficiency of binglog file fast synchronization to the TiDB same-city disaster backup library.

In some optional implementations of this embodiment, in step S202, the flash NAS device includes a first flash memory device and a second flash memory device, and synchronizing TiDB binlog files in the same-city disaster recovery library to the flash NAS device using a preset timing backup script includes:

Moving and storing a binlog file in TiDB same-city disaster backup libraries into first flash memory equipment;

If the verification result is that the difference exists, updating the data in the second flash memory device by adopting the data in the first flash memory device according to the difference in the verification result.

Specifically, the flash NAS device in this embodiment includes a first flash device and a second flash device, where the first flash device is a primary storage device, the second flash device is a backup storage device, and the second flash device is installed in the same region of TiDB production libraries. When the network is abnormally interrupted, the backup storage equipment in the same regional network environment as TiDB production libraries is adopted, so that the database backup files can be obtained from the backup storage equipment in the same city under the condition that cross-regional network access is not feasible, and the database can be quickly restored.

Further, the binlog files in TiDB same-city disaster backup libraries are moved and stored in the first flash memory device, incremental and full-quantity difference verification is carried out on the first flash memory device and the second flash memory device at regular time to obtain a verification result, when the verification result is that the difference exists, the data in the second flash memory device is updated according to the difference in the verification result, and synchronous update of the first flash memory device and the second flash memory device is rapidly achieved.

Where delta refers to the last updated data and full refers to all data.

The method comprises the steps of performing incremental and full-quantity difference verification on a first flash memory device and a second flash memory device at fixed time, and obtaining verification results, wherein the verification results comprise that the full-quantity and incremental file differences of the first flash memory device and the second flash memory device are compared through a timing script, and the main comparison points comprise, but are not limited to, file size, file modification time, file creation time and the like. The specific comparison mode can adopt a character string matching mode.

In this embodiment, the binlog file in the TiDB same-city disaster backup library is moved and stored in the first flash memory device, incremental and full-quantity difference verification is performed on the first flash memory device and the second flash memory device at regular time to obtain a verification result, and when the verification result is that there is a difference, the data in the second flash memory device is updated by adopting the data in the first flash memory device according to the difference in the verification result, so that the quick synchronous update of the first flash memory device and the second flash memory device is realized.

In some optional implementations of this embodiment, moving and saving the binlog file in TiDB co-city disaster recovery repository to the first flash device includes:

Taking a binlog file in TiDB same-city disaster backup libraries as data to be imported, and acquiring a storage path of first flash memory equipment corresponding to the data to be imported;

based on the number of target threads, constructing multiple threads, and synchronizing data to be imported to a storage path of the first flash memory device by adopting the multiple threads.

Specifically, each TiDB node server corresponds to a preset storage path in the first flash memory device, takes a binlog file in TiDB same-city disaster backup library as data to be imported, determines the storage path of the first flash memory device corresponding to the data to be imported according to the TiDB node server corresponding to the data to be imported, determines the number of target threads according to the imported data and the first flash memory device, and constructs a plurality of threads of the number to perform data synchronization.

The method comprises the steps of setting the minimum thread number according to the CPU core number of a host, amplifying the thread number in a slow and multiple mode, observing the IO and network load of the host, and dynamically determining the number of the imported threads according to the load of the host.

In this embodiment, the number of threads is dynamically determined, and the binlog file is synchronized to the first flash memory device in a multithreading manner, which is favorable for saving system resources and improving data backup efficiency.

In some optional implementations of the present embodiment, in step S203, if it is monitored that the TiDB production library is abnormal, performing data reduction processing on the TiDB production library based on the binlog file stored in the flash NAS device includes:

acquiring a full backup file and an incremental backup file from flash NAS equipment as files to be restored;

loading the full backup file into a data table in a TiDB production library, and loading the backup file into a data table in a TiDB production library;

Specifically, the files corresponding to all data stored in the flash memory NAS device are used as full-cost files, the file corresponding to the data backed up last time is used as a gifting file, when the abnormality of the production library is monitored TiDB, the application service stops writing the table to the TiDB production library, the data in the data table of the TiDB production library is emptied, then the full-cost file and the adding file are used as files to be restored, the data table in the TiDB production library is restored by using the files to be restored, the data in the data table in the TiDB production library is verified after the data restoration is completed, after the verification is passed, the success of the data restoration is confirmed, and the application service is started.

In this embodiment, when the abnormality of TiDB production libraries is monitored, the read-write processing is stopped on the data table of TiDB production libraries, the file to be restored is acquired from the flash memory NAS device and restored to the data table of TiDB production libraries, the data in the data table of TiDB production libraries after restoration is verified, and after verification, the application service is started, so that when the abnormality of TiDB production libraries is realized, automatic and rapid restoration is performed, and the database restoration efficiency is improved.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.

Fig. 3 shows a schematic block diagram of a maintenance apparatus for a database TiDB in one-to-one correspondence with the maintenance method for the database of the embodiment TiDB described above. As shown in fig. 3, the TiDB database maintenance device includes a data acquisition module 31, a file synchronization module 32, and a data recovery module 33. The functional modules are described in detail as follows:

The data acquisition module 31 is used for acquiring data in the TiDB production library through TiDB-Binlog cluster components, obtaining a Binlog file corresponding to the data in the TiDB production library, and synchronizing the Binlog file to the TiDB same-city disaster backup library;

the file synchronization module 32 is configured to synchronize a binlog file in a TiDB same-city disaster recovery backup repository to the flash NAS device by using a preset timing backup script;

And the data recovery module 33 is configured to perform data recovery processing on the TiDB production library based on the binlog file stored in the flash NAS device if the abnormality of the TiDB production library is monitored.

Optionally, the data acquisition module 31 includes:

The merging unit is used for receiving the basic files through the Drainer component and merging the basic files to obtain merged binglog files;

Optionally, the flash NAS device includes a first flash device and a second flash device, and the file synchronization module 32 includes:

The synchronization unit is used for moving and storing the binlog files in the TiDB same-city disaster backup library into the first flash memory device;

The verification unit is used for carrying out increment and full-quantity difference verification on the first flash memory device and the second flash memory device at regular time to obtain a verification result;

Optionally, the synchronization unit includes:

the path acquisition subunit is used for taking the binlog file in the TiDB same-city disaster backup library as data to be imported and acquiring a storage path of the first flash memory device corresponding to the data to be imported;

The thread number determining subunit is used for determining the number of target threads based on the data to be imported and the first flash memory device;

And the multithreading synchronization subunit is used for constructing multithreading based on the target thread number and synchronizing the data to be imported to the storage path of the first flash memory device by adopting the multithreading.

Optionally, the data recovery module 33 includes:

The file to be restored determining unit is used for acquiring the full backup file and the backup adding file from the flash NAS equipment to serve as the file to be restored;

For specific limitations on the maintenance device of the TiDB database, reference may be made to the above limitation on the maintenance method of the TiDB database, and no further description is given here. The modules in the TiDB database maintenance device described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only a computer device 4 having a component connection memory 41, a processor 42, a network interface 43 is shown in the figures, but it is understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and its hardware includes, but is not limited to, a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), a Programmable gate array (Field-Programmable GATE ARRAY, FPGA), a digital Processor (DIGITAL SIGNAL Processor, DSP), an embedded device, and the like.

The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.

The memory 41 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or D interface display memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used for storing an operating system and various application software installed on the computer device 4, such as program codes for controlling electronic files, etc. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a Central processing unit (Central ProcessingUnit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute a program code stored in the memory 41 or process data, such as a program code for executing control of an electronic file.

The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.

The present application also provides another embodiment, namely, a computer readable storage medium storing an interface display program, where the interface display program is executable by at least one processor, so that the at least one processor performs the steps of the method for maintaining TiDB databases as described above.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.

It is apparent that the above-described embodiments are only some embodiments of the present application, but not all embodiments, and the preferred embodiments of the present application are shown in the drawings, which do not limit the scope of the patent claims. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a thorough and complete understanding of the present disclosure. Although the application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing description, or equivalents may be substituted for elements thereof. All equivalent structures made by the content of the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the scope of the application.

Claims

1. A method for maintaining a TiDB database, wherein the method for maintaining a TiDB database comprises:

Synchronizing binlog files in the TiDB same-city disaster recovery library to a flash NAS device by adopting a preset timing backup script, wherein the flash NAS device comprises a first flash memory device and a second flash memory device, the first flash memory device is a main storage device, the second flash memory device is a standby storage device, and the second flash memory device is erected in the same region of a TiDB production library;

if TiDB production library abnormality is monitored, carrying out data reduction processing on the TiDB production library based on a binlog file stored in the flash NAS device;

the synchronizing the binlog file in the TiDB same-city disaster recovery backup repository to the flash NAS device using a preset timing backup script includes:

performing incremental and full-quantity difference verification on the first flash memory device and the second flash memory device at regular time to obtain a verification result, wherein the incremental is the data updated last time, and the full-quantity is all data;

if the verification result is that the difference exists, updating the data in the second flash memory device by adopting the data in the first flash memory device according to the difference in the verification result;

and performing incremental and full-quantity difference verification on the first flash memory device and the second flash memory device at fixed time to obtain a verification result, wherein the verification result comprises that the full-quantity and incremental file differences of the first flash memory device and the second flash memory device are compared through a fixed time script, and the main comparison points comprise file size, file modification time and file creation time, and a character string matching mode is adopted in a specific comparison mode.

2. The method of claim 1, wherein the TiDB-Binlog cluster component comprises a Drainer component and a pump component, wherein the collecting TiDB data in the production library by the TiDB-Binlog cluster component, obtaining a Binlog file corresponding to the data in the TiDB production library, and synchronizing the Binlog file to the TiDB homocity disaster recovery library comprises:

3. The method for maintaining TiDB a database as defined in claim 1, wherein moving and saving the binlog file in the TiDB homometropolitan disaster recovery library to the first flash device includes:

4. A method for maintaining a TiDB database according to any one of claims 1 to 3, wherein if an anomaly in a TiDB production library is monitored, performing data reduction processing on the TiDB production library based on a binlog file stored in the flash NAS device includes:

5. A maintenance device for a TiDB database, wherein the maintenance device for a TiDB database comprises:

the file synchronization module is used for synchronizing the binlog files in the TiDB same-city disaster recovery library to the flash NAS device by adopting a preset timing backup script, wherein the flash NAS device comprises a first flash memory device and a second flash memory device, the first flash memory device is a main memory device, the second flash memory device is a standby memory device, and the second flash memory device is erected in the same region of the TiDB production library;

The data recovery module is used for carrying out data recovery processing on the TiDB production library based on the binlog file stored in the flash NAS device if the TiDB production library is monitored to be abnormal;

the file synchronization module comprises:

the verification unit is used for carrying out incremental and full-quantity difference verification on the first flash memory device and the second flash memory device at regular time to obtain a verification result, wherein the incremental is the data updated last time, and the full quantity is all data;

the updating unit is used for updating the data in the second flash memory device by adopting the data in the first flash memory device according to the difference in the verification result if the verification result is that the difference exists;

The verification unit is specifically configured to compare the total and incremental file differences of the first flash memory device and the second flash memory device through a timing script, where the main comparison points include a file size, a file modification time and a file creation time, and a specific comparison mode adopts a character string matching mode.

6. The TiDB database maintenance device of claim 5, wherein the TiDB-Binlog cluster component includes a Drainer component and a pump component, and the data acquisition module includes:

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements a method of maintaining a TiDB database according to any one of claims 1 to 4 when the computer program is executed.

8. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements a method of maintaining a TiDB database as claimed in any one of claims 1 to 4.