CN114510464A - A management method and management system for a highly available database - Google Patents

Abstract

The invention discloses a high-availability database management method and management system, belonging to the technical field of database management. The management method includes: creating or modifying description information; creating, modifying or deleting custom resources according to the description information; monitoring The self-defined resources are generated, and configuration resources are generated; database clusters are managed according to the configuration resources. Through the description information, the configuration information and deployment information of the database are described in a unified manner, which is conducive to the unified management and deployment of custom resources, and the unified management of the database through the custom resources prevents wrong operations caused by the configuration of a large number of internal resources.

Description

A management method and management system for a highly available database

technical field

The invention relates to the technical field of database management, in particular to a management method and management system of a highly available database.

Background technique

Custom Resource is an extension to the Kubernetes API. It can appear or disappear in a running cluster through dynamic registration. Cluster administrators can update custom resources independently of the cluster. Once a custom resource is installed, users can use kubectl to create and access objects within it, just as they do for built-in resources like pods.

Kubernetes is the current mainstream cloud-native container platform, which supports general stateless applications, stateful applications and other workloads to run on it. For databases such as MYSQL, it can be run as a stateful application. Operators are extensions to Kubernetes that automate repetitive tasks on Kubernetes. The Operator pattern encapsulates the written task automation code. The operator mode concept of Kubernetes makes it possible to manage one or more custom resources through a custom controller without modifying the code of Kubernetes itself, so as to realize the function of expanding the cluster.

With the increase of business volume, there will be more and more changes to multiple internal resources. By using custom resources, the configuration properties required by users can be centralized into a single resource for unified management, so as to avoid the crash of the database caused by wrong changes.

SUMMARY OF THE INVENTION

Aiming at the above technical problems existing in the prior art, the present invention provides a management method and management system for a high-availability database, which uniformly describes the deployment or management of the database through description information, and manages the database according to the description information, thereby simplifying database management. , to prevent erroneous operation.

The invention discloses a management method for a highly available database. The management method includes: creating or modifying description information; creating, modifying or deleting custom resources according to the description information; monitoring the custom resources and generating a configuration resource; manage the database cluster according to the configuration resource.

Preferably, the description information includes any of the following information or a combination thereof:

Database configuration information, instance creation strategy, upgrade strategy, master-slave strategy, master election strategy, and replication strategy.

Preferably, the method for database failure recovery includes:

Listen to the instance of the database through the Operator;

Determine whether the main instance is faulty;

If so, select a normal instance as the primary instance according to the primary selection strategy;

Modify the replication policy to point the data replication to the primary instance.

Preferably, the method for selecting a master instance includes:

access a live instance and obtain the deferred state of said instance;

The primary instance is selected from the surviving instances in a way that minimizes latency.

Preferably, the replication strategy includes the connection address, user name, password, and replication method of the primary instance;

The read-only policy of the master instance is set to off, and the read-only policy of the slave instance is set to on;

The methods for reconnecting the failed instance include:

Detect failure instances;

Determine whether the faulty instance is recovered;

If so, set the recovered failed instance as a slave instance and synchronize the replication policy.

Preferably, the configuration resources include any one of the following resources or a combination thereof:

Configuration files, stateful applications, resource objects for storage volumes, and service discovery objects;

Instance creation methods include:

Create a stateful application according to the creation strategy;

Create container resources based on stateful application creation events;

According to the configuration file, the container resource is configured to obtain a database instance.

Preferably, the upgrade method includes:

Create an upgrade strategy, the upgrade description information of the upgrade strategy includes: database configuration update, container cluster deployment configuration update, and database version update;

Generate or modify custom resources according to the upgrade description information;

Monitor the custom resource through the Operator, and update the configuration file;

According to the configuration file, upgrade the database, container or cluster.

Preferably, the method for instance uninstallation includes:

Delete the corresponding custom resources of the instance;

Monitor the deletion event of the custom resource through the Operator, and delete the corresponding configuration resource;

According to the deletion event of the configuration resource, the container group of the instance is deleted.

The present invention also provides a management system for implementing the above management method, comprising:

Describe the information management module, custom resource management module, monitoring module and execution module;

The description information management module is used to create or modify description information;

The custom resource management module is used to create, modify or delete custom resources according to the description information;

The monitoring module is used to monitor the user-defined resources and generate configuration resources;

The execution module is configured to manage the database cluster according to the configuration resource.

Preferably, the monitoring module is also used for:

An instance of the listening database;

Determine whether the main instance is faulty;

If so, select a normal instance as the primary instance according to the primary selection strategy;

Modify the replication policy to point the data replication to the primary instance.

Compared with the prior art, the present invention has the beneficial effects that the configuration information and deployment information of the database are described in a unified manner through the description information, which is beneficial to the unified management and deployment of the self-defined resources, and the unified management of the database through the self-defined resources. , to prevent incorrect operations due to a large number of APIs.

Description of drawings

Fig. 1 is the management method flow chart of the high-availability database of the present invention;

FIG. 2 is a logical block diagram of the management system of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

Below in conjunction with accompanying drawing, the present invention is described in further detail:

A management method for a highly available database, as shown in Figure 1, the management method includes:

Step 101: Create or modify description information. The description information can be saved in the form of files, such as yaml or json files; the description information is used to describe the attributes of database management, such as the internal configuration information of the database: root password, size of various cache areas, data storage mechanism, etc. Relevant configuration information in the cluster: number of instances, hardware resource quotas, network policies, scheduling policies, etc.

Step 102: Create, modify or delete a custom resource according to the description information.

Step 103: Monitor the custom resource and generate a configuration resource.

The configuration resource includes any one of the following resources or a combination thereof: a configuration file (Configmap), a stateful application (Statefulset), a resource object (PVC) of a storage volume, and a service discovery object (service). In a specific embodiment, an operator is used to monitor custom resources.

Step 104: Manage the containerized database according to the configuration resource. The database may be a MYSQL database, but is not limited thereto. A containerized database can have multiple instances, forming a database cluster.

Through the description information, the configuration information and deployment information of the database are described in a unified manner, which is conducive to the unified management and deployment of custom resources, and the unified management of the database through the custom resources can prevent incorrect operations caused by a large number of APIs and improve the performance of the database. availability.

In step 101, the description information includes any of the following information or a combination thereof:

Database configuration information, instance creation strategy, upgrade strategy, master-slave strategy, master election strategy, and replication strategy. The custom resource corresponds to the above description information.

Database management includes database failure recovery, database upgrade, instance creation, and instance uninstallation.

Example 1

Methods for database failure recovery include:

Step 201: Monitor the instance of the database through the Operator. Operators can periodically try to connect to each database instance through scheduled tasks, and execute corresponding SQL statements for detection.

Step 202: Determine whether the primary instance is faulty. The Operator program periodically probes each MYSQL instance, that is, connects to MYSQL instances distributed on different nodes through the configured MYSQL connection address, user name, password, port and other information. If the connection times out, the instance is faulty. After the connection is successful, execute the "SELECT1" command. If the normal result cannot be returned, it means that the instance is faulty, otherwise it is a normal instance. Mark normal instances and abnormal instances.

If yes, go to step 203: select a normal instance as the primary instance according to the primary selection policy.

Wherein, the method for selecting a master instance includes: Step 211 : Access a surviving instance, and obtain the delay state of the instance. Step 212: Select a master instance from the surviving instances in a manner that minimizes the delay.

If not, go to step 206: monitor the instance continuously or periodically.

Step 205 : Modify the replication policy and load balancing policy, etc., copy the data of the slave instance to the master instance, and set the read-only policy of the master instance to off and the read-only policy of the slave instance to on.

The replication policy includes the connection address, user name, password, and replication mode of the primary instance. Specifically, by default, MYSQL with serial number 0 is used as the master database, other database instances are used as slave databases, and the configuration data is copied from the master database to the slave database. When the replication is finally started, the Binlog log written by the main database to the database is pulled from the database, and played back in itself to achieve the purpose of data replication, so as to ensure the consistency of the data.

Step 206: The faulty instance is reconnected. The method for reconnecting the faulty instance includes: detecting the faulty instance; judging whether the faulty instance is recovered; if so, setting the recovered faulty instance as a slave instance, and synchronizing the replication strategy. Operators can perform automatic repairs for specific failure scenarios, and can recover in a short time without the participation of operation and maintenance personnel.

The Operator program will record and record the metadata of the master-slave relationship during operation. At the same time, two service discovery with load balancing capability are created as the entry for writing data and reading data. The client implements read-write separation by accessing the domain names discovered by these two services.

It should be pointed out that the types of instances that cannot be obtained include: failure of the master database, active storage of the slave database; failure of the slave database, active storage of the master database; failure of both the master database and the slave database. When the master library fails and the slave library is active, it is necessary to perform a master-slave switchover to make one of the slave libraries become the master library. When the slave library fails and the master library is active, there is no need to perform master-slave switchover. The load balancer in Kubernetes will automatically remove the unreachable slave library. When both the master library and the slave library are faulty, the system as a whole is faulty and cannot be repaired.

Among them, the MYSQL cluster as a whole can adopt the architecture of one master and multiple slaves, and each instance is independent of each other. That is, one instance provides the main data writing capability as the master instance, and the other instances provide the data reading capability as the slave instance. This embodiment implements the master-slave architecture implementation of the MYSQL cluster and automatic fault repair through the Operator, thereby maintaining the high availability of the MYSQL cluster, for example, performing master-slave switching of the database when a downtime failure occurs to ensure that the read and write functions of the database are normal.

You can use the Operator to access each MYSQL database instance in turn after the database is started and configure the replication strategy. Specifically, configure the database connection address, user name, password, replication method and other information of the master database for the slave database. The data is incrementally replicated by the master database. To the slave library to ensure data redundancy. At the same time, the Kubernetes platform provides a load balancing entry with read-write separation, including a service entry for writing data and a service exit for reading data. The exits are the master library and the slave library, and the operator maintains the mapping relationship.

Example 2

Methods for database instance creation include:

Step 301: Create a stateful application according to the creation policy.

Step 302: Create a container resource according to the creation event of the stateful application. Container resources can be created through the Kubernetes platform.

Step 303: Configure the container resource according to the configuration file to obtain a database instance.

Example 3

Methods of database upgrade include:

Step 401: Create an upgrade policy, the upgrade description information of the upgrade policy includes: database configuration update, container cluster deployment configuration update, and database version update. For example, upgrade the MYSQL version to the specified version; horizontally expand the database cluster, that is, create an instance again.

Step 402: Generate or modify a custom resource according to the upgrade description information.

Step 403: Monitor the custom resource through the Operator, and update the configuration file.

Step 404: Upgrade the database, container or cluster according to the configuration file.

Example 4

Methods for instance uninstallation include:

Step 501: Delete the custom resource corresponding to the instance.

Step 502: Monitor the deletion event of the custom resource through the Operator, and delete the corresponding configuration resource.

Step 503: According to the deletion event of the configuration resource, delete the container resource of the instance, such as a container group (POD).

Due to the relationship between resources, the database custom resources belong to the top layer. The controller of the cloud native platform will automatically delete all associated lower-level resources according to the cascade relationship. If the custom resources are deleted, the built-in resources such as Statefulset, Service, and Configmap will be automatically deleted. Since the Statefulset and Pod are in a cascade relationship, after the Statefulset is deleted, the Pod will be automatically deleted. This process is controlled by the controller of the cloud native platform. Data volumes used by the database are retained or automatically deleted depending on how they are actually defined.

Example 5

This embodiment provides a management system for implementing the above-mentioned high-availability database management method, as shown in FIG. 2 , including a description information management module 1, a custom resource management module 2, a monitoring module 3, and an execution module 4;

The description information management module 1 is used to create or modify the description information 11;

The custom resource management module 2 is used to create, modify or delete custom resources 12 according to the description information 11;

The monitoring module 3 is used to monitor the custom resource 12, and generate the configuration resource 13 according to the custom resource;

The execution module 4 is configured to manage a containerized database cluster according to the configuration resource 13 , and the database cluster includes a plurality of instances 15 . Specifically, the execution module 4 can manage the database cluster through the Kubernetes platform.

Wherein, the monitoring module 3 is also used to execute the database failure recovery method in Embodiment 1:

An instance of the listening database;

Determine whether the main instance is faulty;

If so, select a normal instance as the primary instance according to the primary selection strategy;

Modify the replication policy to point the data replication to the primary instance.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a management method for a highly available database, wherein the management method comprises: create or modify descriptive information; Create, modify or delete custom resources based on the description; Monitor the custom resource, and generate a configuration resource; The database cluster is managed according to the configuration resource. 2. The management method according to claim 1, wherein the description information comprises any one of the following information or a combination thereof: Database configuration information, instance creation strategy, upgrade strategy, master-slave strategy, master election strategy, and replication strategy. 3. The management method according to claim 2, wherein the method for database failure recovery comprises: Listen to the instance of the database through the Operator; Determine whether the main instance is faulty; If so, select a normal instance as the primary instance according to the primary selection strategy; Modify the replication policy to point the data replication to the primary instance. 4. management method according to claim 3, is characterized in that, the method for selecting master instance comprises: access a live instance and obtain the deferred state of said instance; The primary instance is selected from the surviving instances in a way that minimizes latency. 5. The management method according to claim 4, wherein the replication strategy comprises the connection address, user name, password, and replication mode of the primary instance; The read-only policy of the master instance is set to off, and the read-only policy of the slave instance is set to on; The methods for reconnecting the failed instance include: Detect failure instances; Determine whether the faulty instance is recovered; If so, set the recovered failed instance as a slave instance and synchronize the replication policy. 6. The management method according to claim 2, wherein the configuration resource comprises any one of the following resources or a combination thereof: Configuration files, stateful applications, resource objects for storage volumes, and service discovery objects; Instance creation methods include: Create a stateful application according to the creation strategy; Create container resources based on stateful application creation events; According to the configuration file, the container resource is configured to obtain a database instance. 7. The management method according to claim 6, wherein the method for upgrading comprises: Create an upgrade strategy, the upgrade description information of the upgrade strategy includes: database configuration update, container cluster deployment configuration update or database version update; Generate or modify custom resources according to the upgrade description information; Monitor the custom resource through the Operator, and update the configuration file; According to the configuration file, upgrade the database, container or cluster. 8. The management method according to claim 6, wherein the method for instance unloading comprises: Delete the corresponding custom resources of the instance; Monitor the deletion event of the custom resource through the Operator, and delete the corresponding configuration resource; According to the deletion event of the configuration resource, the container resource of the instance is deleted. 9. A management system for implementing the management method according to any one of claims 1-8, characterized in that, comprising: a description information management module, a self-defined resource management module, a monitoring module and an execution module; The description information management module is used to create or modify description information; The custom resource management module is used to create, modify or delete custom resources according to the description information; The monitoring module is used to monitor the user-defined resources and generate configuration resources; The execution module is configured to manage the database cluster according to the configuration resource. 10. The management system according to claim 9, wherein the monitoring module is further used for: An instance of the listening database; Determine whether the main instance is faulty; If so, select a normal instance as the primary instance according to the primary selection strategy; Modify the replication policy to point the data replication to the primary instance.

Images