CN117632651A

CN117632651A - A fault self-healing system and method based on MySQL database

Info

Publication number: CN117632651A
Application number: CN202311651606.7A
Authority: CN
Inventors: 廖明礼
Original assignee: Tianyi Digital Life Technology Co Ltd
Current assignee: Tianyi Shilian Technology Co ltd
Priority date: 2023-12-04
Filing date: 2023-12-04
Publication date: 2024-03-01

Abstract

This application discloses a fault self-healing system and method based on MySQL database. The system collects the monitoring indicator data of the MySQL database through the alarm monitoring module, performs fault alarm monitoring, obtains fault alarm information, and analyzes the fault alarm through the fault analysis and decision-making module. and fault alarm decision-making, determine the target fault alarm scenario and the target fault alarm repair script, use the fault repair processing module to use the target fault alarm repair script to perform fault alarm repair processing on the MySQL database based on the fault alarm information, and obtain the fault repair processing results. This application can realize a closed-loop fault handling process for the MySQL database, including fault alarms, fault analysis decisions, and fault self-healing, improve the efficiency of fault troubleshooting and repair, effectively improve the stability of the online database, and solve the problem of low timeliness of online database fault recovery. problems, widely used in the field of database technology.

Description

Fault self-healing system and method based on MySQL database

Technical Field

The application relates to the technical field of databases, in particular to a fault self-healing system and method based on a MySQL database.

Background

In the present information age, data is a core resource of an enterprise, and effective management and utilization of data are critical to the development of the enterprise. Relational databases are a common way of data storage and management for most businesses.

MySQL is widely applied to various fields as a common relational database, as enterprises develop, mySQL database examples are more and more, operation and maintenance scales are more and more, on-line faults are more and more, so that the processing timeliness of database operation and maintenance personnel for each fault is continuously reduced, the time required from the occurrence of the fault to the positioning and solving is long, the service quality of the whole database is reduced, the availability is reduced, the troubleshooting and repairing efficiency of the fault is also reduced, and the stability of the on-line database is not guaranteed.

Disclosure of Invention

In order to solve at least one technical problem in the related art, the embodiment of the application provides a fault self-healing system and a method based on a MySQL database, which aim to realize fault self-healing of the database, improve the troubleshooting and repairing efficiency of the fault and effectively improve the stability of the database on line.

In one aspect, an embodiment of the present application proposes a fault self-healing system based on MySQL database, the system including:

The alarm monitoring module is used for collecting monitoring index data of the MySQL database, and carrying out fault alarm monitoring according to the monitoring index data to obtain fault alarm information;

the fault analysis decision module is used for acquiring the fault alarm information, carrying out fault alarm analysis on the fault alarm information, determining a target fault alarm scene corresponding to the fault alarm information, carrying out fault alarm decision according to the target fault alarm scene and the fault alarm information, and determining a target fault alarm repair script;

and the fault repairing processing module is used for acquiring the target fault alarm repairing script, and performing fault alarm repairing processing on the MySQL database by utilizing the target fault alarm repairing script according to the fault alarm information to acquire a fault repairing processing result.

In some embodiments, the system further comprises:

and the self-healing notification module is used for acquiring the fault repair processing result and pushing the fault repair processing result to the user terminal.

In some embodiments, the alarm monitoring module comprises a data acquisition unit and an alarm message acquisition unit;

the data acquisition unit is used for acquiring the monitoring index data corresponding to the monitoring index type according to the preset monitoring index type, acquiring preset fault alarm judgment conditions, carrying out fault alarm judgment on the monitoring index data by utilizing the fault alarm judgment conditions, and determining whether to generate the fault alarm information;

The alarm message acquisition unit is used for acquiring the fault alarm information and writing the fault alarm information into a fault alarm message queue.

In some embodiments, the fault analysis decision module includes a script management unit and an alarm analysis unit;

the script management unit is used for storing and managing fault alarm related scripts; the fault alarm related script at least comprises a fault alarm analysis script and a plurality of fault alarm repair scripts;

the alarm analysis unit is used for acquiring the fault alarm information from the fault alarm message queue, calling the fault alarm analysis script, reading the fault alarm information, analyzing the fault type and matching the fault scene, and determining the target fault alarm scene from a plurality of preset fault alarm scenes; the target fault alarm scene is the fault alarm scene corresponding to the fault alarm information.

In some embodiments, the fault analysis decision module further comprises an alert decision unit;

the alarm decision unit is used for acquiring the target fault alarm scene, carrying out fault alarm decision according to the target fault alarm scene, and determining the target fault alarm repair script from a plurality of fault alarm repair scripts; the target fault alarm repair script is the fault alarm repair script corresponding to the fault alarm information;

The alarm decision unit is also used for outputting a fault repairing instruction, controlling the fault repairing processing unit according to the fault repairing instruction and issuing the target fault alarm repairing script to the fault repairing processing unit.

In some embodiments, the fault repair processing unit is configured to parse the fault alarm information to obtain fault alarm parsing information;

the fault repairing unit is further configured to receive the target fault alarm repairing script and the fault repairing instruction, and perform fault alarm repairing processing according to the fault alarm analysis information by using the target fault alarm repairing script, so as to obtain the fault repairing processing result.

In some embodiments, the fault alert scenario includes at least a host downtime alert, a host CPU usage high alert, a host Load high alert, a host memory usage high alert, a disk space shortage alert, a host time error alert, a MySQL active session alert, a MySQL standby library delay alert, a MySQL connection count alert, a MySQL database downtime alert, a MySQL master slave replication disconnection alert, a MySQL database QPS/TPS increase above threshold alert, and a MySQL database slow SQL Load alert.

In some embodiments, the alarm message collection unit is configured to set a plurality of message partitions of the fault alarm message queue, and write a plurality of fault alarm information into each message partition;

the alarm analysis unit is used for taking an alarm analysis process as a consumer, and consuming a plurality of fault alarm information in each message partition through the alarm analysis process to obtain a plurality of consumption groups consisting of a plurality of fault alarm messages;

the alarm analysis unit is further used for acquiring the fault alarm information from each consumption group.

In some embodiments, the fault repairing unit is further configured to monitor a fault repairing overall process corresponding to the fault alarm repairing process, and generate corresponding fault repairing log information according to the fault repairing overall process and the fault repairing result.

On the other hand, the embodiment of the application provides a fault self-healing method based on a MySQL database, which comprises the following steps:

collecting monitoring index data of a MySQL database, and carrying out fault alarm monitoring according to the monitoring index data to obtain fault alarm information;

performing fault alarm analysis on the fault alarm information, determining a target fault alarm scene corresponding to the fault alarm information, performing fault alarm decision according to the target fault alarm scene and the fault alarm information, and determining a target fault alarm repair script;

And according to the fault alarm information, performing fault alarm repair processing on the MySQL database by using the target fault alarm repair script to obtain a fault repair processing result.

According to the fault self-healing system and method based on the MySQL database, the monitoring index data of the MySQL database is collected through the alarm monitoring module, fault alarm monitoring is conducted, fault alarm information is obtained, fault alarm analysis and fault alarm decision are conducted through the fault analysis decision module, a target fault alarm scene and a target fault alarm repairing script are determined, fault alarm repairing processing is conducted on the MySQL database through the fault repairing processing module according to the fault alarm information by utilizing the target fault alarm repairing script, and a fault repairing processing result is obtained. The fault processing full-flow closed loop for fault alarming, fault analysis decision and fault self-healing of the MySQL database can be realized, the fault checking and repairing efficiency is improved, the stability of the online database is effectively improved, and the problem of low fault recovery timeliness of the online database is solved.

Drawings

Fig. 1 is a schematic structural diagram of a fault self-healing system based on MySQL database according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of an alarm monitoring module according to an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a failure analysis decision module according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an alarm message collection unit writing fault alarm information into a fault alarm message queue in an embodiment of the present application;

FIG. 5 is a schematic diagram of a structure for acquiring a fault alarm message through consumption of an alarm analysis unit in an embodiment of the present application;

FIG. 6 is another schematic diagram of a structure for acquiring a fault alarm message through consumption of an alarm analysis unit in an embodiment of the present application;

FIG. 7 is another schematic diagram of a configuration for acquiring a fault alarm message through consumption of an alarm analysis unit in an embodiment of the present application;

FIG. 8 is a schematic diagram of another structure of a MySQL-based database according to an embodiment of the present application;

FIG. 9 is a flowchart of a fault self-healing method based on MySQL database according to an embodiment of the present application;

fig. 10 is a schematic hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.

First, several nouns referred to in this application are parsed:

MySQL: mySQL is one of the most popular relational database management systems, and in terms of WEB applications, mySQL is one of the best RDBMS (relational database management system) application software. MySQL is a relational database management system that keeps data in different tables rather than placing all data in one large warehouse, which increases speed and flexibility. The SQL language used by MySQL is the most commonly used standardized language for accessing databases. MySQL software adopts a double-authorization policy and is divided into a community version and a commercial version, and because of the characteristics of small size, high speed and low total possession cost, especially the open source code, mySQL is generally selected as a website database for developing small and medium-sized websites and large-sized websites.

Hook (Hook): is a platform of Windows message handling mechanism on which an application may set a sub-program to monitor certain messages specifying a window, and the monitored window may be created by other processes. When the message arrives, it is processed before the target window processing function. The hook mechanism allows an application to intercept a processing window message or a particular event. The hook is effectively a piece of program that processes a message, and is suspended from the system by a system call. Whenever a particular message is sent, the hooking program captures the message before the destination window is reached, i.e., the hooking function gains control first. The hook function can process (change) the message, can continue to transmit the message without processing, and can forcedly end the transmission of the message.

Anstable: the system is an automatic operation and maintenance tool, integrates the advantages of a plurality of operation and maintenance tools (puppet, cfengine, chef, func, fabric) based on Python development, and achieves the functions of batch system configuration, batch program deployment, batch operation commands and the like. The conventional architecture is relatively simple and only needs to connect clients through SSH to execute tasks.

Saltstack: saltstack is a set of C/S architecture developed based on Python, and has configuration management tools integrating functions of configuration management, remote execution, monitoring and the like, so that the functions are very powerful, and the fusion degree and the reusability of each module are very high; the world-wide fastest lightweight message queue ZeroMQ is used, so that Saltstack can perform various operations on tens of thousands of servers in seconds, identity is confirmed by using an RAS Key mode, AES encryption is adopted for transmission, and safety performance is higher.

Kafka: kafka is an open source stream processing platform developed by the Apache software foundation, written by Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action flow data for consumers in a web site. Such actions (web browsing, searching and other user actions) are a key factor in many social functions on modern networks. These data are typically addressed by processing logs and log aggregations due to throughput requirements. This is a viable solution for log data and offline analysis systems like Hadoop, but with the limitation of requiring real-time processing. The purpose of Kafka is to unify on-line and off-line message processing through the Hadoop parallel loading mechanism, and also to provide real-time messages through the clusters.

Script: is an executable file written in accordance with a certain format, also called a macro or batch file, using a specific descriptive language. When executing the script, the computer performs a series of operations. These operations may involve other applications such as word processing, spreadsheets, and data management.

And (3) self-healing of faults: and (3) receiving the alarm positioning of the monitoring platform, matching with a preset fault processing flow, further realizing the automatic recovery of the fault by an automatic means, finding an alarm in real time, performing pre-diagnosis analysis, automatically recovering the fault, and opening a peripheral system to realize the closed loop of the whole flow.

QPS: query-per-second (QPS) is a measure of how much traffic a particular query server handles in a specified time. The performance of a machine is typically measured as a query rate per second, which corresponds to the number of tasks/sec, i.e., response requests per second, i.e., maximum throughput.

TPS: transaction per second (Transactions Per Second), a transaction refers to the process by which a client sends a request to a server and the server then reacts. A specific transaction definition may be an interface, interfaces, a business process, etc. TPS is typically the unit of measurement of the results of a software test.

The MySQL database-based fault self-healing system and method are widely applied to various fields, and are particularly important in ensuring the running stability of the online database and improving the fault recovery timeliness of the online database.

The subject application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

Referring to fig. 1, fig. 1 is an optional structural schematic diagram of a fault self-healing system based on MySQL database according to an embodiment of the present application, where the system may include, but is not limited to, the following modules:

the fault analysis decision module is used for acquiring fault alarm information, carrying out fault alarm analysis on the fault alarm information, determining a target fault alarm scene corresponding to the fault alarm information, carrying out fault alarm decision according to the target fault alarm scene and the fault alarm information, and determining a target fault alarm repair script;

the fault repairing processing module is used for acquiring a target fault alarm repairing script, and performing fault alarm repairing processing on the MySQL database by utilizing the target fault alarm repairing script according to the fault alarm information to acquire a fault repairing processing result.

In some embodiments, the fault self-healing system based on MySQL database further includes a self-healing notification module.

The self-healing notification module is used for acquiring the fault repair processing result and pushing the fault repair processing result to the user terminal.

In some embodiments, after the fault repair processing module completes fault repair, the fault repair processing result is issued to the self-healing notification module, and the self-healing notification module pushes the fault repair processing result to a user terminal corresponding to a database server maintainer in a mail, weChat, SMS or other mode, so that a full-flow closed loop of fault alarm, analysis, repair and self-healing notification is achieved.

In some embodiments, the fault self-healing system based on MySQL database provided in the embodiments of the present application initializes 10 or more database fault alarm scenarios, when a fault alarm message of the database is monitored, starts an analysis process to perform fault type analysis and match the fault alarm scenario, performs self-healing script matching after obtaining a target fault alarm scenario, notifies a fault repair process to issue a target fault alarm repair script, executes a repair command (a fault repair instruction) to repair a fault of the MySQL database, and sends a fault repair processing result notification to a database maintainer (the user terminal) after repair is completed, thereby implementing a full-flow closed loop of fault alarm, fault analysis, fault self-healing, and fault self-healing notification.

In some embodiments, referring to fig. 2, fig. 2 is an optional structural schematic diagram of an alarm monitoring module in an embodiment of the present application, where the alarm monitoring module includes a data acquisition unit and an alarm message acquisition unit.

The data acquisition unit is used for acquiring monitoring index data corresponding to the monitoring index type according to the preset monitoring index type, acquiring preset fault alarm judgment conditions, carrying out fault alarm judgment on the monitoring index data by utilizing the fault alarm judgment conditions, and determining whether fault alarm information is generated or not;

The alarm message acquisition unit is used for acquiring fault alarm information and writing the fault alarm information into the fault alarm message queue. In some embodiments, specifically, the alarm message collection unit starts a Listener process to monitor Hook (Hook) service, receives the fault alarm message pushed by the data collection unit, writes the monitored fault alarm message into the fault alarm message queue in the form of a consumption group, and improves the alarm message analysis capability.

In some embodiments, referring to fig. 3, fig. 3 is an optional structural schematic diagram of a fault analysis decision module in an embodiment of the present application, where the fault analysis decision module includes a script management unit, an alarm analysis unit, and an alarm decision unit.

the alarm analysis unit is used for acquiring fault alarm information from the fault alarm message queue, calling a fault alarm analysis script, reading the fault alarm information, analyzing fault types and matching fault scenes, and determining a target fault alarm scene from a plurality of preset fault alarm scenes; the target fault alarm scene is a fault alarm scene corresponding to the fault alarm information;

The alarm decision unit is used for acquiring a target fault alarm scene, carrying out fault alarm decision according to the target fault alarm scene, and determining a target fault alarm repair script from a plurality of fault alarm repair scripts; the target fault alarm repair script is a fault alarm repair script corresponding to the fault alarm information;

the alarm decision unit is also used for outputting a fault repair instruction, controlling the fault repair processing unit according to the fault repair instruction, and issuing a target fault alarm repair script to the fault repair processing unit.

In some embodiments, optionally, the fault alert scenario of the fault self-healing system initialization includes, but is not limited to, including a host down alert, a host CPU high usage alert, a host Load high alert, a host memory high usage alert, a disk space shortage alert, a host time error alert, a MySQL active session alert, a MySQL standby delay alert, a MySQL connection number alert, a MySQL database down alert, a MySQL master slave replication disconnect alert, a MySQL database QPS/TPS increase beyond a threshold alert, a MySQL database slow SQL Load alert, and the like.

In some embodiments, optionally, the alarm analysis unit first performs fault type analysis on the fault alarm information, where the fault types may include, but are not limited to, mySQL database alarm and host load alarm, and then determines a target fault type corresponding to the fault alarm information, and then performs fault scene matching with multiple fault alarm scenes corresponding to the target fault type, to determine a target fault alarm scene corresponding to the fault alarm information.

In some embodiments, the alarm decision unit performs a fault alarm decision according to a target fault alarm scenario, determines a corresponding target fault alarm repair script by using a fault decision function, specifically, determines a target fault decision function matched with the target fault alarm scenario from multiple fault decision functions, combines the fault alarm information and the target fault alarm scenario, and determines the target fault alarm repair script by using the target fault decision function.

Alternatively, the fault decision function may include, but is not limited to including, the following functions:

Analyze_MySQL_Repl_Delay (ip, port, dbType): the fault alarm scene matched with the function is MySQL standby library delay alarm;

Analyze_high_Load (ip, port, dbType): the fault alarm scene matched with the function is a host CPU utilization rate high alarm and a host Load high alarm;

Analyze_MySQL_threads (ip, port, dbType) (ip, port, dbType): the fault alarm scene matched with the function is MySQL active session connection number high alarm;

Analyze_node_Time_sync (ip, port, dbType): the fault alarm scene matched by the function is MySQL server time error high alarm.

When the target fault alarm scene is MySQL standby Delay alarm, the alarm decision unit makes a fault alarm decision through the above-mentioned analysis_mysql_repl_delay function, specifically, reads information such as a database host ip address, a database port, a database type and the like contained in the fault alarm information through the analysis_mysql_repl_delay function, analyzes a MySQL standby Delay cause, executes a showcase command to read variable parameter values of slave_io_ Running, slave _sql_running and second_behend_master, confirms that the current copy thread is normally Running, has Delay time greater than or equal to 5s, decides to perform MySQL Slave Delay repair, and determines that the target fault alarm repair script is a Slave Delay fault repair script.

When the target fault alarm scene is a host CPU (central processing unit) High-utilization rate alarm or a host Load High-alarm, the alarm decision unit makes a fault alarm decision through the analysis_high_load (ip, port, dbType) function, specifically reads information such as a database host ip address, a database port, a database type and the like contained in alarm information through the analysis_high_load (ip, port, dbType) function, analyzes a host Load High reason, detects whether a current process has a MySQL process, confirms whether the current process has a slow query exceeding 1s or not, accords with a High Load caused by the slow query, decides to perform MySQL slow SQL repair, and determines the target fault alarm repair script to be a MySQL slow SQL repair report_sql script.

When the target fault alarm scene is a MySQL active session connection number high alarm, the alarm decision unit makes a fault alarm decision through the above-mentioned analysis_mysql_threads (ip, port, dbType) function, specifically, reads the database host ip address, database port, database type and other information contained in the alarm information through analysis_mysql_threads (ip, port, dbType), analyzes the MySQL current connection number, executes a select count (x) from information_schema_processing command to read the current connection number, executes a show variableslike 'max_connections' command to read the configured maximum connection number parameter value, and uses the rate=count_process/max_connections to determine that the current connection number is greater than or equal to 70%, and makes a decision to perform a fault repair script for the MySQL active session.

When the target fault alarm scene is a MySQL server Time error high alarm, the alarm decision unit makes a fault alarm decision through the above-mentioned analysis_node_time_sync (ip, port, dbType) function, specifically, reads information such as a database host ip address, a database port, a database type and the like contained in alarm information, analyzes the Time difference between the current Time of the host and the Time zone standard Time, and if diff=datetime (standard_time) -datetime (current_time) satisfies an error greater than or equal to 5s, decides to perform host Time error high fault repair, and determines the target fault alarm repair script to be a host Time error high repair report_node_sync script.

In some embodiments, the fault repair processing unit is configured to parse the fault alert information to obtain fault alert parse information.

In some embodiments, the fault repairing unit is further configured to receive a target fault alarm repairing script and a fault repairing instruction, and perform fault alarm repairing processing according to the fault alarm analysis information by using the target fault alarm repairing script, so as to obtain a fault repairing processing result.

In some embodiments, the fault repairing processing unit receives the target fault warning repairing script and the fault repairing instruction sent by the warning decision unit, executes the fault self-healing function, and performs fault warning repairing processing on the MySQL database.

In some embodiments, the alarm message collection unit is configured to set a plurality of message partitions of the fault alarm message queue, and write a plurality of fault alarm messages into each message partition;

the alarm analysis unit is used for taking the alarm analysis process as a consumer, consuming a plurality of fault alarm information in each message partition through the alarm analysis process, and obtaining a plurality of consumption groups formed by the plurality of fault alarm information;

the alarm analysis unit is also used for acquiring fault alarm information from each consumption group.

In some embodiments, the collection and consumption functions of the fault alert message may be implemented, for example, with reference to Kafka.

In some embodiments, specifically, the alarm message collection unit pulls a plurality of fault alarm information from the data collection unit, and writes each fault alarm information in batches into each message partition in the fault alarm message queue.

Referring to fig. 4, fig. 4 is an optional schematic diagram of writing, by an alarm message collection unit, fault alarm information into a fault alarm message queue in the embodiment of the present application, where fault alarm information 1 to N is provided, message partition queues a to C are set in a fault alarm consumption queue, each fault alarm information is written into each message partition queue in batches according to a preset writing rule, the message partition queue a includes fault alarm information 1, 3, 4, 6, …, N-1, the message partition queue B includes fault alarm information 2, 5, 7, 9, …, N-2, and the message partition queue C includes fault alarm information 8, 10, 11, …, N.

In some embodiments, the alarm analysis unit takes the alarm analysis process as a consumer, follows a rule that one message partition can only be consumed by consumers of the same message group, and can consume each message partition queue in the fault alarm message queues in a mixed mode or a single mode according to requirements to obtain a plurality of consumption groups consisting of a plurality of fault alarm information. The consumption modes may include, but are not limited to, a publish-subscribe mode and a point-to-point mode, and are specifically as follows:

publish-subscribe mode: after the producer publishes the message, the consumer subscribes to the corresponding topic partition and then consumes the message, the message is one-to-many like a notice in a broadcast of a school, all students can hear the notice, and for example, a subscription number in software is watched by a plurality of people, and when the number of the subscription number is mainly published an article, the concerned people can receive the notice. The application scene of the publishing and subscribing mode is quite many, for example, a plurality of services at the downstream need to use the same data, if the services are realized in a coding mode, the services can be called in an RPC mode, but the coupling of a system is caused, if a message middleware is used, the upstream only manages to deliver the message, and after the downstream service subscribes, the message can be consumed, so that the coupling is greatly reduced;

Point-to-point mode: the point-to-point mode is a one-to-one mode, and a consumption group is created, and the data of each partition can only be consumed by consumers in the consumption group, so that the point-to-point mode is realized. The application scenario of the peer-to-peer mode is also quite many, because the data can only be used by one consumer, for example, peak clipping can be performed, for example, the upstream service sends quite much data, if the downstream service only has one consumer instance, the message may be backlogged, and at this time, the consumption speed of the message is increased by more than a few consumer instances to consume together, but some factors, such as the sequence of the messages, must be considered.

Referring to fig. 5, fig. 5 is an optional structural schematic diagram of obtaining a fault alarm message through consumption of an alarm analysis unit in the embodiment of the present application, where the fault alarm consumption queue includes message partition queues a and B, alarm analysis processes C1 to C4 are included in the alarm analysis unit, the alarm analysis processes C1 to C4 form a consumer group, a publish-subscribe mode is adopted to consume the fault alarm consumption queue, the alarm analysis process C1 corresponds to the consumption group a, the alarm analysis process C2 corresponds to the consumption group B, the alarm analysis process C3 corresponds to the consumption group C, the alarm analysis process C4 corresponds to the consumption group D, the alarm analysis processes C1 and C2 consume a plurality of fault alarm information in the message partition queue a, and the alarm analysis processes C3 and C4 consume a plurality of fault alarm information in the message partition queue B.

Referring to fig. 6, fig. 6 is another optional structural schematic diagram of obtaining fault alarm messages through consumption of an alarm analysis unit in the embodiment of the present application, where the fault alarm consumption queue includes message partition queues A, B and C, alarm analysis processes C1 to C5 are included in the alarm analysis unit, the alarm analysis processes C1 to C5 form a consumer group, a point-to-point mode is adopted to consume the fault alarm consumption queue, alarm analysis processes C1 to C3 correspond to consumption group a, alarm analysis processes C4 and C5 correspond to consumption group B, multiple fault alarm information in the alarm analysis processes C1 and C4 consume the fault alarm information in the message partition queue a, multiple fault alarm information in the alarm analysis processes C2 and C5 consume the message partition queue B, and the alarm analysis process C3 consumes the fault alarm information in the message partition queue C.

Referring to fig. 7, fig. 7 is another optional structural schematic diagram of obtaining fault alarm messages through consumption of an alarm analysis unit in this embodiment, where the fault alarm consumption queue includes message partition queues A, B, C and D, alarm analysis processes C1 to C6 are in the alarm analysis unit, the alarm analysis processes C1 to C6 form a consumer group, and simultaneously adopt a peer-to-peer mode and a publish-subscribe mode, consume the fault alarm consumption queue, the alarm analysis processes C1 to C3 correspond to the consumption group a, the alarm analysis processes C4 and C5 correspond to the consumption group B, the alarm analysis process C6 corresponds to the consumption group C, the alarm analysis process C1 consumes a plurality of fault alarm messages in the message partition queues a and B, the alarm analysis process C2 consumes a plurality of fault alarm messages in the message partition queue C, the alarm analysis process C3 consumes a plurality of fault alarm messages in the message partition queues a and B, the alarm analysis process C4 consumes a plurality of fault alarm messages in the alarm analysis process C5 consumes a plurality of fault alarm messages in the message partition queues C and D, and the alarm analysis process C6 consumes a plurality of fault messages in the message partition queues C and C.

In some embodiments, the fault repairing unit is further configured to monitor a fault repairing overall process corresponding to the fault alarm repairing process, and generate corresponding fault repairing log information according to the fault repairing overall process and a fault repairing result.

Specifically, the fault repairing processing unit receives a fault repairing instruction output by the alarm decision unit, the alarm decision unit issues a target fault alarm repairing script to the fault repairing processing unit through an automatic operation and maintenance tool such as an active and a Saltstack, and executes fault repairing instructions such as a repair_slow_sql, a repair_node_time_sync, a repair_mySQL_threads and/or a repair_mySQL_reply_delay to repair the fault, and after the fault repairing is successful, the fault processing process and the fault processing result are saved in a storage disk in a repairing log file format, so that the traceability is provided.

In some embodiments, referring to fig. 8, fig. 8 is another optional structural schematic diagram of a fault self-healing system based on MySQL database provided in the embodiments of the present application, where the fault self-healing system includes an alarm monitoring module, a fault analysis decision module, a fault repair processing module and a self-healing notification module, the alarm monitoring module includes a data acquisition unit and an alarm message acquisition unit, and the fault analysis decision module includes a script management unit, an alarm analysis unit and an alarm decision unit.

Referring to fig. 9, fig. 9 is an optional flowchart of a fault self-healing method based on MySQL database according to an embodiment of the present application, where the method may include, but is not limited to, steps S101 to S103:

step S101, monitoring index data of a MySQL database is collected, fault alarm monitoring is carried out according to the monitoring index data, and fault alarm information is obtained;

step S102, performing fault alarm analysis on the fault alarm information, determining a target fault alarm scene corresponding to the fault alarm information, performing fault alarm decision according to the target fault alarm scene and the fault alarm information, and determining a target fault alarm repair script;

and step S103, performing fault alarm repair processing on the MySQL database by utilizing the target fault alarm repair script according to the fault alarm information to obtain a fault repair processing result.

The specific implementation of the fault self-healing method based on the MySQL database is basically the same as the specific embodiment of the fault self-healing system based on the MySQL database, and is not repeated here.

The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the fault self-healing method based on the MySQL database when executing the computer program. The electronic equipment can be any intelligent terminal including a tablet personal computer, a vehicle-mounted computer and the like.

Referring to fig. 10, fig. 10 illustrates a hardware structure of an electronic device according to another embodiment, the electronic device includes:

the processor 901 may be implemented by a general purpose CPU (central processing unit), a microprocessor, an application specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided by the embodiments of the present application;

the memory 902 may be implemented in the form of read-only memory (ReadOnlyMemory, ROM), static storage, dynamic storage, or random access memory (RandomAccessMemory, RAM). The memory 902 may store an operating system and other application programs, and when the technical solution provided in the embodiments of the present disclosure is implemented by software or firmware, relevant program codes are stored in the memory 902, and the processor 901 invokes the fault self-healing method based on MySQL database to execute the embodiments of the present disclosure;

an input/output interface 903 for inputting and outputting information;

the communication interface 904 is configured to implement communication interaction between the device and other devices, and may implement communication in a wired manner (e.g. USB, network cable, etc.), or may implement communication in a wireless manner (e.g. mobile network, WIFI, bluetooth, etc.);

A bus 905 that transfers information between the various components of the device (e.g., the processor 901, the memory 902, the input/output interface 903, and the communication interface 904);

wherein the processor 901, the memory 902, the input/output interface 903 and the communication interface 904 are communicatively coupled to each other within the device via a bus 905.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the fault self-healing method based on the MySQL database when being executed by a processor.

The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

According to the fault self-healing system and method based on the MySQL database, monitoring index data of the MySQL database are collected through the alarm monitoring module, fault alarm monitoring is conducted, fault alarm information is obtained, fault alarm analysis and fault alarm decision are conducted through the fault analysis decision module, a target fault alarm scene and a target fault alarm repairing script are determined, fault alarm repairing processing is conducted on the MySQL database through the fault repairing processing module according to the fault alarm information by utilizing the target fault alarm repairing script, and a fault repairing processing result is obtained. The fault processing full-flow closed loop for fault alarming, fault analysis decision and fault self-healing of the MySQL database can be realized, the fault checking and repairing efficiency is improved, the stability of the online database is effectively improved, and the problem of low fault recovery timeliness of the online database is solved.

The embodiments described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and as those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

It will be appreciated by those skilled in the art that the technical solutions shown in the figures do not constitute limitations of the embodiments of the present application, and may include more or fewer steps than shown, or may combine certain steps, or different steps.

The system embodiments described above are merely illustrative, in that the units illustrated as separate components may or may not be physically separate, i.e., may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.

The terms "first," "second," "third," "fourth," and the like in the description of the present application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in this application, "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

In the several embodiments provided in this application, it should be understood that the disclosed systems and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative, e.g., the division of the above elements is merely a logical functional division, and there may be additional divisions in actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, system or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including multiple instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, or other various media capable of storing a program.

Preferred embodiments of the present application are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.

Claims

1. A MySQL database-based fault self-healing system, the system comprising:

2. A self-healing system according to claim 1, wherein the system further comprises:

3. The fault self-healing system according to claim 1, wherein the alarm monitoring module includes a data acquisition unit and an alarm message acquisition unit;

4. A fault self-healing system according to claim 3, wherein the fault analysis decision module comprises a script management unit and an alarm analysis unit;

5. The fault self-healing system of claim 4, wherein the fault analysis decision module further comprises an alarm decision unit;

6. The fault self-healing system according to claim 1, wherein the fault repair processing unit is configured to parse the fault alarm information to obtain fault alarm parsing information;

7. The self-healing system according to claim 4, wherein the fault alert scenario includes at least a host down alert, a host CPU usage high alert, a host Load high alert, a host memory usage high alert, a disk space shortage alert, a host time error alert, a MySQL active session alert, a MySQL standby library delay alert, a MySQL connection number alert, a MySQL database down alert, a MySQL master slave replication disconnect alert, a MySQL database QPS/TPS increase exceeding a threshold alert, and a MySQL database slow SQL Load alert.

8. The fault self-healing system according to claim 4, wherein the alarm message collection unit is configured to set a plurality of message partitions of the fault alarm message queue, and write a plurality of fault alarm information into each of the message partitions;

9. The self-healing system according to claim 6, wherein the fault repair processing unit is further configured to monitor a fault repair overall process corresponding to the fault alarm repair process, and generate corresponding fault repair log information according to the fault repair overall process and the fault repair process result.

10. The fault self-healing method based on the MySQL database is characterized by comprising the following steps of: