CN118689710A - A disaster recovery switching device for automated operation and maintenance system - Google Patents

Abstract

The invention discloses a disaster recovery switching device for an automatic operation and maintenance system, which is used for restoring a database of the device to a point and improving the recovery efficiency of data; the problem of saving the data of the component, the flow and the like which are being edited is considered, the possibility of data loss is reduced, and the reliability of disaster recovery switching is improved; meanwhile, the problem of proxy end connection efficiency is solved, and disaster recovery switching efficiency is greatly improved. The self-defined configuration monitoring mode of the invention improves user experience, flexibility and expansibility, solves the defect that the traditional mode relies on a network to judge whether a disaster occurs in a data center, reduces misjudgment caused by occasional network fluctuation, and improves the accuracy of disaster judgment.

Description

A disaster recovery switching device for automated operation and maintenance system

Technical Field

The present invention relates to the field of operation and maintenance systems, and in particular to a disaster recovery switching device for an automated operation and maintenance system.

Background Art

With the rapid development of IT technology, most companies currently use IT technology to support the normal operation of business systems, and the architecture of the business system itself is becoming more and more complex, involving more and more operational links and related personnel, especially in the financial industry. In order to meet the operation and maintenance management needs of the financial industry, ensure the quality of operation and maintenance management, and reduce operation and maintenance costs, most financial industries have used automated operation and maintenance systems to assist in the internal operation and maintenance management of the enterprise. Therefore, when a disaster occurs, it will become particularly important to ensure the availability and continuity of the automated operation and maintenance system.

In this context, the current automated operation and maintenance system faces the following problems when responding to disasters:

1. Manual operation is cumbersome and costly: Due to the complexity of the system architecture, the operation and maintenance personnel need to be very familiar with the system architecture, otherwise they cannot respond to disasters quickly and efficiently. In addition, the operation and maintenance personnel need to perform a series of operations when performing disaster switching, which consumes a lot of human resources and costs. In addition, manual judgment and disaster recovery switching operations are prone to errors, which affects the stability of the system.

2. Disaster recovery switching takes a long time: On the one hand, the complexity of the system and the tediousness of manual operation lead to a long time for disaster recovery switching. On the other hand, the automated operation and maintenance system has a large amount of data. When disaster recovery occurs, data recovery takes a long time. Furthermore, the automated operation and maintenance system relies on the agent to execute various business needs. When a disaster occurs, the connection processing of the agent takes a long time. All of the above reasons have greatly reduced the availability and continuity of the automated operation and maintenance system.

3. Single switching strategy: Currently, the disaster recovery switching operation of the automated operation and maintenance system can only be performed when a disaster actually occurs. The user subjectively judges and manually executes which backup data center to switch to in order to ensure the normal operation of the automated operation and maintenance system. This single switching judgment strategy can only be determined by human subjectivity, and cannot form an automatic and objective judgment, thus losing objectivity.

4. Poor data consistency: When a disaster occurs while a user is performing editing operations on components or processes, the unsaved data being edited will be lost, requiring the user to re-edit it, resulting in poor data consistency and business continuity.

Under the premise of the above background, there are currently some disaster recovery monitoring and switching methods, but none of them can solve all the above problems at the same time. Therefore, a disaster recovery switching device for an automated operation and maintenance system is needed to solve the problems mentioned above.

Summary of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a disaster recovery switching device for an automated operation and maintenance system.

The purpose of the present invention is achieved through the following technical solutions:

A disaster recovery switching device for an automated operation and maintenance system comprises a data acquisition and analysis module, a disaster recovery scheme selection module, a disaster recovery switching execution module, a disaster recovery switching verification module, and a disaster recovery switching data recovery module; wherein:

The data collection and analysis module is used to obtain environmental data from the automated operation and maintenance system and determine whether a disaster has occurred in the current data center based on preset judgment rules;

The disaster recovery plan selection module is used to determine which backup data center will be switched to when a disaster occurs. When the automatic switching task is configured, the backup data center is obtained from the automated operation and maintenance system configuration. Otherwise, the user determines the backup data center by clicking.

The disaster recovery switching execution module is used to execute disaster recovery switching, switch the data center from the primary data center to the backup data center, and perform a series of switching operations, including database state change, database restore point setting, and firewall setting;

The disaster recovery switch verification module is used to execute the verification process after the disaster recovery switch, check whether the process, components, tasks are normal, and whether the connection of the agent is normal, confirm that the disaster recovery switch is executed correctly, and the backup data center is currently available;

The disaster recovery switching data recovery module is used to determine whether the original main data center has returned to normal based on the data from the data collection and analysis module. If it has returned to normal, it will automatically or manually switch back to the production data center.

In the disaster recovery plan selection module, the automatic switching task is configured by the user in advance to configure the disaster recovery connection relationship of each data center, that is, when a disaster occurs in the current data center, which backup data center will be used to continue operation.

The specific working process of the disaster recovery solution selection module is as follows:

Step 1: The automated operation and maintenance system monitors several indicators of the production environment in real time and calculates the status value of the production data center. If the status value of the production data center is greater than a preset value, the current production data center is judged to be in an abnormal state and the process goes to step 2.

Step 2: Determine whether the automatic operation and maintenance system is configured with an automatic switching task. If so, the system automatically obtains the disaster recovery connection relationship of the current data center and performs a disaster recovery switching operation, so that the automatic operation and maintenance system switches to the backup data center. Otherwise, the user subjectively selects which backup data center to switch to and clicks the switch button for that data center.

When actually performing disaster recovery switching, the time point when the disaster occurred is first set as the database restore point. The setting method is that the management system of the automated operation and maintenance system first cancels the recovery operation of the managed database currently in progress, then creates a recovery point and specifies the recovery database, and finally disconnects the connection between the primary database and the backup database to improve the efficiency of data recovery after disaster recovery. Data recovery only starts from the restore point, and the execution data within the disaster time period is incrementally restored to the production database, greatly shortening the data recovery time;

Then the management system of the automated operation and maintenance system changes the "Quick Connection" setting of the server to the enabled state, which is used by the agent to quickly connect to the server and ensure that the business can be executed normally using the agent; then the firewall configuration of the operating system and server is turned on to intercept all requests sent to the production data center and return the interception information to the sender of the request; then the recovery operation of the currently ongoing managed database is canceled and the standby data center is activated, so that the database of the standby data center becomes readable and writable, making preliminary preparations for running the business in the standby data center in the future, and writing the operating data to the standby database.

In step 1, the state value Q of the production data center is calculated as follows:

If the system administrator has no special requirements for the five monitoring indicators of memory usage, network latency, CPU usage, thread usage, and number of online servers, the default algorithm of the automated operation and maintenance system is used to determine the status of the production data center:

Q=0.18x+0.3y+0.15z+0.12i+0.25j;

Among them, the memory ratio is x, the network delay is y, the CPU ratio is z, the number of threads is i, and the number of online servers is j;

If the system administrator has special requirements for the five monitoring indicators of memory share, network latency, CPU share, thread share, and number of online servers, the administrator can customize the indicator critical value, indicator occurrence number, and indicator weight to calculate the status value Q of the new production data center.

The agent of the automated operation and maintenance system is started in a "quick connection" manner, that is, first ensuring that the agent can quickly and stably connect to the server, and then processing the connection check of the agent in an asynchronous manner.

Assuming that the database of the automated operation and maintenance system is created at time point t0, then when no disaster occurs, the data is synchronized from the primary data center to the backup data center in real time, and the data of the two data centers remain synchronized; when a disaster occurs at time point t2, the primary data center no longer synchronizes data to the backup data center, and time point t2 is used as the database restore point; during the disaster period from t2 to t6, the automated operation and maintenance system has switched to the backup data center, and all generated operation data is in the backup data center; when the disaster is recovered normally at time point t6, the operation data in the time period from t2 to t6 is restored to the primary data center, and from then on, the data continues to be synchronized from the primary data center to the backup data center in real time.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The present invention uses database restore points to improve data recovery efficiency.

2. The present invention takes into account the preservation of data such as components and processes being edited, reduces the possibility of data loss, and improves the reliability of disaster recovery switching.

3. The present invention solves the problem of proxy connection efficiency and greatly improves the efficiency of disaster recovery switching.

4. The custom-configured monitoring method of the present invention improves user experience, flexibility and scalability, while solving the drawbacks of the traditional method of relying on the network to determine whether a disaster has occurred in the data center, reducing misjudgments caused by occasional network fluctuations and improving the accuracy of disaster judgment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is an architecture diagram of the automated operation and maintenance system of the present invention.

FIG2 is a diagram showing a multi-data center structure of the automated operation and maintenance system of the present invention.

FIG3 is a schematic diagram of the structure of the disaster recovery switching device for the automated operation and maintenance system according to the present invention.

FIG. 4 is a schematic diagram of data recovery to a primary data center according to the present invention.

FIG5 is a flow chart of the disaster recovery switching according to the present invention.

DETAILED DESCRIPTION

The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

1. Introduction to the disaster recovery architecture of the automated operation and maintenance system.

As shown in Figure 1, we first introduce the architecture of the automated operation and maintenance system (as shown in Figure 1), which consists of three parts: server, control, and agent. The server is used to provide automated operation and maintenance services and related functions. It is deployed in a cluster and uses redis clusters to improve read and write capabilities. The database type supports multiple types such as mysql, oracle, and Gauss. This article takes oracle as an example and adopts the mode of real-time synchronization between the main database and the backup database; the control connects to the server and is the user's operation page. You can edit components, processes, tasks, etc. in the control console; the agent connects to the server and is mainly responsible for executing tasks and processes. Only when the agent is stably connected to the server can the normal execution of tasks and processes be guaranteed.

As shown in Figure 2, the automated operation and maintenance system can deploy a production data center (i.e., the main data center) and multiple backup data centers. The production data center and each backup data center can be connected to each other for disaster recovery. Under normal circumstances, the automated operation and maintenance system runs in the production data center, and other backup data centers only serve as backup operating environments. Data is synchronized from the production data center to each backup data center in real time.

2. Disaster event judgment model.

The present invention establishes a disaster event judgment model. Through the automated operation and maintenance system, the five indicators of memory share, network delay, CPU share, thread share, and number of online servers of the production data center are obtained and calculated every 5 seconds. Combined with the number of indicator occurrences and indicator weights, it is calculated whether the current production data center is in an abnormal state. If it is in an abnormal state, a switching operation is performed; otherwise, the status quo is maintained.

Implementation case 1: If the system administrator has no special requirements for the five monitoring indicators, the system's own algorithm is used by default to determine the status of the production data center. The specific algorithm is shown below.

For memory usage, when the percentage is in the range of [0,60%] and the number of occurrences is less than or equal to 5 times, it is judged as a normal alarm level; when the percentage is in the range of [0,60%] and the number of occurrences is greater than 5 times, it is judged as a general alarm level; when the percentage is in the range of (60%,80%] and the number of occurrences is less than or equal to 5 times, it is judged as a general alarm level; when the percentage is in the range of (60%,80%] and the number of occurrences is greater than 5 times, it is judged as a minor alarm level; when the percentage is in the range of (80%,100%] and the number of occurrences is less than or equal to 5 times, it is judged as a minor alarm level; when the percentage is in the range of (80%,100%] and the number of occurrences is greater than 5 times, it is judged as a serious alarm level.

Table 1 Memory usage alarm levels

For network delay, when the network delay is in the range of [0ms, 50ms] and the number of occurrences is less than or equal to 10 times, it is judged as a normal alarm level; when the network delay is in the range of [0ms, 50ms] and the number of occurrences is greater than 10 times, it is judged as a general alarm level; when the network delay is in the range of (50ms, 100ms] and the number of occurrences is less than or equal to 10 times, it is judged as a general alarm level; when the network delay is in the range of (50ms, 100ms] and the number of occurrences is greater than 10 times, it is judged as a minor alarm level; when the network delay is in the range of (100ms, +∞) and the number of occurrences is less than or equal to 10 times, it is judged as a minor alarm level; when the network delay is in the range of (100ms, +∞) and the number of occurrences is greater than 10 times, it is judged as a serious alarm level.

Table 2 Network delay alarm levels

For the CPU ratio, when the ratio is in the range of [0,60%] and the number of occurrences is less than or equal to 8 times, it is judged as a normal alarm level; when the ratio is in the range of [0,60%] and the number of occurrences is greater than 8 times, it is judged as a general alarm level; when the ratio is in the range of (60%,80%] and the number of occurrences is less than or equal to 8 times, it is judged as a general alarm level; when the ratio is in the range of (60%,80%] and the number of occurrences is greater than 8 times, it is judged as a minor alarm level; when the ratio is in the range of (80%,100%] and the number of occurrences is less than or equal to 8 times, it is judged as a minor alarm level; when the ratio is in the range of (80%,100%] and the number of occurrences is greater than 8 times, it is judged as a serious alarm level.

Table 3 CPU usage alarm levels

For the thread number ratio, when the ratio is in the range of [0,60%] and the number of occurrences is less than or equal to 8 times, it is judged as a normal alarm level; when the ratio is in the range of [0,60%] and the number of occurrences is greater than 8 times, it is judged as a general alarm level; when the ratio is in the range of (60%,80%] and the number of occurrences is less than or equal to 8 times, it is judged as a general alarm level; when the ratio is in the range of (60%,80%] and the number of occurrences is greater than 8 times, it is judged as a minor alarm level; when the ratio is in the range of (80%,100%] and the number of occurrences is less than or equal to 8 times, it is judged as a minor alarm level; when the ratio is in the range of (80%,100%] and the number of occurrences is greater than 8 times, it is judged as a serious alarm level.

Table 4 Thread ratio alarm level

For the number of servers online, when the number of online servers is not zero and the number of occurrences is greater than or equal to 2 times, it is judged as a normal alarm level; when the number of online servers is not zero and the number of occurrences is less than 2 times, it is judged as a general alarm level; when the number of online servers is zero and the number of occurrences is greater than or equal to 2 times, it is judged as a serious alarm level; when the number of online servers is zero and the number of occurrences is less than 2 times, it is judged as a minor alarm level.

Table 5 Alarm levels for online server count

Assume that the memory ratio is x, the network latency is y, the CPU ratio is z, the thread ratio is i, the number of online servers is j, and the weights of these indicators are 0.18, 0.3, 0.15, 0.12, and 0.25 respectively, and the sum of the weights is 1. Then the state model Q of the production data center is established as:

Q=0.18x+0.3y+0.15z+0.12i+0.25j;

The automated operation and maintenance system obtains and calculates the data of five indicators of the production data center every 5 seconds: memory share, network latency, CPU share, thread share, and number of online servers. It also determines the alarm level of the event based on the number of occurrences of the indicators. If the alarm level of the event is severe, the variable corresponding to the indicator is 4; if the alarm level of the event is minor, the variable corresponding to the indicator is 3; if the alarm level of the event is general, the variable corresponding to the indicator is 2; if the alarm level of the event is normal, the variable corresponding to the indicator is 1. Then, substitute it into Q to calculate the status value of the production data center. If the status value Q is greater than 3.6, it is determined that the production data center is in an abnormal state and a disaster recovery switch operation is required. For example: According to the rules in the previous article, the alarm level of memory usage is severe, the alarm level of network delay is minor, the alarm level of CPU usage is general, the alarm level of thread number usage is normal, and the alarm level of online server number is normal. Then Q=0.18*4+0.3*3+0.15*2+0.12*1+0.25*1=2.29. If the value is less than 3.6, the status of the data center is normal and no disaster recovery switching is required.

Implementation case 2: If the system administrator has special requirements for the five monitoring indicators, the administrator can customize the indicator critical value, indicator occurrence frequency and indicator weight. Assume that the information configured by the system administrator is shown in the following table.

Table 6 Custom indicator configuration table

Based on the indicator critical values and indicator occurrence times customized by the system administrator, the rules for the automated operation and maintenance system to determine the event alarm level are as follows.

For memory usage, when the percentage is in the range of [0,70%] and the number of occurrences is less than or equal to 5 times, it is judged as a normal alarm level; when the percentage is in the range of [0,70%] and the number of occurrences is greater than 5 times, it is judged as a general alarm level; when the percentage is in the range of (70%,100%] and the number of occurrences is less than or equal to 5 times, it is judged as a minor alarm level; when the percentage is in the range of (70%,100%] and the number of occurrences is greater than 5 times, it is judged as a serious alarm level.

Table 7 Memory usage alarm levels

For network delay, when the network delay is in the range of [0ms, 50ms] and the number of occurrences is less than or equal to 10 times, it is judged as a normal alarm level; when the network delay is in the range of [0ms, 50ms] and the number of occurrences is greater than 10 times, it is judged as a general alarm level; when the network delay is in the range of (50ms, +∞) and the number of occurrences is less than or equal to 10 times, it is judged as a minor alarm level; when the network delay is in the range of (50ms, +∞) and the number of occurrences is greater than 10 times, it is judged as a severe alarm level.

Table 8 Network delay alarm levels

For the CPU ratio, when the ratio is in the range of [0,65%] and the number of occurrences is less than or equal to 5 times, it is judged as a normal alarm level; when the ratio is in the range of [0,65%] and the number of occurrences is greater than 5 times, it is judged as a general alarm level; when the ratio is in the range of (65%,100%] and the number of occurrences is less than or equal to 5 times, it is judged as a minor alarm level; when the ratio is in the range of (65%,100%] and the number of occurrences is greater than 5 times, it is judged as a severe alarm level.

Table 9 CPU usage alarm levels

For the thread number ratio, when the ratio is in the range of [0,68%] and the number of occurrences is less than or equal to 5 times, it is judged as a normal alarm level; when the ratio is in the range of [0,68%] and the number of occurrences is greater than 5 times, it is judged as a general alarm level; when the ratio is in the range of (68%,100%] and the number of occurrences is less than or equal to 5 times, it is judged as a minor alarm level; when the ratio is in the range of (68%,100%] and the number of occurrences is greater than 5 times, it is judged as a serious alarm level.

Table 10 Thread ratio alarm level

For the number of servers online, when the number of online servers is not zero and the number of occurrences is greater than or equal to 2 times, it is judged as a normal alarm level; when the number of online servers is not zero and the number of occurrences is less than 2 times, it is judged as a general alarm level; when the number of online servers is zero and the number of occurrences is greater than or equal to 2 times, it is judged as a serious alarm level; when the number of online servers is zero and the number of occurrences is less than 2 times, it is judged as a minor alarm level.

Table 11 Alarm levels for online server count

Combined with the five indicator weights customized by the system administrator, the production data center status model Q is:

Q=0.2x+0.3y+0.1z+0.1i+0.3j;

The automated operation and maintenance system obtains and calculates the status value of the production data center every 5 seconds. The calculation method is the same as before and will not be repeated. If the status value Q is greater than 3.6, it is determined that the production data center is in an abnormal state and a disaster recovery switching operation is required.

This judgment mode combines the number of indicator occurrences and weights to clarify the administrator's priority and focus on indicators, solving the drawbacks of the traditional method of relying on the network to judge whether a disaster has occurred in the data center, improving the accuracy of disaster judgment, and optimizing the decision-making process. At the same time, the custom configuration monitoring method has the advantages of improving user experience, flexibility, and scalability.

3. Disaster Switching Model

As shown in Figure 3, the disaster recovery switching device for the automated operation and maintenance system is suitable for disaster recovery switching condition monitoring and automatic operation and maintenance system primary and standby data center switching. The device specifically includes: data collection and analysis module, disaster recovery plan selection module, disaster recovery switching execution module, disaster recovery switching verification module, and disaster recovery switching data recovery module.

Data collection and analysis module: used to obtain environmental data of the automated operation and maintenance system, and determine whether a disaster has occurred in the current data center based on judgment rules.

Disaster recovery plan selection module: used to determine which backup data center to switch to when a disaster occurs. When the automatic switching task is configured, the backup data center is obtained from the system configuration, otherwise the user determines the backup data center by clicking.

Disaster recovery switching execution module: used to execute disaster recovery switching, switch the data center from the primary data center to the backup data center, and perform a series of switching operations, including database status change, database restore point setting, firewall setting, etc.

Disaster recovery switch verification module: used to execute the verification process after disaster recovery switch, check whether the process, components, tasks, etc. are normal, and whether the connection of the agent is normal, confirm that the disaster recovery switch is executed correctly, and that the backup data center is currently available.

Disaster recovery switching data recovery module: used to determine whether the original main data center has returned to normal based on the data from the "data collection and analysis module". If it has returned to normal, it will automatically or manually switch back to the production data center.

When a disaster occurs, a series of operations such as switching data centers need to be performed to ensure that the processes and tasks of the automated operation and maintenance system can run normally in the backup data center. This article provides two switching methods, namely manual switching and automatic task switching. Automatic task switching is when the user configures the disaster recovery connection relationship of each data center in advance, that is, when a disaster occurs in the current data center, which backup data center to use to continue running. Manual switching is when a disaster actually occurs, the user subjectively determines which backup data center to switch to, and clicks the switch button for that data center. As shown in Figure 5, the specific switching steps are as follows:

Step 1: The management system monitors the five indicators of the production environment in real time: memory usage, network latency, CPU usage, thread usage, and number of online servers. The status value Q of the production data center is calculated using the algorithm described above. If Q is greater than 3.6, the current production data center is judged to be in an abnormal state.

Step 2: When the current environment is determined to be abnormal based on the monitoring data in step 1, determine whether an automatic switching task is configured. If an automatic switching task is configured, the system automatically obtains the disaster recovery connection relationship of the current data center and executes the disaster recovery switching operation, so that the automated operation and maintenance system switches to the backup data center. Otherwise, the user subjectively determines which backup data center to switch to and clicks the switch button for that data center.

When actually performing disaster recovery switching, the time point when the disaster occurred is first set as the database restore point. The setting method is that the management system first cancels the recovery operation of the currently ongoing managed database, then creates a recovery point and specifies the recovery database, and finally disconnects the connection between the primary database and the backup database. The operation aims to improve the efficiency of data recovery after disaster recovery. Data recovery only starts from the restore point, and the execution data within the disaster time period is incrementally restored to the production database, greatly shortening the data recovery time.

Then the management system changes the "Quick Connection" setting of the server to the enabled state, which is used by the agent to quickly connect to the server and ensure that the business can be executed normally using the agent. Then, the firewall configuration of the operating system and the server is enabled to intercept all requests sent to the production data center and return the interception information to the sender of the request. Then, the recovery operation of the currently ongoing managed database is canceled, and the backup data center is activated, so that the database of the backup data center becomes readable and writable, making preliminary preparations for running the business in the backup data center in the future, and writing the running data to the backup database.

In this embodiment, the issue of saving the data of components, processes, etc. being edited is taken into consideration. For data such as components, processes, and indicators, even if the user does not click the Save button, the server will save the data as a temporary db file every 5 minutes. The file will be synchronized with the database to each backup data center. When the user clicks Save, the temporary file is automatically cleared to release storage space. If a disaster occurs while the user is editing, after the disaster recovery switch is completed, the backup data center will automatically read the data of the temporary db file to the backup data center for use, reducing the possibility of data loss and improving the reliability of the disaster recovery switch.

In this embodiment, the problem of agent connection efficiency is solved. Since the agent is the key to task execution, the backup data center needs to handle the connection problems of tens of thousands of agents. In traditional disaster recovery switching, batches of agent connections will cause the server to be unresponsive or even restart. In this method, the agent is started using the "quick connection" method, which first ensures that the agent can connect to the server quickly and stably, and then processes the agent connection check in an asynchronous manner. After using this method, the connection rate of 20,000 agents has been shortened from the original 15 to 20 minutes to the current 1 to 2 minutes, greatly improving the efficiency of disaster recovery switching. In addition, this solution also solves the problem of console freezes caused by a large number of agent connections, and enables the console to operate normally by configuring the system to retain the number of threads.

Normal agent startup requires checking the agent's indicators, tasks and other information. Only when the check is passed will the agent connect to the server. This checking process is very time-consuming. The logic of turning on "Quick Connection" is to connect to the server first, and then start checking when the server is free.

Step 3: After the automated operation and maintenance system has switched to the backup data center, it is necessary to run the corresponding verification process to check whether key components, processes, tasks, etc. exist, and whether the agent is online. If the verification process is executed normally, it indicates that the disaster recovery switch is successful. Otherwise, it indicates that the disaster recovery switch has failed, and a notification is sent to the corresponding administrator.

Step 4: The management system still monitors in real time whether the production environment has returned to normal. When the abnormal judgment conditions are not met, it is determined that the production environment has returned to normal, and the system's switching method is obtained. If the switching method is a manual switching task, the administrator user clicks on the production data center to switch, otherwise the system automatically obtains the pre-configured disaster recovery switching plan. When actually performing disaster recovery, first activate the production data center so that the database of the production data center becomes readable and writable. Then turn off the system's firewall settings so that all requests can be sent to the production data center. Next, starting from the restore point, all data generated after the restore point time is restored to the production database. After the data recovery is completed, clear the database restore point.

As shown in Figure 4, in this embodiment, the problem of data recovery efficiency is solved by using database restore points. Assuming that the database is created at time point t0, the data is synchronized from the primary data center to the backup data center in real time during the period when no disaster occurs, and the data of the two data centers remain synchronized. When a disaster occurs at time point t2, the primary data center no longer synchronizes data to the backup data center, and uses time point t2 as the database restore point. During the disaster period from t2 to t6, the automated operation and maintenance system has switched to the backup data center, and all generated operation data is in the backup data center. When the disaster is recovered normally at time point t6, the operation data in the time period from t2 to t6 is restored to the primary data center, and from then on the data remains synchronized from the primary data center to the backup data center in real time. In this way, the efficiency of data recovery to the primary data center is greatly improved.

In this embodiment, the problems of proxy connection efficiency and data recovery efficiency are solved. If the automatic switching task is configured, the system can automatically determine the situation of the environment and implement automatic switching, providing users with a smooth, seamless and natural experience.

Step 5: After the automated operation and maintenance system has switched to the production data center, the corresponding verification process needs to be run to check whether key components, processes, tasks, etc. exist, and whether the agent is online. If the verification process is executed normally, it indicates that the disaster recovery switch is successful. Otherwise, it indicates that the disaster recovery switch has failed, and a notification is sent to the corresponding administrator.

The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims (6)

1. A disaster recovery switching device for an automatic operation and maintenance system is characterized in that: the disaster recovery system comprises a data acquisition and analysis module, a disaster recovery scheme selection module, a disaster recovery switching execution module, a disaster recovery switching verification module and a disaster recovery switching data recovery module; wherein:

The data acquisition and analysis module is used for acquiring the environmental data of the automatic operation and maintenance system and judging whether the disaster occurs in the current data center according to a preset judgment rule;

The disaster recovery scheme selection module is used for determining to which backup data center to switch when a disaster occurs; when the automatic switching task is configured, acquiring a standby data center from the configuration of the automatic operation and maintenance system, otherwise, determining the standby data center by a user in a clicking mode;

The disaster recovery switching execution module is used for executing disaster recovery switching, switching the data center from the main data center to the standby data center, and executing a series of switching operations, wherein the switching operations comprise database state transition, database restore point setting and firewall setting;

The disaster recovery switching verification module is used for executing verification flow after disaster recovery switching, checking whether the flow, the components and the tasks are normal or not and whether the connection of the proxy end is normal or not, and confirming that the disaster recovery switching is executed correctly and that the standby data center is in an available state currently;

and the disaster recovery switching data recovery module is used for judging whether the original main data center is recovered to be normal according to the data of the data acquisition and analysis module, and automatically or manually switching back to the production data center if the original main data center is recovered to be normal.

2. The disaster recovery switching device for an automated operation and maintenance system of claim 1, wherein: in the disaster recovery scheme selection module, the automatic switching task is that a user configures the disaster recovery connection relation of each data center in advance, namely, when the current data center has a disaster, which standby data center is used for continuous operation.

3. The disaster recovery switching device for an automated operation and maintenance system of claim 1, wherein: the specific working process of the disaster recovery scheme selection module is as follows:

Step 1, an automatic operation and maintenance system monitors a plurality of indexes of a production environment in real time, calculates a state value of a production data center, judges that the current production data center is in an abnormal state if the state value of the production data center is larger than a preset value, and transfers to the step 2;

Step 2, judging whether the automatic operation and maintenance system is configured with an automatic switching task, if so, automatically acquiring disaster recovery connection relation of the current data center by the system, executing disaster recovery switching operation to enable the automatic operation and maintenance system to switch to the standby data center, otherwise, subjectively selecting which standby data center needs to be switched to by a user, and clicking a switching button of the data center;

When disaster backup switching is actually executed, firstly, a time point at which a disaster occurs is set as a database restoration point, wherein the setting mode is that a management system of an automatic operation and maintenance system firstly cancels the restoration operation of a managed database which is currently in progress, then creates a restoration point and designates a restoration database, finally disconnects the connection operation between a main database and a backup database and is used for improving the data restoration efficiency after disaster restoration, the data restoration is only started from the restoration point, the execution data in a disaster time period is restored to a production database in an incremental mode, and the data restoration time is greatly shortened;

Then the management system of the automatic operation and maintenance system changes the quick connection of the server to be set as an enabling state, and is used for the quick connection of the proxy end to the server, so that the normal execution of the service can be ensured by using the proxy; then starting firewall configuration of an operating system and a server, intercepting all requests sent to a production data center, and returning interception information to a sender of the requests; then, the recovery operation of the currently-ongoing managed database is canceled, the standby data center is activated, so that the database of the standby data center becomes a readable and writable state, a preparation is made for running the service in the standby data center at a later stage, and running data is written into the standby database.

4. The disaster recovery switching device for an automated operation and maintenance system of claim 1, wherein: in step 1, the state value Q of the production data center is calculated as follows:

If the system administrator has no special requirements on the memory ratio, the network delay, the CPU ratio, the thread number ratio and the server online number of the five monitoring indexes, the state of the production data center is judged by default using an algorithm of an automatic operation and maintenance system:

Q=0.18x+0.3y+0.15z+0.12i+0.25j；

Wherein, the memory duty ratio is x, the network delay is y, the CPU duty ratio is z, the thread number duty ratio is i, and the server on-line number is j;

if the system administrator has special requirements on the memory ratio, the network delay, the CPU ratio, the thread number ratio and the server online number of the five monitoring indexes, the administrator can self-define and configure the index critical value, the index occurrence number and the index weight, so that the state value Q of the new production data center is calculated.

5. The disaster recovery switching device for an automated operation and maintenance system of claim 1, wherein: the proxy end of the automatic operation and maintenance system is started in a 'quick connection' mode, namely, the proxy end is ensured to be connected to a server quickly and stably, and then connection checking of the proxy end is processed in an asynchronous mode.

6. The disaster recovery switching device for an automated operation and maintenance system of claim 1, wherein: assuming that the database of the automatic operation and maintenance system is created at the time point t0, synchronizing data from the main data center to the standby data center in real time in the time when no disaster occurs, and keeping the data of the two data centers synchronous; when a disaster occurs at the time point t2, the main data center does not synchronize data to the standby data center any more, and takes the time point t2 as a database restore point; in the period from t2 to t6, the automatic operation and maintenance system is switched to the standby data center, and all generated operation data are in the standby data center; when disaster recovery is normal at the time point of t6, the operation data in the time period from t2 to t6 is restored to the main data center, and the data still remains synchronized from the main data center to the standby data center in real time after that.