JP3325785B2 - Computer failure detection and recovery method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting a failure of a computer.
The present invention relates to a recovery method, and more particularly to a method for detecting a failure occurring in a computer connected to a network with high reliability.

[0002]

2. Description of the Related Art Conventionally, depending on the contents of processing, there is a case where a temporary system shutdown is permitted, but it is necessary to immediately reboot the system to continue the operation of the system. As a cause of the system down, in many cases, the system can be immediately restored by rebooting such as a software failure in addition to a physical device failure.

[0003] In a system including a plurality of computers operating on the same network, for example, a plurality of servers each constructed by a personal computer (PC), each server has its own and mutual failure detection functions, and is operated. Software failures (failures) occurring during the detection can be detected in real time, and the failed and stopped server can be immediately restarted.

FIG. 7 is a diagram showing a conventional failure detection method for a PC server. Each of the PC servers 100 and 200 on the network is equipped with a dedicated board for failure detection in order to realize a failure detection function. This failure detection dedicated board 140,2
When detecting that the mounted PC servers 100 and 200 have failed, the 40 notifies other failure detection boards 140 and 240 of the failure via the dedicated line 150. Further, the failure detection dedicated boards 140 and 240 mutually monitor the operation of the other PC servers 200 and 100 by periodically communicating with the other failure detection dedicated boards 240 and 140. Failure detection dedicated board 24
The failure of the other PC servers 200, 100 can be detected by receiving a notification from the PC server 200, 100 when the notification from the PC server 200, 100 is received, or by detecting that communication with the other dedicated board for failure detection 240, 140 is disabled. As described above, each of the failure detection boards 140 and 240 is provided with the PC server 1 mounted therein.
When the failure of 00 or 200 is detected, a reboot instruction is issued and the PC servers 100 and 200 are restarted.

Therefore, for example, in a system in which an active system and a standby system are set and operated in a PC server, when the active system goes down, it is possible to inform the standby system of the fact that the operating system is down. Automatic switching to real-time, and the operating system that has failed can be immediately rebooted. In addition, even when the standby system has failed, the failure can be recovered by rebooting, and the standby system can be again on standby.

[0006]

However, conventionally, the failure detection function has been realized using only the dedicated board, and if the dedicated board breaks down, the failure occurring in the PC server cannot be detected. I will. In addition, a malfunction may be erroneously recognized due to a malfunction of the dedicated board. If high reliability of the dedicated board is pursued to solve this problem, the manufacturing cost will increase. Therefore, in order to perform the failure detection function, it is desired that the increase in cost be suppressed as much as possible while improving the reliability.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a computer failure detection / recovery method in which a computer determined to have failed is automatically recovered by restarting. Is to provide.

[0008]

In order to achieve the above object, a failure detection / recovery method for a computer according to the present invention uses a plurality of computers among at least three computers connected to a network as failure detection targets. A computer, wherein at least one of the failure detection target computers is a monitoring computer that detects a failure that has occurred in the failure detection target computer, wherein the failure detection target computer is independently operable, Self-failure detecting means for detecting a failure, network communication means for communicating with the other computers via a network, and restart means for restarting itself according to instructions, the monitoring computer is independent operable Te, a failure detection means for detecting the other of said fault detection target computer failure connected via a dedicated line network Network communication means for communicating with the other computers via the network, and operating state monitoring and control means for monitoring the occurrence of a failure in the system and restarting the failure detection target computer in which the failure is detected. The operating state monitoring control unit that has detected a failure via the failure detection unit attempts communication with the failure detection target computer that has detected the failure via a network, and the failure detection target computer that has detected the failure. If there is no response from the above, the restart instruction of the failure detection target computer in which the failure is detected via the failure detection means is performed via a dedicated line , and if the response is received, the failure is detected. It is determined that the self-failure detection means mounted on the detected failure detection target computer has failed, and the restart detection command of the failure detection target computer that has detected the failure via the network. Is provided.

Further, the failure detection target computer is also the monitoring computer.

In a system in which a plurality of computers among at least three computers connected to a network are used as failure detection target computers, the failure detection target computers are independently operable, and the mounted failure detection target computers are provided. Failure detection means for detecting a failure of the target computer and a failure of another of the failure detection target computers connected via a dedicated line ; a network communication means for communicating with the other computers via a network; Restart means for restarting the system, and operating state monitoring control means for monitoring the occurrence of a failure in the system and performing a restart process of the failure detection target computer in which the failure has been detected, via the network. The operating state monitoring control means of the failure detection target computer that has detected a failure in The attempts to communicate with computers, if there is a response from the computer, perform restart indication of the failure detection unit the failure detection target computer failure is detected via the via dedicated lines, is the response If not,
It is characterized in that it is determined that the network communication means mounted therein has failed, and the self-restart is performed.

[0011] According to the above invention, the existing network is effectively used in addition to using the failure detecting means.
A failure detection function has been realized. That is, the path for detecting a failure is duplicated. However,
In the present invention, duplication is performed not by mere duplication of the configuration but by different routes that effectively use the existing configuration. Thereby, the reliability of the failure detection function can be improved while suppressing an increase in cost.

[0012] Further, there is provided an operating state holding means for holding an operating state of the failure detection target computer.

Further, the operating state monitoring control means mounted on the failure detection target computer which has detected the failure controls execution of a restart instruction according to the operating state held by the operating state holding means. I do. That is, by referring to the operating state holding means, it is possible to grasp whether or not a restart instruction has already been issued, so that there is no need to give useless restart instructions.

[0014]

Preferred embodiments of the present invention will be described below with reference to the drawings.

Embodiment 1 FIG. 1 is an overall configuration diagram of a network system according to a first embodiment of a computer failure detection / recovery method according to the present invention. This network system includes two PC servers 1 to be detected as failures.
0 and 20, a PC 30 corresponding to a management device of the network system, and a LAN 2 connecting these. The PC servers 10, 20 and the PC 30 are always operating. LAN boards 12, 22, and 32 are mounted on the PC servers 10, 20 and the PC 30, respectively, and can communicate with other computers. The PC servers 10 and 20 are equipped with failure detection boards 14 and 24, respectively, for realizing a failure detection function. These boards 14 and 24 are connected to the dedicated cables 15 through the configuration control device 4. , 25. The failure detection boards 14 and 24 are provided with the mounted PC servers 10 and 20 respectively.
Can be operated independently even if is stopped, and a restart (reboot) instruction can be issued.

In each of the PC servers 10 and 20, a PC
A monitoring program for monitoring the operating state of the servers 10 and 20 is resident in the memory and is always executed, and constitutes the operating state monitoring units 16 and 26 together with the CPU. The operating state monitoring units 16 and 26 periodically reset specific flags (not shown) held by the failure detection boards 14 and 24. Each of the operating state monitoring units 16 and 26 is connected to the LAN 2
Communicates periodically via the other PC server 20,
They check each other to see if they are running. Further, it is possible to instruct the restart of its own computer and the restart to the other PC servers 10 and 20.

The configuration control device 4 includes all the failure detection targets,
In this example, an operation state table that holds the operation states of the PC servers 10 and 20 is provided, and the operation state of each of the PC servers 10 and 20 is grasped by updating the setting contents. The operating status is “normal” indicating that the operating status is normal, and “failing” indicating that the operating status is not working due to a failure.
And "restarting" indicating that the recovery operation is being performed after the occurrence of the failure.

The feature of the present embodiment is that not only the failure detection using the failure detection dedicated boards 14 and 24 but also the failure detection of the PC servers 10 and 20 by using the existing configuration, that is, the LAN 2. That's what we can do. In other words, the path for detecting a failure is duplicated. However, the present embodiment is characterized in that, instead of merely duplicating the configuration, duplication is performed by using a different path that effectively uses the existing configuration, thereby suppressing the increase in cost and improving the reliability of the failure detection function. The degree can be improved. Note that the failure in the present embodiment refers to an abnormality that can be recovered by rebooting the PC servers 10 and 20.

Next, the operation of the present embodiment will be described. First, the failure of the other PC servers 20, 10 is detected by the failure detection boards 14, 24, and the failed PC server is restarted. The operation will be described with reference to the flowchart shown in FIG. In this description, it is assumed that the PC server 10 breaks down.

The failure detection board 14 periodically resets an internal specific flag by a monitoring program. The reset operation is constantly monitored, and if the specific flag is not reset for a certain period of time. Is detected (step 101), it is determined that the PC server 10 has failed. That is, it is determined that the PC server 10 has failed because the flag reset processing that is periodically performed by the memory-resident monitoring program is not performed. Upon making such a determination, the failure detection dedicated board 14 notifies the configuration controller 4 via the dedicated cable 15 (step 102).

The configuration control device 4 includes the failure detection board 1
4 (step 121), the operating status of the PC server 10 in the operating status table is changed from "normal" to "failed" (step 122). Then, by referring to the setting contents of the operation state table, a notification indicating that the PC server 10 has failed is transmitted to the failure detection dedicated board 24 of the PC server 20 whose operation state is "normal" (step 123). In this example, two PCs
The PC server 20 can be uniquely specified because it is composed of servers. However, if there are three or more PC servers and the operating states of a plurality of PC servers are “normal”, 1 One PC server is specified based on the conditions for selecting one PC server, for example, the order of registration or priority in the operating state table.

The operating state monitoring unit 26 of the PC server 20
When a report indicating that the PC server 10 has failed is received via the failure detection dedicated board 24 (step 141), in order to confirm the validity of the report, the PC reported as having failed is checked.
Attempt to communicate with the server 10 via the LAN 2 (step 142). As a result of this communication, if there is a response from the PC server 10 and the communication ends normally (step 14
3) It is determined that the failure is not the PC server 10 itself but the failure detection dedicated board 14. At this time, the operating state monitoring unit 26 sends the PC server 10 via the LAN 2 to recover the failure of the failure detection dedicated board 14.
(Step 144).

On the other hand, if there is no response from the PC server 10 as a result of the communication to the PC server 10 by the operation status monitoring unit 26, it is determined that the PC server 10 is not operating, and the validity of the report is confirmed. To end. Thereafter, the operating state monitoring unit 26 issues a reset instruction of the PC server 10 to the failure detection board 14 via the failure detection board 24 (step 145).

When the PC server 10 receives a restart instruction from the PC server 20 via the LAN 2 or the failure detection dedicated board 14, it performs a reboot (step 10).
3).

Further, when the operating state monitoring unit 26 issues a restart instruction to the PC server 10 through one of the above-mentioned routes,
The configuration controller 4 is notified that the PC server 10 is being restarted (step 146). Upon receiving the notification from the failure detection dedicated board 24, the configuration control device 4 changes the operating state of the PC server 10 in the operating state table from "failure" to "restarting" (step 124).

When the restart processing is completed in the PC server 10, the PC server 10 itself or the failure detection board 14 returns to a normal state. Thereafter, the operation state monitoring unit 16 notifies the configuration control device 4 that the restart has been completed via the failure detection dedicated board 14 (step 104). The configuration control device 4 includes a failure detection dedicated board 14.
, The operating status of the PC server 10 in the operating status table is changed from “restarting” to “normal” (step 125).

In this way, the fault detected by the fault detection board 14 can be confirmed by using the LAN 2 for the validity of the fault, and by causing the PC server 10 to restart, 10 or the board 14 dedicated to failure detection can be automatically recovered.

Next, the PC servers 10, 2 via the LAN 2
The operation of detecting the failure of 0 and restarting the failed PC server will be described with reference to the flowchart shown in FIG. Here, a description will be given assuming that the PC server 10 has failed.

The PC servers 10 and 20 periodically communicate via the LAN 2 by the monitoring program,
We are checking if they are running mutually.
Here, when the operation state monitoring unit 26 detects an abnormality in the communication with the PC server 10, it is estimated that the PC server 10 is not operating and has failed (step 241), and the operation is performed via the failure detection dedicated board 24. Then, the configuration controller 4 is notified of this (step 242).

Upon receiving the notification (step 221), the configuration control device 4 refers to the operating state of the PC server 10. As a result, "failure" or "restarting"
In this case, the PC server 10 has already received a notification from the PC server 10 that the PC has failed.
The PC server 10 has actually failed. Further, if the operating state of the PC server 10 is “normal”, there is a possibility that a notification to that effect has not yet been received from the PC server 10 due to a failure of the failure detection dedicated board 14 or the like. Therefore, the configuration control device 4 notifies the PC server 20 of any of them (step 222).

If the contents of the report from the configuration control device 4 indicate that the failure of the PC server 10 has been recognized (step 244), the PC server 20 ends the processing. This is because the PC server 10 does not need to perform the process related to the restart of the PC server 10 because the PC server 10 may have rebooted and restarted. Also, if the system has three or more PC servers, there is a possibility that another PC server has already proceeded with the restart processing. On the other hand, when the operation state of the PC server 10 is “normal”, the PC server 20 attempts communication with another PC 30 connected to the LAN 2 (steps 244 and 24).
5). If there is a response from the PC 30, it is determined at this point that the PC server 10 from which no response was obtained in the previous communication process has failed. On the other hand, if there is no response from the PC 30, it means that communication with both the PC server 10 and the PC 30 has failed. Therefore, the PC board 10 has not failed but the LAN board 22 mounted on the Judge that you are.

Therefore, the operating state monitoring unit 26 is connected to the PC 30
If there is a response from the PC, the failure detection of the PC server 10 is notified again via the failure detection dedicated board 24, and an instruction to restart the PC server 10 is issued (step 24).
6,247). If there is no response from PC 30,
Reboot is performed by notifying that its own PC server 20 has failed and by resetting itself (steps 246, 248, 249).

If the notification from the PC server 20 indicates that the PC server 10 is to be restarted (step 223), the configuration controller 4 changes the operating state of the PC server 10 from "normal" to "restarting" in the operating state table. To PC
The PC server 10 is attached to the failure detection board 14 of the server 10.
Is issued (step 224). PC server 1
0 reboots itself in accordance with the instruction of the failure detection dedicated board 14, and when the processing is completed, the operation state monitoring unit 16
The completion of the restart is notified to the configuration controller 4 via the failure detection board 14 (steps 201 and 20).
2). When the configuration control device 4 receives the notification from the failure detection dedicated board 14, the configuration control device 4 stores the PC server 1 in the operating state table.
The operating state of the “0” is changed from “restarting” to “normal” (step 225).

On the other hand, the configuration control device 4
If the notification from the server indicates that the PC server 20 is out of order, the operating status of the PC server 20 in the operating status table is changed from "normal" to "restarting" (step 22).
6). Then, when receiving the reboot completion report of the PC server 20 (step 250), the operating state of the PC server 20 is changed from "restarting" to "normal" (step 227). The restart of the PC server 20 is not performed by itself, but after the processing of step 226 by the configuration control device 4, a restart instruction is issued via the failure detection board 24 as in the case of the PC server 10. You may make it perform by receiving.

As described above, the validity of the failure can be confirmed by trying the failure detected using the LAN 2 by communicating with the other PC 30 connected to the LAN 2. Alternatively, the PC server 20 can be automatically recovered.

As described above, according to the present embodiment, the failure detection function can be performed by using not only the failure detection boards 14 and 24 but also the existing LAN 2, so that the failure detection function is reliably exhibited. And reliability can be improved. Further, it is possible to specify the failure of the means used for failure detection. Further, the reliability of the failure detection function can be improved while suppressing an increase in cost.

Although the above-described processing in FIGS. 2 and 3 may operate only one of them, the reliability can be further improved by performing the processing simultaneously and in parallel as in the present embodiment. it can.

When detecting a failure of the PC servers 10 and 20 via the LAN 2 described with reference to FIG.
As is clear from the above description, the PC 30 to which a communication trial is performed via the PC 2 is used only as a communication trial partner, and has nothing to do with the presence or absence of the failure detection function. Therefore,
The PC server 10, 2 is provided with a failure detection function in the PC 30.
There is no problem even if it is equal to 0. However, in this case, the PC 30 is also equipped with a failure detection dedicated board and is connected to the configuration control device 4.

Embodiment 2 By the way, in the first embodiment, two PC servers are connected to the LAN 2 as failure detection target computers, and each has a function as a monitoring computer that detects and recovers failure of another PC server. . That is, all the PC servers 10 and 20 have the same function. In the following embodiments,
Examples of applications such as the number of connected units and allocation of functions will be described.

For example, as shown in FIG. 4, a plurality of PC servers 42-1, 42-2 as monitoring computers for detecting and recovering failures of the PC server 40 with respect to one PC server 40 as a failure detection target computer. , 42-n are provided as a group of monitoring computers. That is, the PC in this case
The server 40 detects that a failure has occurred and is not connected to another PC.
No function to detect server failure is required. Meanwhile, PC
The server 42 is a computer having no problem even if it breaks down. With such a configuration, even if one of the PC servers 42 fails, the failed PC server 40 can be reliably detected and recovered. However, in order to restart the failed PC server 40, only one monitoring computer needs to operate, and this is done by assigning priorities among the PC servers 42 to prevent redundant operation. Need to be Or, the PC server 4 in the operation status table
Only the operating state corresponding to 0 that has been changed from “normal” to “under failure” may perform the restart processing.

Embodiment 3 As shown in FIG. 5, one PC server 52 is provided as a dedicated monitoring computer, and another PC server 50- provided as a failure detection target computer is provided.
, 50-n may be collectively detected and recovered. In this case, the configuration control device provided as the operating state holding means can be built in the PC server 52 without independently providing the configuration control device.

Embodiment 4 As shown in FIG. 6, a plurality of PC servers 60-1, 60 are used as the failure detection target computers.
, 60-n and a plurality of PC servers 62-1, 62-2,..., 62-n as monitoring computers.

[0043]

According to the present invention, a fault can be detected effectively using not only the fault detecting means but also the existing network, so that the reliability of the fault detecting function can be improved while suppressing an increase in cost. It becomes possible. In other words, by having two means for detecting a failure, it is possible not only to reliably detect a failure even if a failure occurs in the means used for detecting the failure, but also to confirm that the failure is the failure of the means itself. Can also be recognized. Further, since there are two paths for restarting the failure detection target computer, the failure detection target computer in which the failure has occurred can be surely rebooted and recovered.

Further, since the operating state holding means is provided, the operating state of the failure detection target computer can be grasped. Therefore, the computer for which the restart processing is to be executed is specified, or the restart processing is repeatedly executed. Can be prevented.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a network system according to a first embodiment of a computer failure detection / recovery method according to the present invention.

FIG. 2 is a block diagram of a first embodiment in which another PC server failure is detected by a failure detection dedicated board;
It is the flowchart which showed the operation | movement which restarts a C server.

FIG. 3 is a diagram showing a P via a LAN 2 in the first embodiment.
9 is a flowchart illustrating an operation of detecting a failure of the C server and restarting the failed PC server.

FIG. 4 is an overall configuration diagram of a network system according to a second embodiment of the computer failure detection / recovery method according to the present invention.

FIG. 5 is an overall configuration diagram of a network system according to a third embodiment of the computer failure detection / recovery method according to the present invention.

FIG. 6 is an overall configuration diagram of a network system according to a fourth embodiment of the computer failure detection / recovery method according to the present invention.

FIG. 7 is a diagram showing a conventional PC server failure detection method.

[Explanation of symbols]

2 LAN, 4 configuration control device, 10, 20, 40, 4
2, 50, 52, 60, 62 PC server, 12, 2
2, 32 LAN board, 14, 24 Failure detection dedicated board, 15, 25 Dedicated cable, 16, 26 Operating status monitor, 30 PC.

Claims (5)

(57) [Claims] A plurality of computers among at least three computers connected to a network are set as failure detection target computers,
In a system, at least one of which is a monitoring computer that detects a failure that has occurred in the failure detection target computer, the failure detection target computer is operable independently, and detects a failure of the mounted failure detection target computer. Self-failure detection means, a network communication means for communicating with the other computers via a network, and a restart means for restarting itself according to an instruction.The monitoring computer operates independently A failure detection unit that is capable of detecting a failure of the other failure detection target computer connected via a dedicated line ; a network communication unit that communicates with another computer via a network; Operating state monitoring control means for monitoring the occurrence of a failure and performing a restart process of the failure detection target computer in which the failure is detected The operating state monitoring and control unit that detects a failure via the failure detection unit attempts communication with the failure detection target computer where the failure is detected via a network, and detects the failure where the failure is detected. If there is no response from the detection target computer, a restart instruction of the failure detection target computer in which a failure has been detected via the failure detection means is performed via a dedicated line, and if there is a response, the failure is detected. Determining that the self-failure detection means mounted on the failure detection target computer has a failure, and instructing a restart of the failure detection target computer in which the failure has been detected via a network. Recovery method. 2. The failure detection / recovery method for a computer according to claim 1, wherein said failure detection target computer is also said monitoring computer. 3. A system in which a plurality of computers out of at least three computers connected to a network are used as failure detection target computers, wherein the failure detection target computers are independently operable, and the installed failure detection Failure detection means for detecting a failure of the target computer and a failure of the other failure detection target computer connected via a dedicated line; network communication means for communicating with the other computer via a network; Restart means for restarting the system, and operating state monitoring control means for monitoring the occurrence of a failure in the system and performing a restart process of the failure detection target computer in which the failure has occurred, and via the network The operating state monitoring control means of the failure detection target computer that has detected a failure in The attempts to communicate with computers, if there is a response from the computer, perform restart indication of the failure detection unit the failure detection target computer failure is detected via the via dedicated lines, is the response If not, a failure detection / recovery method for a computer is characterized in that it is determined that the network communication means mounted therein has failed and the self-restart is performed. 4. The failure detection / recovery method for a computer according to claim 1, further comprising an operation state holding unit that holds an operation state of the failure detection target computer. 5. The operation state monitoring and control means mounted on the failure detection target computer which has detected a failure performs execution control of a restart instruction in accordance with an operation state held by the operation state holding means. The computer failure detection and recovery method according to claim 4.