JP2016057658A - Fault information management system and fault information management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a fault information management system in which, when a fault occurs, the factor that caused a fault and the situation of a function call are checked and determination is made as to the possibility of the same fault or not, and a fault information file is not written out in only the case of a fault attributable to the same cause.SOLUTION: The present invention is a fault information management system for managing an information file that includes fault information automatically generated at the time of process abnormality in a computer system, wherein the fault information management system includes dump request means for requesting that an information file that includes fault information be generated at the time of process abnormality, a system dump collection unit for generating the information file that includes fault information at request from the dump request means, a storage unit for storing the information file that includes fault information generated by the system dump collection unit, and an external storage device for storing a previous information file that includes previous fault information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a failure information management system and a failure information management method, and more particularly to a failure information management system and a failure information management method when the same failure occurs.

  In general, when a failure occurs in a node on a computer network, the node's memory image (context, stack, data) is output as failure information to a dump file, and the cause of the failure is analyzed by analyzing the dump file. The method is taken. A dump file is created in a dump area, which may be shared by one or more nodes.

  Normally, after a failure occurs, the operation is to restart the node. Therefore, when a dump area is shared while multiple nodes share the dump area, the failure-dump collection -Re-startup-Possible failure is repeated and the same dump file may be collected repeatedly.

  In addition, when a job operation is performed, when a job is restarted after a failure, the job whose execution is interrupted due to the failure is often re-executed. In such a case, if a failure occurs due to the job, there is a possibility that failure-dump collection-restart-job re-execution-failure may be repeated, and the same dump file may be collected repeatedly. is there.

  In addition, distributed parallel programs such as MPI (Message Passing Interface) are often synchronized and the same processing is executed at each node. If the dump area is shared, the same dump file may be collected from multiple nodes.

  In such a case, when the remaining capacity of the dump area is low, if another failure occurs in another node, there is a possibility that necessary information cannot be collected due to insufficient dump capacity. Furthermore, there is a problem that it takes time to analyze a plurality of dump files.

  As a technique related to the above, Patent Document 1 discloses that the number of information files stored in an external storage device does not reach a preset upper limit, the same information file does not already exist in the external storage device, and information When the file is a complete file that can be analyzed, a technique is disclosed in which the generated information file is stored in an external storage device, and the generated information file is deleted in other cases.

  Further, in Patent Document 2, it is determined whether or not failure information satisfying a predetermined similarity condition is stored for failure information received from a node, and depending on the determination result, depending on the process, Output enable / disable information indicating whether or not failure analysis information including the contents of the used main storage device is output to the node, and the failure information satisfying a predetermined similarity condition with respect to the received failure information is stored in the failure information storage unit. A technique for registering received failure information in a failure information storage unit when it is determined that it is not stored is disclosed.

Japanese Patent Laid-Open No. 11-024966 JP 2014-096031 A

  However, in Patent Document 1, when the same process is repeatedly re-executed and there is a repeated failure, it is not determined whether or not the failure location that was in the repeated failure is always the same as the failure that occurred before. Rather, as a mere prediction, if the same process is repeatedly injured, it is judged as the same problem. There is a possibility that a different problem occurs between the previous failure and the next failure. In such a case, however, there is a problem that the failure information file may not be written due to the number limitation.

  Further, Patent Document 2 assumes that inter-node communication is assumed, and does not write out the same kind of failure analysis file in the same distributed parallel program using inter-node communication, and shares dump areas in a plurality of systems. There was a problem that the same kind of failure analysis file was not written out.

  The object of the present invention is to solve this problem. When a failure occurs, the failure information management system determines whether the failure is the same and prevents the failure information file from being written in the case of the failure of the same cause. Is to provide.

  In the present invention, in order to solve the above problems, in a failure information management system for managing an information file including failure information automatically generated when a process of the computer system is abnormal, an information file including failure information is generated when the process is abnormal A dump request means for requesting to perform, a system dump collection section for generating an information file including failure information in response to a request from the dump request section, and a storage section for storing an information file including the failure information generated by the system dump collection section And an external storage device for storing a previous information file including previous failure information. At least one external storage device is provided in the computer system, and from each of the computers constituting the computer system, when a process error occurs Stores fault information that is automatically generated and Sampling unit is characterized by having a dump file output determining means for determining whether a match by comparing the contents of the failure information and the previous fault information.

  According to the present invention, in a failure information management method for a failure information management system that manages an information file including failure information that is automatically generated when a process in the computer system is abnormal, an information file that includes failure information is generated when the process is abnormal. A step of requesting, a step of generating an information file including failure information in response to a request from the dump request means, a step of storing an information file including failure information generated by the system dump collecting unit, and previous failure information Including at least one external storage device included in the computer system, and from each of the computers constituting the computer system, fault information automatically generated in the event of a process abnormality is stored. System dump collection with steps to memorize It is characterized by having a step of determining whether a match by comparing the contents of the failure information and the previous fault information.

  According to the present invention, when a failure occurs, it is determined whether or not the failure is the same by looking at the cause of the failure and the function call status, so that the failure information file is not written in the case of the failure of the same cause. A failure information management system can be provided.

It is a block diagram which shows the structure of the failure information management system in the 1st Embodiment of this invention. It is a figure which shows one example of the content of the failure information in the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the failure information management system in the 1st Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(Embodiment)
The configuration of the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of a failure information management system according to an embodiment of the present invention.

  In FIG. 1, the failure information management system 10 includes a node 1000, a dump request unit 100, a system dump collection unit 200, a main storage unit 300, and a dump area unit 2000 in the external storage device 20. . The system dump collection unit 200 includes a dump file output determination unit 210, a dump file output unit 220, and a failure traceback information collection unit 230. The main storage unit 300 stores failure information when a failure occurs as failure traceback information 310 in response to a request from the failure traceback information collection unit 230. The dump area unit 2000 includes a plurality of dump files 400-1,... 400-n as information files, and each dump file includes dump failure traceback information 410-1,. n is stored. Even when there are a plurality of nodes, dump files are collectively stored in the dump area unit 2000 of the external storage device 20.

  FIG. 2 is a diagram showing details of the failure traceback information 310. The failure traceback information 310 includes kernel information, the cause of the failure, the name of the failed command, and the function call status (traceback information) up to the failure. The traceback information represents a history of instructions up to the occurrence of a failure.

  For example, from the top to the third line of FIG. 2, in the kernel information, Kninfo indicates where the kernel was created, and Cause indicates what causes the failure. comm is a comment field.

  Next, from the fourth line to the bottom is the function call status (traceback information), which displays the function address and the called function. The upper line shows the new call. If the cause of the failure is the same as the function call status (traceback information), it can be estimated that the failure is the same. The failure traceback information 310 is written as dump failure traceback information 410-1 in the dump file 400-1.

  The operation of the embodiment of the present invention will be described in detail with reference to FIGS.

  FIG. 3 is a flowchart showing the operation of the embodiment of the present invention.

  First, when a failure occurs in the node 1000 (S301), the dump request unit 100 notifies the dump file output determination unit 210 of the system dump collection unit 200 of the occurrence of the failure (S302). The dump file output determination unit 210 notifies the failure traceback information collection unit 230 and requests to create the failure traceback information 310 (S303). The failure traceback information collecting unit 230 creates the failure traceback information 310 from the failure information in the main storage unit 300 (S304).

  The dump file output determination unit 210 searches the dump file 400-1 in the dump area 2000 and reads the dump failure traceback information 410-1 in the dump file 400-1 (S305). The dump file output determination unit 210 compares the failure traceback information 310 and the dump failure traceback information 410-1, and investigates whether there is information having the same cause of failure and the same traceback information as the failure traceback information 310. (S306).

  When there is no information having the same traceback information (No in S307), the dump file output determination unit 210 searches for the next dump file 400-2 in the dump area unit 2000 (Yes in S308), and dumps in the same manner. The fault traceback information 410-2 is read, and the fault traceback information 310 and the dump fault traceback information 410-2 are compared (S305 to S307).

  All the dump files 400-1 to 400-n in the dump area unit 2000 are searched, and the dump failure traceback information 410-1 to 410-n of all the dump files 400-1 to 400-n is stored in the failure traceback. If the information 310 does not match (No in S307, No in S308), the dump file output unit 220 adds the dump file 400- (n + 1) including the dump failure traceback information 410- (n + 1) to the dump area unit 2000. It outputs (S309).

  Conversely, if there is information having the same traceback information (Yes in S307), the process ends without outputting a new dump file (S310).

  As described above, according to the embodiment of the present invention, when a failure occurs, it is determined whether the failure is the same by looking at the cause of the failure and the function call status. It is possible to provide a failure information management system in which a failure information file is not written only in

  According to the embodiment of the present invention, it is not necessary to save a duplicate dump file, so that the disk capacity of the dump area can be reduced. Further, since duplicate dump files are not stored, there is an effect that the efficiency of analysis work is improved without analyzing duplicate dump files.

  The present invention is not limited to the above-described embodiment, and can be implemented with various changes and modifications without departing from the gist of the present invention.

  The present invention is applicable to a failure information management system in which a dump file is generated when a failure occurs.

DESCRIPTION OF SYMBOLS 10 Failure information management system 20 External storage device 100 Dump request means 200 System dump collection part 210 Dump file output determination means 220 Dump file output part 230 Failure trace back information collection part 300 Main storage part 310 Failure trace back information 400-1 to 400 -(N + 1) Dump file 410-1 to 410- (n + 1) Dump failure traceback information 1000 Node (computer)
1001 Node (computer)
2000 Dump area

Claims (6)

In a failure information management system for managing an information file including failure information automatically generated when a process of a computer system is abnormal,
Dump request means for requesting to generate the information file including the failure information when the process is abnormal;
A system dump collection unit that generates the information file including the failure information in response to a request from the dump request unit;
A storage unit for storing the information file including the failure information generated by the system dump collection unit;
An external storage device for storing a previous information file containing previous failure information;
Have
The external storage device is provided in at least one computer system and stores the failure information automatically generated from each computer constituting the computer system when a process abnormality occurs,
The system dump collection unit includes a dump file output determination unit that determines whether or not the failure information and the previous failure information match by comparing the contents of the failure information and the previous failure information.   2. The failure according to claim 1, wherein the system dump collection unit stores the information file including the failure information in the external storage device when the determination result of the dump file output determination unit does not match. Information management system.   The failure information management system according to claim 1, wherein the failure information includes traceback information representing a history of instructions up to the occurrence of a failure.   4. The failure information management according to claim 1, wherein the external storage device stores the failure information that is automatically generated when a process abnormality occurs in the computer system without duplication. 5. system. In a failure information management method of a failure information management system for managing an information file including failure information automatically generated when a process of a computer system is abnormal,
Requesting to generate the information file including the failure information when the process is abnormal;
Generating the information file including the failure information in response to a request from a dump request unit;
Storing the information file including the failure information generated by a system dump collection unit;
Storing a previous information file containing previous failure information;
Have
An external storage device is provided in at least one of the computer systems, and has a step of storing the failure information automatically generated at the time of a process abnormality from each computer constituting the computer system,
The system dump collection unit includes a step of comparing the failure information with the content of the previous failure information and determining whether or not they match each other.   7. The failure information management method according to claim 6, wherein the system dump collection unit includes a step of storing the information file including the failure information when the determination result of the dump file output determination unit does not match. .

Images