CN116094906A - Fault root cause location method and device - Google Patents

Abstract

The application discloses a fault root cause positioning method and device, and belongs to the technical field of operation and maintenance. The method is applied to a management node for managing a service cluster, wherein the service cluster comprises a plurality of service nodes, and the service nodes are used for realizing user services. The method comprises the following steps: acquiring networking information of a plurality of service nodes; when a service cluster fails, acquiring link states of links among a plurality of service nodes; and carrying out convergence analysis based on the link state and networking information, and determining the root cause of the fault. The method and the device can rapidly locate the fault root cause, and improve the efficiency of locating the fault root cause.

Description

Fault root cause location method and device

technical field

The present application relates to the technical field of operation and maintenance, in particular to a method and device for locating the root cause of a fault.

Background technique

The service cluster is used to implement the user's business and provide services for the user. A service cluster includes multiple service nodes. Moreover, the scale of the service cluster is usually large, and different service nodes may span multiple service nodes. During the operation and maintenance of the live network, if a certain device in the link fails, it will take a lot of time and manpower to locate the root cause of the failure. Moreover, if it takes too long to locate the root cause of the fault, it may also lead to an expansion of the customer's damage surface.

At present, when a link between service nodes fails, the management node will send out an alarm message, and indicate the source service node and the destination service node in the link in the alarm message. Then, the operation and maintenance personnel check the network devices connected to the source service node and the destination service node one by one to determine the root cause of the failure.

However, the efficiency of locating the root cause of the fault in this way of locating the root cause of the fault is very low.

Contents of the invention

The present application provides a fault root cause location method and device. The present application improves the efficiency of locating the root cause of a fault, and reduces the probability of service being affected by a fault. The technical scheme that this application provides is as follows:

In a first aspect, the present application provides a method for locating the root cause of a fault. The method is applied to a management node that manages a service cluster. The service cluster includes multiple service nodes, and the service nodes are used to implement user services. The method includes: obtaining networking information of multiple service nodes; obtaining link states of links between multiple service nodes when a service cluster fails; performing aggregation analysis based on link states and networking information to determine the root cause because.

In the method for locating the root cause of a fault provided in this application, through aggregation and analysis based on link status and networking information, the root cause of the fault can be automatically located globally, and the root cause of the fault can be quickly located, which improves the accuracy of the fault diagnosis. The efficiency of locating the root cause reduces the probability of service impact due to faults.

In an implementation manner, multiple service nodes access the network through an access network device located at the access layer. Based on the link status and networking information, aggregate analysis is performed to determine the root cause of the failure, including: when the link status of the link between the target service node and other service nodes cannot be obtained, or the link status indication from other service nodes When the link with the target service node is broken, determine the target service node as the candidate root cause of the failure; based on the networking information, obtain the first access network device connected to the target service node and the first service node connected to the first access network device, The first service node is a service node other than the target service node connected to the first access network device; when the link status indicates that the first service node includes a normal node, it is determined that the target service node is the root cause of the fault.

In addition, the access network device accesses the network through the convergence network device located at the convergence layer. Aggregate analysis based on the link state and networking information to determine the root cause of the fault also includes: when the link state indicates that the first service node is a candidate root cause of the fault, based on the networking information, obtain the first access network device The first convergence network device connected, and the second service node connected to other access network devices connected to the first convergence network device, the second service node is a service node other than the target service node connected to the first convergence network device; when When the link state indicates that the second serving node includes a normal node, it is determined that the first access network device is the root cause of the failure.

Further, the aggregation network equipment accesses the network through the core network equipment located at the core layer. Carrying out aggregation analysis based on the link status and networking information to determine the root cause of the failure, also includes: when the link status indicates that the second service node is a candidate for the root cause of the failure, based on the networking information, obtaining the connection of the first aggregation network device The first core network device connected to the first core network device, and the third service node connected to other aggregation network devices connected to the first core network device, the third service node is a service node other than the target service node connected to the first core network device; when the link When the state indicates that the third serving node includes a normal node, it is determined that the first convergence network device is the root cause of the fault.

Otherwise, performing aggregation analysis based on the link state and networking information to determine the root cause of the fault, further includes: when the link state indicates that the third service node is a candidate root cause of the fault, determining the first core network device as the root cause of the fault.

Optionally, the service node may detect the link status of the link between the service nodes connected to it, and send the link status to the management node. Obtaining link states of links between multiple service nodes includes: receiving link states of links between the service node and other service nodes provided by each service node.

In one implementation, before the management node obtains the link status of the links between multiple service nodes, the management node can also first select a representative service node in the service cluster, so that the service nodes in the service cluster can obtain The link state of the link between itself and each representative service node. Then the method may further include: the management node determines a plurality of service nodes to be tested among the plurality of service nodes, and provides information of a plurality of service nodes to be tested to each service node, so that each service node obtains the service node and each The link state of the link between the service nodes to be tested. Correspondingly, the management node receives the link status of the link between the service node and other service nodes provided by each service node, including: the management node receives the link status between the service node and each service node to be tested provided by each service node The link state of the road.

Select the service node to be tested in the service cluster through the management node, so that the service node in the service cluster obtains the link status of the link between itself and each service node to be tested, so that the service node does not need to obtain the link status between the service node and the service cluster The link state of the link between each other service node can reduce the consumption caused by obtaining the link state, reduce the probability of a network storm in the service cluster, and reduce the impact of obtaining the link state on the performance of the service cluster. This effect is especially evident when the service cluster is large in scale.

Optionally, the overall network range of all service nodes to be tested in the service cluster may cover the network range of the service cluster. Wherein, the network range refers to the transmission range that can be reached through the network. In this way, the overall network scope of all service nodes to be tested covers all cabinets and all network segments of the network used by the service cluster, which ensures the comprehensiveness of the obtained link information and the accurate determination of the root cause of the fault.

In an implementation manner, the link state is reflected by one or more of the following: link connectivity state and transmission delay. The connection status of the link is used to indicate whether the link is connected or disconnected. When the connection status of the link indicates that the link is disconnected, the service nodes cannot use the link to transmit data. The transmission delay can reflect the state of the link. When the transmission delay is too large (such as exceeding the expected transmission delay threshold), the link may be broken, or the link may be connected but in a poor state. At this time, if the service node uses this link to transmit data, the service time limit of the service node cannot be satisfied, and the service cluster will also fail. Therefore, the link status can be reflected through the transmission delay of the link.

In a second aspect, the present application provides a device for locating the root cause of a fault. The device is applied to a management node for managing a service cluster. The service cluster includes a plurality of service nodes, and the service nodes are used to implement user services. The device includes: an acquisition module, used to acquire networking information of multiple service nodes; an acquisition module, also used to acquire link states of links between multiple service nodes when the service cluster fails; a processing module, used Based on the aggregation analysis based on the link status and networking information, determine the root cause of the fault.

Optionally, multiple service nodes access the network through an access network device located at the access layer. Then the processing module is specifically used for: when the link status of the link between the target service node and other service nodes cannot be obtained, or when the link status indication from other service nodes is disconnected from the target service node, determine the target service node Is the candidate root cause of the failure; based on the networking information, obtain the first access network device connected to the target service node, and the first service node connected to the first access network device, the first service node is the first access network device connected to service nodes other than the target service node; when the link state indicates that the first service node includes a normal node, determine that the target service node is the root cause of the fault.

Optionally, the access network device accesses the network through the convergence network device located at the convergence layer. Then the processing module is specifically used for: when the link state indicates that the first service node is a candidate root cause of failure, based on the networking information, obtain the first convergence network device connected to the first access network device, and the first convergence network device The second service node connected to other access network devices connected to the device, the second service node is a service node other than the target service node connected to the first convergence network device; when the link status indicates that the second service node includes a normal node, Determine that the first access network device is the root cause of the fault.

Optionally, the aggregation network device accesses the network through the core network device at the core layer. The processing module is specifically used for: when the link state indicates that the second service node is a candidate root cause of failure, based on the networking information, obtain the first core network device connected to the first convergence network device, and the first core network device The third service node connected to other converging network devices connected, the third service node is the service node other than the target service node connected to the first core network device; when the link status indicates that the third service node includes a normal node, determine the third service node A converged network device is the root cause of the failure.

Or, when the link state indicates that all the third serving nodes are candidate root causes of the failure, determine that the first core network device is the root cause of the failure.

Optionally, the obtaining module is specifically configured to: receive link statuses of links between the service node and other service nodes provided by each service node.

Optionally, the acquisition module is specifically used to: determine a plurality of service nodes to be tested among the plurality of service nodes; provide each service node with information on a plurality of service nodes to be tested, so that each service node obtains the The link state of the link between the service nodes to be tested; receiving the link state of the link between the service node and each service node to be tested provided by each service node.

Optionally, the network ranges of the multiple service nodes to be tested cover the network range of the service cluster.

Optionally, the link state is reflected by one or more of the following: link connectivity state and transmission delay.

In a third aspect, the present application provides a computing device, including a memory and a processor, the memory stores program instructions, and the processor executes the program instructions to perform the method provided in the first aspect of the present application and any possible implementation thereof .

In a fourth aspect, the present application provides a computer cluster, including at least one computing device, each computing device includes a processor and a memory, and the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so as to The computing device cluster is made to execute the method provided in the first aspect of the present application and any possible implementation manner thereof.

In a fifth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium is a non-volatile computer-readable storage medium, and the computer-readable storage medium includes program instructions, when the program instructions are stored on the computing device During operation, the computing device is made to execute the method provided in the first aspect of the present application and any possible implementation manner thereof.

In a sixth aspect, the present application provides a computer program product containing instructions. When the computer program product runs on a computer, the computer executes the method provided in the first aspect of the present application and any possible implementation thereof.

Description of drawings

FIG. 1 is a schematic diagram of an implementation environment involved in a fault root cause location method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of an implementation environment involved in another fault root cause location method provided by an embodiment of the present application;

Fig. 3 is a process schematic diagram of a method for locating the root cause of a fault provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an implementation environment involved in yet another fault root cause location method provided by an embodiment of the present application;

Fig. 5 is a flow chart of another fault root cause location method provided by the embodiment of the present application;

FIG. 6 is a flow chart of a management node performing aggregation analysis based on link status and networking information provided by an embodiment of the present application to determine the root cause of a failure;

FIG. 7 is a visual schematic diagram of a management node performing aggregation analysis based on link status and networking information provided by an embodiment of the present application;

FIG. 8 is a visual schematic diagram of another management node performing aggregation analysis based on link status and networking information provided by an embodiment of the present application;

FIG. 9 is a visual schematic diagram of yet another management node performing aggregation analysis based on link status and networking information provided by an embodiment of the present application;

FIG. 10 is a visual schematic diagram of yet another management node performing aggregation analysis based on link status and networking information provided by the embodiment of the present application;

Fig. 11 is a schematic diagram of a device for locating the root cause of a fault provided by an embodiment of the present application;

Fig. 12 is a schematic structural diagram of a computing device provided by an embodiment of the present application;

Fig. 13 is a schematic structural diagram of a computing device cluster provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

For ease of understanding, the technologies and backgrounds involved in the embodiments of the present application are firstly explained below.

The service cluster is used to implement the user's business and provide services for the user. A service cluster includes multiple service nodes. Service clusters can be deployed in a distributed manner. For example, the service cluster may be a distributed storage cluster. Distributed storage clusters are used to provide distributed storage services. The distributed storage service is provided by one or more storage pools, and each storage pool is composed of one or more storage servers. The storage pool abstracts the hard disks on these storage servers into a unified storage space for users to use. Generally, a distributed storage cluster has a management platform, and the management platform is used to manage the distributed storage cluster. Each storage server is equipped with a client through which it communicates with the management platform. Moreover, in the embodiment of the present application, the service cluster may be a cloud service cluster.

The size of the service cluster is usually very large, and different service nodes may span multiple service nodes. During the operation and maintenance process, if a device in the link of the service cluster fails, it will take a lot of time and manpower to locate the root cause of the failure. Moreover, if it takes too long to locate the root cause of the fault, it may also lead to an expansion of the customer's damage surface.

At present, when a link between service nodes fails, the management node will send out an alarm message, and indicate the source service node and the destination service node in the link in the alarm message. Then, the operation and maintenance personnel check the network devices connected to the source service node and the destination service node one by one to determine the root cause of the failure. For example, according to the network topology of the service cluster, according to the Internet protocol (internet protocol, IP) addresses of the source service node and the destination service node, check the switches and other devices connected to the source server and the destination server to determine the root cause of the failure. Wherein, the network topology refers to a specific physical (ie, real) or logical (ie, virtual) network arrangement among members constituting the network.

However, the efficiency of locating the root cause of the fault in this way of locating the root cause of the fault is very low.

The embodiment of the present application provides a method for locating the root cause of a fault. The method can be applied to the management node that manages the service cluster. A service cluster includes multiple service nodes. Service nodes are used to implement user services. The method includes: obtaining networking information of multiple service nodes; obtaining link states of links between multiple service nodes when a service cluster fails; performing aggregation analysis based on link states and networking information to determine the root cause because.

In this fault root cause location method, through aggregation and analysis based on link status and networking information, the fault root cause location can be automatically performed on a global scale, and the root cause of the fault can be quickly located, which improves the root cause of the fault. The efficiency of positioning reduces the probability of service impact due to faults.

FIG. 1 is a schematic diagram of an implementation environment involved in a fault root cause location method provided in an embodiment of the present application. As shown in FIG. 1 , the implementation environment includes: a management node 10 and a service cluster. The service cluster includes multiple service nodes 20 . The service node 20 is used to implement user services. The management node 10 is used to locate the root cause of the fault of the service cluster by using the method for locating the root cause of the fault provided in the embodiment of the present application. For example, the management node 10 is used to obtain the networking information of the service nodes 20, and, when the service cluster fails, to obtain the link status of the links between multiple service nodes 20, and based on the link status and networking information to perform Convergence analysis to determine the root cause of the failure.

The management node 10 can establish a communication connection with the service nodes 20 in the service cluster. For example, a communication connection may be established between the management node 10 and the service node 20 through a network. Optionally, the network may be a local area network, the Internet, or other networks, which are not limited in this embodiment of the present application.

Optionally, the management node 10 may be a computing device with a computing function. In one manner, the management node 10 may be a server. And the server can be a server, or a server cluster composed of several servers, or a cloud computing service center. Among them, a large number of basic resources owned by the cloud service provider are deployed in the cloud computing service center. For example, computing resources, storage resources, and network resources are deployed in the cloud computing service center. The cloud computing service center can use the large amount of basic resources to implement the method for locating the root cause of the fault provided in the embodiment of the present application.

When the service node 20 is realized through the cloud computing service center, the function realized by the fault root cause location method provided by the embodiment of the present application can be abstracted into a kind of fault root cause location cloud service by the cloud service provider on the cloud platform, and the cloud platform can use The resources in the cloud computing center provide users with cloud services for locating the root cause of the fault. After the user purchases the fault root cause location cloud service on the cloud platform, the user can use the fault root cause location cloud service to locate the root cause of the fault for the user's service cluster.

Alternatively, when the service node 20 is implemented by a cloud computing service center, the functions implemented by the fault root cause location method provided in the embodiment of the present application may also be provided as additional services of other cloud services. For example, when the service cluster is a distributed storage cluster, and the distributed storage cluster is used to provide storage cloud services, the fault root cause location cloud service provided in this embodiment of the application can be provided as an additional service of the storage cloud service, that is, the fault root The cause location method is used to locate the root cause of the failure of the storage cloud service, so as to ensure the service quality of the storage cloud service and improve user experience.

Optionally, the cloud platform may be a cloud platform of a central cloud, a cloud platform of an edge cloud, or a cloud platform including a central cloud and an edge cloud, which is not specifically limited in this embodiment of the present application. Moreover, in the implementation environment provided by the embodiment of the present application, the service node 20 may also be implemented by other resource platforms except the cloud platform, which is not specifically limited in the embodiment of the present application. At this time, the service node 20 can be realized by resources in other resource platforms, and provide related services to users.

In the embodiment of the present application, as shown in FIG. 2 , the management node 10 may belong to a service cluster, that is, the service cluster includes the management node 10 . At this time, the management node 10 can not only locate the fault root cause of the service cluster, but also manage the service nodes 20 in the service cluster. For example, the management node 10 is used to schedule tasks for the service node 20 and the like. The management node 10 records relevant information of all service nodes 20 in the service cluster.

Alternatively, as shown in FIG. 1 , the management node 10 may not belong to the service cluster, that is, the service cluster does not include the management node 10 . In an implementation manner, the management node 10 may be an operation and maintenance platform independent of the service cluster. For example, the operation and maintenance platform can manage the operation and maintenance of the entire data center, and the service cluster can be deployed in an availability zone (AZ) of the data center. At this time, the management node 10 may also perform root cause location on other clusters or systems. For example, when the function realized by the fault root cause location method provided in the embodiment of the present application is provided as an additional service of other cloud services, since the cloud platform can provide various cloud services, the fault root cause location method can be used for cloud platform All cloud services provided are used for fault root cause location.

Optionally, when performing aggregation analysis based on the link state and networking information to determine the root cause of the fault, the management node 10 may also visualize the process. As shown in Figure 3, after the management node 10 obtains the networking information of multiple service nodes 20 and the link status of the links between the multiple service nodes 20, when performing aggregation analysis, it can display on the display interface of the management node 10 The process of the aggregation analysis is displayed in real time for users to view.

In addition, the service nodes 20 in the service cluster are usually connected through the network. When the service cluster fails, the service node 20 may fail, or the link between the service nodes 20 may fail. Then, as shown in FIG. 4 , the implementation scenario may further include: an access switch 30 , an aggregation switch 40 and a core switch 50 . It should be noted that FIG. 4 is only an example of connection between service nodes 20, and the connection mode between service nodes 20 may change according to application requirements of different scenarios, which is not limited in this embodiment of the present application. Moreover, the manner of realizing the connection between the service nodes 20 may also change according to the application scenarios. For example, the network equipment between the service nodes 20 may include not only switches, but also routers or optical network terminations (optical network termination, ONT, also called optical modems). For another example, the access switch 30 can be converted into another type of access network device located at the access layer, the aggregation switch 40 can be converted into another type of aggregation network device located at the aggregation layer, and the core switch 50 can be converted into a network device located at the core layer. Other types of core network equipment.

It should be understood that the above content is an exemplary description of the application scenario of the fault root cause location method provided by the embodiment of the present application, and does not constitute a limitation on the application scenario of the fault root cause location method. Those of ordinary skill in the art know that, As business requirements change, its application scenarios can be adjusted according to the application requirements, and the embodiments of this application do not list them one by one.

The method for locating the root cause of a fault provided by the embodiment of the present application will be described below. As shown in Figure 5, the fault root cause location method includes the following steps:

Step 501, the management node acquires networking information of multiple service nodes.

The networking information of the service node is used to indicate the networking status and networking mode of the service node. In a practicable manner, the networking information of the service node may be represented by the network topology of the service node.

Step 502, the management node obtains the link status of the links between multiple service nodes when the service cluster fails.

The link state of the link between service nodes will affect the data transmission process between service nodes. When the link is normal, the service nodes can transmit data normally, and when the link is not normal, the data transmission process between the service nodes will be affected. In an implementation manner, the link state is reflected by one or more of the following: link connectivity state and transmission delay. The connection status of the link is used to indicate whether the link is connected or disconnected. When the connection status of the link indicates that the link is disconnected, the service nodes cannot use the link to transmit data. The transmission delay can reflect the state of the link. When the transmission delay is too large (such as exceeding the expected transmission delay threshold), the link may be broken, or the link may be connected but in a poor state. At this time, if the service node uses this link to transmit data, the service time limit of the service node cannot be satisfied, and the service cluster will also fail. Therefore, the link status can be reflected through the transmission delay of the link. Wherein, when the service node has multiple ports, the link status of the link between the service nodes is the link status of the link between the ports currently used by the service node. For example, a service node has a primary port and a standby port, and when the service node currently uses the primary port to implement services, the link state of the link between the service nodes is the link state of the link between the primary ports of the service nodes.

Optionally, the management node acquires link states of links between multiple service nodes, including: receiving link states of links between the service node and other service nodes provided by each service node. The service node can detect the link state of the link between the service nodes connected to it, and send the link state to the management node. For example, a service node may use a ping command to detect link statuses of links between it and other service nodes. Or, the service node may also detect the link status of the link between it and other service nodes in other ways, which is not specifically limited in this embodiment of the present application.

In one implementation, before the management node obtains the link status of the links between multiple service nodes, the management node can also first select a representative service node in the service cluster, so that the service nodes in the service cluster can obtain The link state of the link between itself and each representative service node. Then the method may further include: the management node determines a plurality of service nodes to be tested among the plurality of service nodes, and provides information of a plurality of service nodes to be tested to each service node, so that each service node obtains the service node and each The link state of the link between the service nodes to be tested. Correspondingly, the management node receives the link status of the link between the service node and other service nodes provided by each service node, including: the management node receives the link status between the service node and each service node to be tested provided by each service node The link state of the road. Wherein, in addition to providing the service node with the information of multiple service nodes to be tested, the management node can also send an instruction to the service node, which instructs the service node to obtain the link status information of the link between the service node and each service node to be tested. instruction. Alternatively, the management node may not send the instruction to the service node, and the instruction may be implemented in other ways. For example, the management node and the service node can agree in advance that when the management node provides the information of the service node to be tested to the service node, the service node needs to obtain the link status of the link between the service node and each service node to be tested, and report to the management The node sends the obtained link state.

In addition, there are many ways for the service node to provide the management node with the link status of the link between the service node and each service node to be tested. For example, after obtaining the link state of the link between the service node and each service node to be tested, the service node can output the link state to a log, and the management node can obtain the link state from the log. Alternatively, the implementation scenario of the method for locating the root cause of a fault provided in the embodiment of the present application may further include a collection node, which is used to obtain the link status from the log, and provide the link status to the management node. In addition, after the management node acquires the information indicating the link state, it may also perform operations such as data cleaning (such as format conversion) on the information, so that the management node can parse the information.

Optionally, the overall network range of all service nodes to be tested in the service cluster may cover the network range of the service cluster. Wherein, the network range refers to the transmission range that can be reached through the network. In this way, the overall network scope of all service nodes to be tested covers all cabinets and all network segments of the network used by the service cluster, which ensures the comprehensiveness of the obtained link information and the accurate determination of the root cause of the fault.

When the scale of the service cluster is large, if each service node obtains the link status for each other service node in the service cluster, this will cause a network storm in the service cluster and affect the performance of the service cluster. Select the service node to be tested in the service cluster through the management node, so that the service node in the service cluster obtains the link status of the link between itself and each service node to be tested, so that the service node does not need to obtain the link status between the service node and the service cluster The link state of the link between each other service node can reduce the consumption caused by obtaining the link state, reduce the probability of a network storm in the service cluster, and reduce the impact of obtaining the link state on the performance of the service cluster. This effect is especially evident when the service cluster is large in scale.

Step 503, the management node performs aggregation analysis based on the link state and networking information, and determines the root cause of the fault.

In the network, the service nodes in the service cluster access the network through the access network device at the access layer, the access network device accesses the network through the aggregation network device at the aggregation layer, and the aggregation network device accesses the network through the core Network devices are connected to the network. When determining the root cause of the failure, aggregate analysis can be performed on the network devices that provide network connections for service nodes. In an implementation manner, as shown in FIG. 6, performing aggregation analysis based on link status and networking information to determine the root cause of the fault may include:

Step 5031, when the link status of the link between the target service node and other service nodes cannot be obtained, or the link status from other service nodes indicates that the link with the target service node is disconnected, the management node determines that the target service node is a candidate failure Root cause.

Failure to obtain the link state of the link between the target service node and other service nodes means that the management node has not received information indicating the link state of the link between the target service node and other service nodes. At this time, it may be because the target service node is disconnected from the network, or the target service node cannot collect link information with other service nodes. Either of these two cases indicates that the target service node is unable to perform network transmission. However, the failure of the target service node to perform network transmission may be due to a problem with the target service node itself, or a problem with the network connected to the target service node. In this case, you can first mark the target service node as a candidate root cause of failure to further determine the target Whether the service node is the root cause of the failure.

Step 5032: Based on the networking information, the management node obtains the first access network device connected to the target service node and the first service node connected to the first access network device. The first service node is the first access network device connected to the first service node. A service node other than the target service node.

After determining that the target service node is the candidate root cause of the failure, it can be judged by referring to the status of the network device providing the network to the target service node, so as to further determine whether the target service node is the root cause of the failure. Since the service nodes in the service cluster access the network through the access network device located at the access layer, the target service node can be determined according to the link status of the service node connected to the first access network device connected to the target service node. Whether the service node is the root cause of the failure. Before this, the first access network device connected to the target service node and the first service node connected to the first access network device may be obtained first. In an implementation manner, the first access network device connected to the target service node and the first service node connected to the first access network device may be determined according to networking information of service nodes in the service cluster.

Step 5033: When the link state indicates that the first service node includes a normal node, the management node determines that the target service node is the root cause of the failure.

After determining the first access network device connected to the target service node and the first service node connected to the first access network device, the link states of all first service nodes connected to the first access network device may be obtained, and Whether the link of the first serving node is normal is determined according to the link state of each first serving node. When the link of the first service node is normal, it means that the first service node can perform data transmission through the link, and the first service node is a normal node. Moreover, if the first access network device is the root cause of the fault, all first service nodes connected to the first access network device cannot work normally. Therefore, when the link state indicates that there are normal nodes among all the first service nodes connected to the first access network device, it can be determined that the first access network device is in a normal state, and then it can be determined that the target service node itself has a problem, that is, Determine the target service node as the root cause of the failure. At this time, the failure of the service cluster can be called a failure of the service node.

Optionally, when performing aggregation analysis based on link status and networking information to determine the root cause of a fault, the management node can also visualize the process. In one implementation, management nodes can be visualized at cluster granularity. For example, as shown in Figure 7, in step 5033, the current candidate fault root cause is the target service node, then the management node can at least display the target service node and the access network device connected to the target service node according to the topology diagram (Fig. 7 is the access switch 30) and all first service nodes connected through the access network device. In Fig. 7, the black-filled dots are the service nodes determined as the candidate root cause of the fault, and the white-filled dots are the service nodes not determined as the candidate root cause of the fault. It can be seen from FIG. 7 that if there are normal nodes among all the first service nodes connected to the first access network device, it can be determined that the target service node is the root cause of the failure.

Step 5034: When the link status indicates that the first service node is a candidate root cause of the failure, the management node obtains the first aggregation network device connected to the first access network device and the first aggregation network device connected to the first aggregation network device based on the networking information. A second service node connected to other access network devices, where the second service node is a service node other than the target service node connected to the first aggregation network device.

Since the probability that all service nodes connected to the same access network device fail at the same time is low, when the link status indicates that all the first service nodes connected to the first access network device are candidate root causes of failure, it can be determined that It is the first access network device or a network device providing a network to the first access network device that fails. Then the management node can obtain the first convergence network device connected to the first access network device and the second service node connected to other access network devices connected to the first convergence network device based on the networking information, so as to determine the fault based on them Root cause.

Step 5035: When the link state indicates that the second serving node includes a normal node, the management node determines that the first access network device is the root cause of the failure.

After determining the first convergent network device connected to the first access network device and all the second service nodes connected to the first converged network device, the link status of all the second service nodes connected to the first converged network device may be acquired, And determine whether the link of the second service node is normal according to the link status of each second service node. When the link of the second service node is normal, it means that the second service node can perform data transmission through the link, and the second service node is a normal node. Moreover, if the first convergence network device is the root cause of the fault, all second service nodes connected to the first convergence network device cannot work normally. Therefore, when the link state indicates that there are normal nodes among all the second service nodes connected to the first aggregation network device, it can be determined that the first aggregation network device is in a normal state, and then it is determined that the first access network device itself has a problem, That is, it is determined that the first access network device is the root cause of the failure. At this time, the failure of the service cluster may be called the failure of the access network device.

As shown in FIG. 8, in step 5035, the current candidate fault root is caused by the first access network device (access switch 30 in FIG. 8), and the management node can at least display the first access network device according to the topology diagram. device, the service node connected to the first access network device, the first convergence network device, and all second service nodes connected through the first convergence network device (convergence switch 40 in FIG. 8 ). In Figure 8, the black-filled dots are the service nodes that are determined as the candidate root cause of the fault, and the white-filled dots are the service nodes that are not determined as the candidate root cause of the fault. It can be known from FIG. 8 that if there are normal nodes among all the second service nodes connected to the first convergence network device, it can be determined that the first access network device is the root cause of the failure.

Step 5036: When the link status indicates that the second service node is a candidate for the root cause of the failure, the management node obtains the first core network device connected to the first convergence network device and other nodes connected to the first core network device based on the networking information. A third service node connected to the aggregation network device, where the third service node is a service node other than the target service node connected to the first core network device.

Since the probability that all service nodes connected to the same aggregation network device fail at the same time is low, when the link status indicates that all the second service nodes connected to the first aggregation network device are candidate root causes of failure, it can be determined that the The first convergent network device or a network device providing a network to the first converged network device fails. Then the management node can obtain the first core network device connected to the first aggregation network device and the third service node connected to other aggregation network devices connected to the first core network device based on the networking information, so as to determine the root cause of the failure based on them .

Step 5037: When the link state indicates that the third serving node includes a normal node, the management node determines that the first convergence network device is the root cause of the fault.

After determining the first core network device connected to the first aggregation network device and all third service nodes connected to the first core network device, the link status of all third service nodes connected to the first core network device may be obtained, and Whether the link of the third service node is normal is determined according to the link state of each third service node. When the link of the third service node is normal, it means that the third service node can perform data transmission through the link, and the third service node is a normal node. Moreover, if the first core network device is the root cause of the fault, all third service nodes connected to the first core network device cannot work normally. Therefore, when the link state indicates that there are normal nodes in all the third service nodes connected to the first core network device, it can be determined that the first core network device is in a normal state, and then it is determined that the first aggregation network device itself has a problem, that is, Determine the first aggregation network device as the root cause of the fault. At this time, the failure of the service cluster can be called the aggregation network device failure.

As shown in FIG. 9, in step 5037, the current candidate failure root is caused by the first convergence network device (convergence switch 40 in FIG. 9), and the management node can at least display the first convergence network device, the The access network device (the access switch 30 in FIG. 9 ), the service node, the first core network device (the core switch 50 in FIG. 9 ) connected to the first aggregation network device, and the network devices connected through the first core network device All third service nodes. In Figure 9, the black-filled dots are the service nodes that are determined as the candidate root cause of the fault, and the white-filled dots are the service nodes that are not determined as the candidate root cause of the fault. It can be seen from FIG. 9 that if there are normal nodes among all the third service nodes connected to the first core network device, it can be determined that the first convergence network device is the root cause of the fault.

Step 5038: When the link state indicates that the third service node is a candidate root cause of the fault, the management node determines that the first core network device is the root cause of the fault.

Since the probability that all service nodes connected to the same core network device fail at the same time is low, when the link status indicates that all the third service nodes connected to the first core network device are candidate root causes of the failure, it can be determined that the When the first core network device fails, it is determined that the first core network device is the root cause of the fault. At this time, the failure of the service cluster can be called the failure of the core network equipment. Among them, service node failures, access network equipment failures, aggregation network equipment failures, and core network equipment failures have an increasing impact.

It should be noted that the above steps 5031 to 5038 are examples of the implementation process of determining the root cause of the fault based on the aggregation analysis based on the link status and networking information, and are not intended to limit the implementation method. Under different application requirements, its implementation manner can be appropriately changed. For example, after determining that a device in a network level is a candidate root cause of a fault, other service nodes connected to a network device in an upper layer connected to the candidate root cause of the fault may be obtained. Then, select the first specified number of other service nodes and the second specified number of other service nodes among all other service nodes in normal state, and select the third specified number of service nodes among the service nodes of the candidate failure root cause node, and then use the first specified number of other service nodes as destination nodes, respectively use the second specified number of other service nodes and the third specified number of service nodes as source nodes, and then determine the relationship between each destination node and each source node and determine whether the candidate root cause of the fault is the root cause of the fault according to the logic of steps 5031 to 5038. By selecting a specified number of service nodes and other service nodes, and determining the root cause of the fault based on the link status between the selected service node and other service nodes, it is possible to reduce the amount of processing required for aggregation analysis and further improve the location of the root cause of the fault s efficiency.

Wherein, when the candidate root cause of the fault is a service node, the service node of the candidate root cause of the fault is the service node. When the candidate root cause of the fault is a network device that provides a network to the service node, the service node of the candidate root cause of the fault is the service node connected to the network device. The hierarchy of the service node and the network equipment that provides the network for it increases sequentially according to the increase of the logical distance to the service node. For example, for service nodes, access network devices, aggregation network devices, and core network devices, since the logical distances from service nodes, access network devices, aggregation network devices, and core network devices to service nodes increase sequentially, the service nodes, access network The levels of ingress network devices, aggregation network devices, and core network devices increase sequentially.

Exemplarily, as shown in FIG. 10 , assuming that the aggregation network device (aggregation switch 40 in FIG. 10 ) is a candidate root cause of failure, the core network device (core switch 50 in FIG. 10 ) connected to the aggregation network device can be obtained. other service nodes. Then, select the first specified number (1 in FIG. 10 ) of other service nodes and the second specified number (1 in FIG. 10 ) of other service nodes among all other service nodes in the normal state, and Select the third specified number of (2 in Figure 10) service nodes among the service nodes connected to the converging network device, then use the first specified number of other service nodes as destination nodes, and use the second specified number of other service nodes as destination nodes respectively and the third specified number of service nodes as source nodes, then determine the link status between each destination node and each source node (Y on the dashed arrow in Figure 10 indicates that the link status is normal, and N indicates that the link status is abnormal ), and determine whether the aggregation network device is the root cause of the failure according to the logic of the above step 5031 to step 5038.

To sum up, in the method for locating the root cause of a fault provided in the embodiment of the present application, through aggregation and analysis based on link status and networking information, the root cause of the fault can be automatically located globally, and the fault can be quickly located. The root cause improves the efficiency of locating the root cause of the fault and reduces the probability of service impact due to faults.

It should be noted that the sequence of steps in the method for locating the root cause of a fault provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be increased or decreased according to the situation. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes, which should be covered within the scope of protection of this application, and thus will not be repeated here.

The method for locating the root cause of a fault in the embodiment of the present application is described above. Corresponding to the above method, the embodiment of the present application also provides a device for locating the root cause of a fault. Fig. 11 is a schematic structural diagram of a device for locating the root cause of a fault provided by an embodiment of the present application. Based on the following multiple modules shown in FIG. 11 , the fault root cause location device shown in FIG. 11 can perform all or part of the operations shown in FIG. 5 above. It should be understood that the device may include more additional modules than those shown or omit some of the modules shown therein, which is not limited in this embodiment of the present application. The device for locating the root cause of a fault is applied to a management node that manages a service cluster. The service cluster includes a plurality of service nodes, and the service nodes are used to implement user services. As shown in Figure 11, the fault root cause location device 110 includes:

An acquisition module 1101, configured to acquire networking information of multiple service nodes.

The acquiring module 1101 is further configured to acquire link states of links between multiple service nodes when the service cluster fails.

The processing module 1102 is configured to perform aggregation analysis based on the link status and networking information to determine the root cause of the fault.

Optionally, multiple service nodes access the network through an access network device located at the access layer. The processing module 1102 is specifically used to determine the target service node when the link status of the link between the target service node and other service nodes cannot be obtained, or when the link status indication from other service nodes is disconnected from the target service node. The node is the candidate root cause of the failure; based on the networking information, the first access network device connected to the target service node and the first service node connected to the first access network device are obtained, and the first service node is the first access network device Connected service nodes other than the target service node; when the link state indicates that the first service node includes a normal node, determine that the target service node is the root cause of the failure.

Optionally, the access network device accesses the network through the convergence network device located at the convergence layer. Then the processing module 1102 is specifically configured to: when the link status indicates that the first service nodes are all candidate root causes of failure, based on the networking information, obtain the first convergence network device connected to the first access network device, and the first convergence network device. The second service node connected to other access network devices connected to the network device, the second service node is a service node other than the target service node connected to the first aggregation network device; when the link status indicates that the second service node includes a normal node , and determine that the first access network device is the root cause of the fault.

Optionally, the aggregation network device accesses the network through the core network device at the core layer. The processing module 1102 is specifically configured to: when the link state indicates that the second service node is a candidate root cause of failure, based on the networking information, obtain the first core network device connected to the first convergence network device, and the first core network device The third service node connected to other aggregation network devices connected to the device, the third service node is a service node other than the target service node connected to the first core network device; when the link status indicates that the third service node includes a normal node, determine The first aggregation network device is the root cause of the fault.

Or, when the link state indicates that all the third serving nodes are candidate root causes of the failure, determine that the first core network device is the root cause of the failure.

Optionally, the obtaining module 1101 is specifically configured to: receive link statuses of links between the service node and other service nodes provided by each service node.

Optionally, the acquisition module 1101 is specifically configured to: determine a plurality of service nodes to be tested among the plurality of service nodes; provide each service node with information on a plurality of service nodes to be tested, so that each service node obtains the The link status of the link between each service node to be tested; receiving the link status of the link between the service node and each service node to be tested provided by each service node.

Optionally, the network ranges of the multiple service nodes to be tested cover the network range of the service cluster.

Optionally, the link state is reflected by one or more of the following: link connectivity state and transmission delay.

To sum up, in the device for locating the root cause of the fault provided in the embodiment of the present application, by performing aggregation analysis based on the link state and networking information, the root cause of the fault can be automatically located globally, and the fault can be quickly located. The root cause improves the efficiency of locating the root cause of the fault and reduces the probability of service impact due to faults.

Wherein, both the obtaining module 1101 and the processing module 1102 can be implemented by software, or can be implemented by hardware. Exemplarily, the following takes the obtaining module 1101 as an example to introduce the implementation of the obtaining module 1101 . Similarly, the implementation manner of the processing module 1102 may refer to the implementation manner of the acquisition module 1101 .

A module is an example of a software functional unit, and the acquisition module 1101 may include codes running on computing instances. Wherein, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above computing instance may be one or more. For example, the acquisition module 1101 may include code running on multiple hosts/virtual machines/containers. It should be noted that multiple hosts/virtual machines/containers used to run the code can be distributed in the same region (region), or in different regions. Further, multiple hosts/virtual machines/containers used to run the code can be distributed in the same availability zone (Availability Zone, AZ), or in different AZs, and each AZ includes one data center or multiple geographically close data centers. Among them, usually a region can include multiple AZs.

Similarly, multiple hosts/virtual machines/containers for running the code can be distributed in the same virtual private cloud (virtual private cloud, VPC), or in multiple VPCs. Among them, usually a VPC is set in a region, and cross-region communication between two VPCs in the same region and between VPCs in different regions needs to set up a communication gateway in each VPC, and realize the interconnection between VPCs through the communication gateway. .

A module is an example of a hardware functional unit, and the obtaining module 1101 may include at least one computing device, such as a server. Alternatively, the acquisition module 1101 may also be implemented by using an application-specific integrated circuit (application-specific integrated circuit, ASIC), or a device implemented by a programmable logic device (programmable logic device, PLD). Wherein, the above-mentioned PLD can be realized by complex programmable logic device (complex programmable logical device, CPLD), field-programmable gate array (field-programmable gate array, FPGA), general array logic (generic array logic, GAL) or any combination thereof.

Multiple computing devices included in the acquisition module 1101 may be distributed in the same region, or in different regions. Multiple computing devices included in the acquisition module 1101 may be distributed in the same AZ, or in different AZs. Likewise, multiple computing devices included in the obtaining module 1101 may be distributed in the same VPC, or may be distributed in multiple VPCs. Wherein, the plurality of computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

It should be noted that, in other embodiments, any one of the acquisition module 1101 and the processing module 1102 may be used to execute any step in the fault root cause location method. The steps that the acquisition module 1101 and the processing module 1102 are responsible for can be specified according to the needs. The acquisition module 1101 and the processing module 1102 respectively implement different steps in the fault root cause location method to realize all functions of the fault root cause location device.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described devices and modules can refer to the corresponding content in the foregoing method embodiments, which will not be repeated here.

An embodiment of the present application provides a computing device. The computing device is used to implement part or all of the functions in the method for locating the root cause of a fault provided in the embodiment of the present application. FIG. 12 is a schematic structural diagram of a computing device provided by an embodiment of the present application. As shown in FIG. 12 , the computing device 1200 includes a processor 1201 , a memory 1202 , a communication interface 1203 and a bus 1204 . Wherein, the processor 1201 , the memory 1202 , and the communication interface 1203 are connected to each other through a bus 1204 .

Processor 1201 may include a general-purpose processor and/or a dedicated hardware chip. A general-purpose processor may include: a central processing unit (central processing unit, CPU), a microprocessor, or a graphics processing unit (graphics processing unit, GPU). The CPU is, for example, a single-core processor (single-CPU), or a multi-core processor (multi-CPU). The dedicated hardware chip is a high-performance processing hardware module. Dedicated hardware chips include at least one of digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (field-programmable gate arrays, FPGAs) or network processors (network processors, NPs). one item. The processor 1201 may also be an integrated circuit chip, which has a signal processing capability. During implementation, part or all of the functions of the fault root cause locating method of the present application may be implemented by an integrated logic circuit of hardware in the processor 1201 or instructions in the form of software.

The memory 1202 is used to store computer programs, and the computer programs include an operating system 1202a and executable codes (ie, program instructions) 1202b. The memory 1202 is, for example, a read-only memory or other types of static storage devices that can store static information and instructions, or a random access memory or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable only read-only disc or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage device, or is capable of carrying or storing Desired executable code in the form of instructions or data structures and any other medium capable of being accessed by a computer, without limitation. For example, the memory 1202 is used to store out port queues and the like. The memory 1202 exists independently, for example, and is connected to the processor 1201 through the bus 1204 . Or the memory 1202 and the processor 1201 are integrated together. The memory 1202 may store executable codes. When the executable codes stored in the memory 1202 are executed by the processor 1201, the processor 1201 is configured to perform some or all functions of the fault root cause location method provided by the embodiment of the present application. For an implementation manner of the processor 1201 executing the process, please refer to relevant descriptions in the foregoing embodiments. The memory 1202 may also include software modules and data required by other running processes such as an operating system.

The communication interface 1203 uses a transceiver module such as but not limited to a transceiver to realize communication with other devices or a communication network. For example, the communication interface 1203 may be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card and other devices with network access functions.

The bus 1204 is any type of communication bus used to interconnect the internal devices of the computing device (eg, the memory 1202, the processor 1201, the communication interface 1203). For example the system bus. In this embodiment of the present application, the above-mentioned devices inside the computing device are interconnected through the bus 1204 as an example for illustration. Optionally, the above-mentioned devices inside the computing device 1200 may also be communicatively connected to each other using other connection methods than the bus 1204 . For example, the above components inside the computing device 1200 are interconnected through internal logic interfaces.

It should be noted that the above-mentioned multiple devices may be respectively disposed on independent chips, or may be at least partly or completely disposed on the same chip. Whether each device is independently arranged on different chips or integrated and arranged on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation forms of the foregoing devices. In addition, the descriptions of the processes corresponding to the above-mentioned figures have their own emphasis. For the parts not described in detail in a certain process, you can refer to the relevant descriptions of other processes.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product that provides the program development platform includes one or more computer instructions. When these computer program instructions are loaded and executed on the computing device, all or part of the function of the fault root cause location method provided by the embodiment of the present application is realized.

Also, computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, computer instructions may be transferred from a website, computer, server, or data center via a wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. A computer readable storage medium stores computer program instructions providing a program development platform.

The embodiment of the present application also provides a computing device cluster. The cluster of computing devices includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

Optionally, the structure of at least one computing device included in the computing device cluster may refer to the computing device 1200 shown in FIG. 1 . The memory 1202 in one or more computing devices 1200 in the computing device cluster may store the same instructions for executing the fault root cause location method.

In some possible implementation manners, the memories 1202 of one or more computing devices 1200 in the computing device cluster may respectively store some instructions for executing the fault root cause location method. In other words, a combination of one or more computing devices 1200 can jointly execute instructions for performing the fault root cause location method.

It should be noted that the memories 1202 in different computing devices 1200 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the fault root cause locating apparatus. That is to say, the instructions stored in the memory 1202 in different computing devices 1200 may implement the functions of one or more of the obtaining module, the determining module and the grouping module.

In some possible implementations, one or more computing devices in a cluster of computing devices may be connected through a network. Wherein, the network may be a wide area network or a local area network or the like. Figure 13 shows a possible implementation. As shown in FIG. 13 , two computing devices 1300A and 1300B are connected through a network. Specifically, it is connected to the network through a communication interface in each computing device. In this class of possible implementations, computing devices 1300A and 1300B include a bus 1302 , a processor 1304 , a memory 1306 , and a communication interface 1308 . The memory 1306 in the computing device 1300A stores therein instructions to perform the functions of the processing modules. Meanwhile, the memory 1306 in the computing device 1300B stores instructions for executing the function of the acquisition module.

It should be understood that the functions of the computing device 1300A shown in FIG. 13 may also be performed by multiple computing devices 1300 . Likewise, the functions of computing device 1300B may also be performed by multiple computing devices 1300 . Moreover, the deployment mode of the modules used to implement the method for locating the root cause of the fault in the computing device can also be adjusted according to application requirements.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium is a non-volatile computer-readable storage medium. The computer-readable storage medium includes program instructions. When the program instructions are run on the computing device , make the computing device implement the method for locating the root cause of the fault as provided in the embodiment of the present application.

The embodiment of the present application also provides a computer program product containing instructions, and when the computer program product is run on the computer, the computer implements the method for locating the root cause of the fault provided in the embodiment of the present application.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, and can also be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, and the like.

It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application, All are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the networking information and link status mentioned in this application are obtained under the condition of full authorization.

In the embodiments of the present application, the terms "first", "second" and "third" are used for description purposes only, and cannot be understood as indicating or implying relative importance. The term "at least one" means one or more, and the term "plurality" means two or more, unless otherwise clearly defined.

The term "and/or" in this application is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and A and B exist alone. There are three cases of B. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

The above are only optional embodiments of the application, and are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the concept and principles of the application shall be included in the protection of the application. within range.

Claims (21)

1. A method for locating the root cause of a fault, wherein the method is applied to a management node that manages a service cluster, the service cluster includes a plurality of service nodes, and the service nodes are used to implement user services, the Methods include: Acquiring networking information of the plurality of service nodes; When the service cluster fails, acquire the link status of the links between the multiple service nodes; Aggregate analysis is performed based on the link state and the networking information to determine the root cause of the fault. 2. The method according to claim 1, wherein the plurality of service nodes access the network through an access network device located at the access layer, and the Perform aggregate analysis to determine the root cause of failures, including: When the link status of the link between the target service node and other service nodes cannot be obtained, or when the link status indication from other service nodes is disconnected from the target service node, determine that the target service node is a candidate failure root because; Based on the networking information, obtain the first access network device connected to the target service node and the first service node connected to the first access network device, where the first service node is the first access network device A service node other than the target service node connected to the inbound network device; When the link state indicates that the first service node includes a normal node, determine that the target service node is the root cause of the failure. 3. The method according to claim 2, wherein the access network device accesses the network through a convergence network device located at the convergence layer, and the convergence analysis is performed based on the link state and the networking information , to determine the root cause of the failure, including: When the link state indicates that the first service node is a candidate root cause of the failure, based on the networking information, obtain the first aggregation network device connected to the first access network device, and the first A second service node connected to other access network devices connected to the convergence network device, where the second service node is a service node other than the target service node connected to the first convergence network device; When the link state indicates that the second serving node includes a normal node, determine that the first access network device is the root cause of the failure. 4. The method according to claim 3, wherein the converged network device is connected to the network through a core network device located at the core layer, and the converged analysis is performed based on the link state and the networking information, Determine the root cause of the failure, including: When the link state indicates that the second service node is a candidate root cause of failure, based on the networking information, obtain the first core network device connected to the first aggregation network device, and the first core A third service node connected to other aggregation network devices connected to the network device, where the third service node is a service node connected to the first core network device except the target service node; When the link status indicates that the third serving node includes a normal node, determine that the first aggregation network device is the root cause of the failure. 5. The method according to claim 4, wherein said performing aggregation analysis based on said link state and said networking information to determine the root cause of the failure further comprises: When the link state indicates that all the third service nodes are candidate root causes of the failure, determine that the first core network device is the root cause of the failure. 6. The method according to any one of claims 1 to 5, wherein the acquiring the link status of the links between the multiple service nodes comprises: The link status of links between the service node and other service nodes provided by each service node is received. 7. The method according to claim 6, wherein, before acquiring the link states of the links between the plurality of service nodes, the method further comprises: determining a plurality of service nodes to be tested among the plurality of service nodes; providing each service node with the information of the plurality of service nodes to be tested, so that each service node obtains the link state of the link between the service node and each service node to be tested; The receiving the link state of the link between the service node and other service nodes provided by each service node includes: Receive the link state of the link between the service node and each service node to be tested provided by each service node. 8. The method according to claim 7, wherein the network range of the plurality of service nodes to be tested covers the network range of the service cluster. 9. The method according to any one of claims 1 to 8, wherein the link status is reflected by one or more of the following: connectivity status and transmission delay of the link. 10. A device for locating the root cause of a fault, wherein the device is applied to a management node that manages a service cluster, the service cluster includes a plurality of service nodes, and the service nodes are used to implement user services, the Devices include: An acquisition module, configured to acquire networking information of the plurality of service nodes; The obtaining module is further configured to obtain the link status of the links between the multiple service nodes when the service cluster fails; A processing module, configured to perform aggregation analysis based on the link state and the networking information to determine the root cause of the fault. 11. The device according to claim 10, wherein the plurality of service nodes access the network through an access network device located at the access layer, and the processing module is specifically used for: When the link status of the link between the target service node and other service nodes cannot be obtained, or when the link status indication from other service nodes is disconnected from the target service node, determine that the target service node is a candidate failure root because; Based on the networking information, obtain the first access network device connected to the target service node and the first service node connected to the first access network device, where the first service node is the first access network device A service node other than the target service node connected to the inbound network device; When the link state indicates that the first service node includes a normal node, determine that the target service node is the root cause of the failure. 12. The device according to claim 11, wherein the access network device accesses the network through a convergence network device located at the convergence layer, and the processing module is specifically used for: When the link state indicates that the first service node is a candidate root cause of the failure, based on the networking information, obtain the first aggregation network device connected to the first access network device, and the first A second service node connected to other access network devices connected to the convergence network device, where the second service node is a service node other than the target service node connected to the first convergence network device; When the link state indicates that the second serving node includes a normal node, determine that the first access network device is the root cause of the failure. 13. The apparatus according to claim 12, wherein the convergence network device accesses the network through a core network device located at the core layer, and the processing module is specifically used for: When the link state indicates that the second service node is a candidate root cause of failure, based on the networking information, obtain the first core network device connected to the first aggregation network device, and the first core A third service node connected to other aggregation network devices connected to the network device, where the third service node is a service node connected to the first core network device except the target service node; When the link status indicates that the third serving node includes a normal node, determine that the first aggregation network device is the root cause of the failure. 14. The device according to claim 13, wherein the processing module is specifically used for: When the link state indicates that all the third service nodes are candidate root causes of the failure, determine that the first core network device is the root cause of the failure. 15. The device according to any one of claims 10 to 14, wherein the acquisition module is specifically used for: The link status of links between the service node and other service nodes provided by each service node is received. 16. The device according to claim 15, wherein the acquiring module is specifically used for: determining a plurality of service nodes to be tested among the plurality of service nodes; providing each service node with the information of the plurality of service nodes to be tested, so that each service node obtains the link state of the link between the service node and each service node to be tested; Receive the link state of the link between the service node and each service node to be tested provided by each service node. 17. The apparatus according to claim 16, wherein the network range of the plurality of service nodes to be tested covers the network range of the service cluster. 18. The device according to any one of claims 10 to 17, wherein the link status is reflected by one or more of the following: connectivity status and transmission delay of the link. 19. A computing device cluster, characterized in that it includes at least one computing device, each computing device includes a processor and a memory, and the processor of the at least one computing device is used to execute the program stored in the memory of the at least one computing device instructions, so that the cluster of computing devices executes the method according to any one of claims 1-9. 20. A computer program product containing instructions, wherein when the instructions are executed by a cluster of computing devices, the cluster of computing devices executes the method according to any one of claims 1 to 9. 21. A computer-readable storage medium, characterized by comprising computer program instructions, and when the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices executes the method according to any one of claims 1 to 9. method.

Images