CN105515812A - Fault processing method of resources and device - Google Patents

Abstract

The invention provides a fault processing method of resources and a device. The fault processing method comprises steps of monitoring whether an assigned resource of a node in a network storage cluster system break down, wherein the assigned resource is a corresponding resource of an assigned resource type in pre-divided resource types in the network storage cluster system; and selecting and taking charge of a target object of the assigned resource according to a preset strategy when the assigned resource break down. In this way, problems in the prior art can be solved that takeover procedures are complex, errors are easy to make, and load of a takeover node is increased since businesses of the node which is still isolated even resource faults on the node belongs to partial faults are transferred to other takeover nodes; takeover procedures are simplified; error rate is reduced; and load of the takeover node is reduced.

Description

Resource troubleshooting method and device

technical field

The present invention relates to the communication field, in particular, to a method and device for resource fault handling.

Background technique

Network-attached storage systems are widely used in enterprise management platforms, and the security and reliability of their performance can be directly related to the daily operations of enterprises. Therefore, network-attached storage systems need to ensure stability and high availability.

According to the statistics made by Gartner, the causes of abnormal operation of the system can be mainly divided into the following aspects: application problems (40%), operation problems (40%), operating system failures (10%) and hardware failures (10%) ), for a network-attached storage cluster system, in many cases, it may also be that the software and hardware resources of a front-end access network port and a back-end storage resource are abnormal. In this scenario, except for the abnormal module that cannot run on the node, other modules are running normally. At this time, the technical solution adopted in the existing technology is to isolate the entire node and transfer the business to other nodes that can run normally. The above-mentioned technical solution will complicate the entire takeover process, and the probability of errors will increase accordingly. At the same time, the entire takeover will take a long time. pressure.

In addition, in the current network storage cluster, the fault management module mainly manages the storage resources on the local node, and the exception handling of the module itself is realized through the re-election of the node and the generation of a new takeover node. The most famous election algorithm is the Paxos algorithm, which is used in many open source projects, but the single-instance election of the basic node object cannot solve the election of multiple specific object resources in the node.

In related technologies, resource failures on nodes are partial failures in many cases, but the node is still isolated and the business of the node is transferred to other takeover nodes. The problem of taking over the load of nodes has not yet been proposed an effective solution.

Contents of the invention

In order to solve the above technical problems, the present invention provides a method and device for resource fault handling.

According to one aspect of the present invention, a resource failure processing method is provided, including: monitoring whether a specified resource of a node in the network storage cluster system fails, wherein the specified resource is a pre-partitioned resource in the network storage cluster system A resource corresponding to the specified resource type in the resource type; when the specified resource fails, select a target object to take over the specified resource according to a preset strategy.

Preferably, monitoring whether a specified resource of a node in the network storage cluster system fails includes: dividing resources of all nodes in the network storage cluster system into resource types; configuring resources of the same resource type in all nodes as one A service group: judging whether the specified resource is faulty by detecting the status of the specified resource in the service group.

Preferably, it is determined that the designated resource fails under the following conditions: when the state of the physical network port of the designated resource changes from the running state to the standby state, it is determined that the designated resource fails.

Preferably, selecting a target object to take over the specified resource according to a preset policy includes: selecting a service unit to take over the specified resource in the service group where the specified resource is located; using the node where the service unit is located as the target.

Preferably, the service unit that takes over the specified resource is selected in the service group where the resource is located in one of the following ways: select the service unit from the service group according to the preset priority; select the service unit according to the service group The value of the IP address of the service unit in selects the service unit.

Preferably, after the target takeover object takes over the failed specified resource, it further includes: saving switching information of the specified failure, wherein the switching information includes at least one of the following: The original node information of the specified resource and the resource type corresponding to the specified resource; when the original node where the specified resource is located fails and recovers, switch the specified resource back to the original node according to the switching information.

According to another aspect of the embodiments of the present invention, there is also provided a resource failure processing device, including: a monitoring module, configured to monitor whether a specified resource of a node in a network storage cluster system fails, wherein the specified resource is the A resource corresponding to a specified resource type among the pre-divided resource types in the network storage cluster system; a selection module configured to select a target object to take over the specified resource according to a preset policy when the specified resource fails.

Preferably, the monitoring module includes: a division unit, configured to divide resources of all nodes in the network storage cluster system into resource types; a configuration unit, configured to configure resources of the same resource type in all nodes as A service group; a judging unit, configured to judge whether the designated resource is faulty by detecting the status of the designated resource in the service group.

Preferably, the judging unit is configured to determine that the specified resource is faulty when the state of the physical network port of the specified resource changes from a running state to a standby state.

Preferably, the selection module includes: a selection unit, configured to select a service unit to take over the specified resource in the service group where the specified resource is located; a determination unit, configured to use the node where the service unit is located as the specified resource the service unit described above.

Through the present invention, after classifying the resources on the nodes, when the specified resources fail, the technical solution that only the failed resources can be transferred to other nodes solves the problem of resources on the nodes in many cases in the related art. The faults are all partial faults, but the node is still isolated, and the takeover process caused by transferring the node's business to other takeover nodes is complicated and error-prone. At the same time, it also increases the load of the takeover node, simplifies the takeover process, and reduces The error rate is reduced, and the load burden on the takeover node is also reduced.

Description of drawings

The accompanying drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention. In the attached picture:

FIG. 1 is a flow chart of a resource fault handling method according to an embodiment of the present invention;

FIG. 2 is a structural block diagram of a resource failure processing device according to an embodiment of the present invention;

Fig. 3 is another structural block diagram of a resource failure processing device according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of a resource protection group model according to a preferred embodiment of the present invention;

Fig. 5 is a flow chart of fault handling of resources according to a preferred embodiment of the present invention;

Fig. 6 is a flow chart of resource switchback in the preferred embodiment of the present invention.

detailed description

Hereinafter, the present invention will be described in detail with reference to the drawings and examples. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

In this embodiment, a resource fault handling method is provided. FIG. 1 is a flowchart of a resource fault handling method according to an embodiment of the present invention. As shown in FIG. 1 , the process includes the following steps:

Step S102, monitoring whether a specified resource of a node in the network storage cluster system fails, wherein the above-mentioned specified resource is a resource corresponding to a specified resource type among the pre-divided resource types in the above-mentioned network storage cluster system;

Step S104, when the designated resource fails, select a target object to take over the designated resource according to a preset strategy.

Through the above steps, after the resources on the nodes are classified, when one of the classified resources fails, only the failed designated resources can be transferred to other nodes, which solves the problem of related technologies In many cases, the resource failure on the node is a partial failure, but the node is still isolated, and the business of the node is transferred to other takeover nodes, which leads to a complex takeover process, which is prone to errors, and also increases the load on the takeover node. The problem is that the takeover process is simplified, the error rate is reduced, and the load burden on the takeover node is also reduced. Without isolation, it is necessary to avoid multi-terminal loading of resources, ensure business consistency, and continue to provide external services.

Optionally, the above step S102 can be implemented in multiple ways. In an example of the embodiment of the present invention, the following technical solution can be used to implement: divide the resources of all nodes in the network storage cluster system into resource types; combine all the above The resources of the same resource type in the nodes are configured as a service group; by detecting the status of the above-mentioned specified resources in the above-mentioned service group, it is judged whether the above-mentioned specified resources have failed, that is, the resources of the same resource type in all nodes in the network storage cluster system are logically divided In a service group, it detects whether the resource in the service group with the same resource type is faulty. Since a service group corresponds to a resource type, the fault type can be detected conveniently and quickly, and resource management is facilitated.

Since all physical network ports have two states: running state (ACTIVE) and standby state (STANDBY), when the state of the physical network port of the specified resource changes from the running state to the standby state, it can be determined that the specified resource is faulty.

In another optional embodiment of the embodiment of the present invention, the above step S104 can be implemented in the following manner: select a service unit to take over the specified resource in the service group where the above-mentioned specified resource is located; take the node where the above-mentioned service unit is located as the above-mentioned target Object, when monitoring resources in a service group, when a failure of a specified resource is detected, the service unit corresponding to the resource of the same resource type as the specified resource that has failed can be found in the same service group. After the service unit is determined , the node where the service unit is located is the above-mentioned target object (also can be understood as a takeover node).

In order to ensure the consistency of node services in the system, after the target object takes over the specified resource, the embodiment of the present invention also provides the following technical solution: after the target object takes over the execution resource, save the switchover information of the above-mentioned specified failure, wherein, the above-mentioned The switching information includes at least one of the following: the information of the original node where the above-mentioned specified resource is located, and the resource type corresponding to the above-mentioned specified resource; when the original node where the above-mentioned specified resource is located fails and recovers, switch the above-mentioned specified resource back to the above-mentioned original node according to the above-mentioned switching information .

To sum up, the embodiments of the present invention provide a high-availability mechanism for network-attached storage clusters, which solves the problems of current network-attached storage operation node data loss, high network load, multi-terminal loading of resources and other partial failure problems.

In order to better understand the fault handling process of the above resources, a preferred embodiment will be described below, but this embodiment of the present invention is not limited.

At first, the noun involved in the preferred embodiment of the present invention is simply explained as follows:

Service instance: the basic unit of protection resources (which can be understood as the resources in the above-mentioned service group), in the network-attached storage cluster, it corresponds to the collection of network virtual network ports and virtual disk objects. Taking the virtual network port as an example, the virtual network port is an abstraction of the aggregation of several physical network ports that currently provide network connections, and is unique within the entire cluster. The virtual network port is bound to the physical network port in the ACTIVE state, and the physical network port carries all services on the external virtual network port. When the physical network port in the ACTIVE state is abnormal, the target object is selected from the protected resource set in the STANDBY state through the configuration policy to take over, ensuring that the external services of the virtual network port are not interrupted.

Service unit: An individual with complete functions, which is deployed on each node in the cluster and can undertake the assignment of service instances. Each node in the storage cluster system includes a service unit consisting of two service instances, the front-end network port and the back-end virtual disk object. Assuming that there are N nodes in the current network-attached storage cluster system, one service unit can only undertake N copies of ACTIVE Service instance assignment, N copies of STANDBY service instance assignment.

Service group: A collection of objects of the same resource type on one or more service units, and specific objects in multiple service groups form a service unit. Taking the virtual network port as an example, all the physical network ports that bear the service of the virtual network port form the service group of the virtual network port. Each service group has an active/standby strategy, and the service groups are completely independent and do not affect each other. Each service group has its unique ID, which is specified when it is created and is unique within the network-attached storage cluster system.

Belonging node: The storage front-end and back-end virtual resources are specified when they are created. The same virtual resource can only belong to one node. When powering on, the service unit object on the home node is preferentially selected as the ACTIVE service instance.

Configuration strategy: The front-end and back-end virtual resources are specified at the time of creation. When the resource is abnormal, the service unit object is selected to take over according to this policy. By default, the service unit corresponding to the IP address with a relatively small IP address value is given priority to take over. At the same time, an interface is provided to support manual intervention , configure different weights for the service unit objects, and take over the failed resources first with the higher weight.

Takeover node (can be understood as the target object of the above embodiment): when the current back-end resource ACTIVE service unit is abnormal, an election is initiated from the STANDBY node according to the configuration policy, and a new ACTIVE service unit object is generated. The node where the service unit object is located is called to take over the node.

Main decision-making node: the node where the ACTIVE service instance elected by the fault management module is powered on. When the fault management module itself is abnormal, the election will be re-initiated, thereby generating a new ACTIVE fault management service instance assignment. The node where the new service instance is located is New master decision node.

The technical solution provided by the preferred embodiment of the present invention can be roughly summarized as follows: by defining a protection resource model and a fault management framework, managing network-attached storage front-end network and back-end storage resources, high availability of the entire storage cluster resource is achieved.

When some front-end and back-end resources are abnormal, heartbeat monitoring is performed on some of the resources in the protection resources. Once the monitoring module detects the abnormality, an alarm will be notified to the fault management module; when the fault management module receives the alarm, it will make decisions according to the protection resource takeover priority Take over the resources that need to be taken over to ensure the continuity of external services; at the same time, record the switching information of the abnormal resource;

Optionally, when the fault is resolved, the state of the faulty module is automatically synchronized to the protection resource group, the monitoring module perceives the fault recovery, and executes a fault recovery request to the fault management module, and the fault management module performs corresponding switchback according to the switching information of the abnormal resource operate.

In the technical solution provided above in the embodiment of the present invention: the resource protection group model can be roughly described as follows: a monitoring module is resident on each node, which is responsible for heartbeat monitoring and management, and is elected in the service group according to the configuration policy in case of abnormality. This module is resident in each node in the form of a daemon thread. The node that is powered on the earliest is the main decision-making node. If multiple nodes are powered on at the same time, the node with a smaller IP address value is elected as the main decision-making node by comparing IPs. Nodes communicate through Remote Procedure Call protocol (RPC) messages. Under normal circumstances, the main decision-making node initiates a heartbeat check, and collects service unit status information on other nodes according to the service group identifier. Other nodes are based on at least one of the following events: To determine whether to resend the beacon for a new election: 1. Whether the regular heartbeat check time exceeds the maximum check time; 2. Whether the service unit in the current ACTIVE state is abnormal. When one of the above conditions is met, it will report to all clusters The station sends a beacon to initiate the election of the ACTIVE service unit.

The failure management module of the main decision-making node elected by the fault management service identifier manages the front-end and back-end resources of the entire storage. The ACTIVE service unit in the front-end and back-end resources performs the work of the service instance, and all services are carried on the service instance. Other services The unit is in the STANDBY state of the service instance. After monitoring the abnormality of the ACTIVE service unit, the fault management module is responsible for the entire takeover cooperation. The specific process cooperation is realized through the following process:

Step 1: Configure virtual network ports and virtual disk shared storage service groups on each node. The front-end virtual network service group is used for user storage network access, and the back-end virtual disk storage service group is used to store shared storage data resources;

Step 2: Assign all virtual resources to the home node, register and configure resources into the resource service unit, under normal circumstances, virtual resources actually run in the service unit on the home node, and the service unit is in the ACTIVE state;

Step 3: The monitoring module performs real-time heartbeat monitoring on all resource protection group resources, and sends an alarm once it finds that the running resources in the protection resource group are abnormal;

Step 4: The fault management module receives an exception, and offline the service unit resources in the service group that are currently running abnormally;

Step 5: According to the current node and service group ID, select the target takeover service unit object according to the configuration policy for migration and record saving, and set the new service unit to ACTIVE state;

Step 6: When the abnormal front-end and back-end resources return to normal, the resource service group will be automatically updated and the fault management module will be notified;

Step 7: The fault management module switches back to the running resource on it according to the migration record at the time of abnormality. Failure recovery, adjust the state of two service unit objects at the same time.

The preferred embodiment of the present invention achieves the following technical effects: through the resource protection group model, the cluster nodes are refined according to the front-end network resources and the back-end storage resources. In the case of abnormal node resources, it is supported to only take over the abnormal part of the node and keep the normal node. run part. Thereby improving the overall performance and realizing the effective utilization of network-attached storage cluster resources; meeting the requirements of high availability, stability and scalability of key businesses, and can be used for fault detection, takeover decision-making, and fault isolation required by multi-machine hot backup of high-availability storage clusters and switching, recovery and expansion; by improving the Paxos algorithm, multi-instance elections are supported according to node and service group identifiers, which improves the flexibility of elections, and the fault management module itself is added to the protection resource group for hot backup, which simplifies system implementation and effectively solves the main decision The fault management module itself on the node is abnormal; deploy a hot standby host in the cluster system to make full use of the host's own computing power, improve the takeover response speed, and reduce costs.

This embodiment also provides a resource failure processing device, which is used to implement the above embodiments and preferred implementation modes, and what has already been described will not be repeated. As used below, the term "module" may be a combination of software and/or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.

Fig. 2 is a structural block diagram of a resource failure processing device according to an embodiment of the present invention. As shown in Fig. 2, the device includes:

The monitoring module 20 is configured to monitor whether a specified resource of a node in the network storage cluster system fails, wherein the above-mentioned specified resource is a resource corresponding to a specified resource type among the pre-divided resource types in the above-mentioned network storage cluster system;

The selection module 22 is connected with the monitoring module 20, and is used for selecting a target object to take over the designated resource according to a preset policy when the designated resource fails.

Through the comprehensive functions of the above-mentioned modules, after classifying the resources on the nodes, when one of the classified resources fails, only the designated resources that have failed can be transferred to other nodes to solve the problem. In related technologies, in many cases, the resource failure on a node is a partial failure, but the node is still isolated, and the business of the node is transferred to other takeover nodes, resulting in a complicated takeover process and error-prone, which also increases the number of takeover nodes. The problem of load, simplifies the takeover process, reduces the error rate, and also reduces the load burden on the takeover node

Fig. 3 is another structural block diagram of a resource failure processing device according to an embodiment of the present invention. As shown in Fig. 3, in addition to all the modules shown in Fig. 2, the device also includes:

In order to realize the above-mentioned function of monitoring whether a specified resource of a node in the network storage cluster system fails, the monitoring module 20, in an optional embodiment of the embodiment of the present invention, the monitoring module 20 may include the following unit: a dividing unit 200 for Resource types of all nodes in the above-mentioned network storage cluster system are divided into resource types; the configuration unit 202 is connected to the division unit 200, and the division unit is used to configure resources of the same resource type in all the above-mentioned nodes as a service group; the judgment unit 204 , connected to the configuration unit 202, for judging whether the above-mentioned designated resource fails by detecting the state of the above-mentioned designated resource in the above-mentioned service group, wherein, the judging unit 204 is used for when the state of the physical network port of the above-mentioned designated resource changes from the running state to the standby state, it is determined that the above-mentioned specified resource is faulty.

Optionally, the selection module 22 may also include the following units: a selection unit 220, configured to select a service unit to take over the specified resource in the service group where the specified resource is located; a determination unit 222, connected to the selection unit 220, for The node where the above-mentioned service unit is located serves as the above-mentioned target object.

In the embodiment of the present invention, the target object in the selection module 22 can be understood as the takeover node in the above embodiment.

The technical solutions of the embodiments of the present invention are further elaborated in conjunction with the following preferred embodiments:

Fig. 4 is a schematic diagram of a resource protection group model according to a preferred embodiment of the present invention. As shown in Fig. 4, there are two service groups: a virtual network port service group and a virtual disk service group, and there are two service instances: a virtual network port service instance and virtual disk service instance. The virtual network port service instance is protected and executed by the virtual network port service group, and the virtual disk service instance is protected and executed by the virtual disk service group. Among them, the solid line arrow points to the ACTIVE service unit object, which actually carries services, and the dotted line arrow points to the STANDBY service unit object, and a new ACTIVE unit is assigned to take over the object in case of an exception.

From the schematic diagram provided in Figure 4, it can be known that in the virtual network port service group, service unit 3 is arranged to perform the ACTIVE work of the virtual network port service instance, and service unit 1 and service unit 2 perform the STANDBY work of the virtual network port service instance, as shown in Fig. The connection between the virtual disk network port service instance and the virtual disk service instance and the service unit in 4. The solid line represents ACTIVE; the dotted line connection is STANDBY designation.

In the virtual disk service group, arrange service unit 2 to execute the ACTIVE work of the virtual disk service instance, and service unit 1 and service unit 3 to execute the STANDBY work of the virtual disk service instance.

Fig. 5 is a flowchart of fault handling of resources according to a preferred embodiment of the present invention, as shown in Fig. 5:

In the scenario where some resources of a node are abnormal, the entire takeover process triggered by a resource failure:

Step S502: The state of the service protection resource of the resource belonging node changes (triggered by equipment failure or man-machine command), changes from ACTIVE to STANDBY state, and notifies the monitoring agent module on this node;

Step S504: The main decision-making node monitoring module communicates with each node monitoring agent through a regular heartbeat, perceives that the state of the corresponding type of protection resource is abnormal, and sends a switching request to the fault management module of this node;

Step S506: The fault management module notifies the abnormal home node agent module to take the affected resource offline, executes the resource offline operation, and returns a resource offline response to the main decision node fault management module after resource cleaning;

Step S508: The main decision node failure management module receives the resource offline response, elects a takeover node for the abnormal resource according to the configuration policy, and sends a resource online request to the takeover node agent module;

Step S510: After receiving the resource online request, the agent module of the target node notifies the fault management module of the main decision-making node after performing the resource online operation to the business module, and replies with a resource online response;

Step S512: The failure management module of the main decision-making node receives the resource online response, considers that the switching is completed, and replies the switching response to the monitoring module of this node, and the process ends.

Fig. 6 is a flow chart of resource switching back in a preferred embodiment of the fundamental invention, as shown in Fig. 6:

In the scenario where some resources of a node are abnormally restored, the entire switchback process triggered by resource failure recovery:

Step S602: The state of the service protection resource of the resource belonging node changes (triggered by equipment failure recovery or man-machine command) from STANDBY to ACTIVE state, and notifies the monitoring agent module on this node;

Step S604: The main decision-making node monitoring module communicates with each node monitoring agent through a regular heartbeat, perceives that the state of the corresponding type of active protection resource is restored, and sends a switching request to the fault management module of this node;

Step S606: the fault management module notifies the agent module of the takeover node to take the resources offline, and after cleaning up the resources, returns a resource offline response to the main decision node fault management module;

Step S608: The main decision node failure management module receives the resource offline response, and sends a resource online request to the original home node agent module;

Step S610: After receiving the resource online request, the agent module of the resource home node executes the resource online operation to the business module, and then returns a resource online response to the fault management module of the master decision node;

Step S612: The failure management module of the main decision-making node receives the resource online response, considers that the switchover is completed, and returns a switchback response to the monitoring module of the local node, and the process ends.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is Better implementation.

In another embodiment, software is also provided, and the software is used to implement the technical solutions described in the above embodiments and preferred implementation manners.

In another embodiment, there is also provided a storage medium, in which the software is stored, the storage medium includes but not limited to: optical discs, floppy disks, hard disks, rewritable memories, and the like.

It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

In summary, the embodiment of the present invention achieves the following technical effects: the takeover process is simplified, the error rate is reduced, and the load burden on the takeover node is also reduced, that is to say, the technical solution of the embodiment of the present invention is adopted: takeover Nodes only take over some resources that have problems. Since the node where the fault is located is not isolated, it is necessary to avoid multi-terminal loading of resources, ensure business consistency, and continue to provide external services.

Obviously, those skilled in the art should understand that each module or each step of the above-mentioned present invention can be realized by a general-purpose computing device, and they can be concentrated on a single computing device, or distributed in a network formed by multiple computing devices Alternatively, they may be implemented in program code executable by a computing device so that they may be stored in a storage device to be executed by a computing device, and in some cases in an order different from that shown here The steps shown or described are carried out, or they are separately fabricated into individual integrated circuit modules, or multiple modules or steps among them are fabricated into a single integrated circuit module for implementation. As such, the present invention is not limited to any specific combination of hardware and software.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a fault handling method for resource, is characterized in that, comprising:

Whether the allocated resource of monitoring network storage cluster system interior joint breaks down, and wherein, described allocated resource is for specifying the resource corresponding to resource type in the resource type that divides in advance in described network storage group system;

When described allocated resource breaks down, select the destination object of the described allocated resource of adapter according to preset strategy.

2. method according to claim 1, is characterized in that, whether the allocated resource of monitoring network storage cluster system interior joint breaks down and comprise:

The division of resource type is carried out to the resource of all nodes in described network storage group system;

Be a service group by resource distribution identical for resource type in described all nodes;

Whether broken down by allocated resource described in the condition adjudgement that detects allocated resource described in described service group.

3. method according to claim 2, is characterized in that, determines that described allocated resource breaks down in a case where:

When the physical internet ports state of described allocated resource transfers state for subsequent use to by run mode, determine that described allocated resource breaks down.

4. method according to claim 2, is characterized in that, selects the destination object of the described allocated resource of adapter, comprising according to preset strategy:

The service unit of the described allocated resource of adapter is selected in the service group at described allocated resource place;

Using the node at described service unit place as described destination object.

5. method according to claim 4, is characterized in that, selects by one of following mode the service unit taking over described allocated resource in the service group at described resource place:

From described service group, described service unit is selected according to the priority preset;

Described service unit is selected according to the IP address values of service unit described in described service group.

6. the method according to any one of claim 1 to 5, is characterized in that, after described destination object is taken over the described allocated resource broken down, also comprises:

Preserve the handover information of described specified fault, wherein, described handover information comprise following one of at least: the resource type that the origin node information at described allocated resource place, described allocated resource are corresponding;

When the origin node fault recovery at described allocated resource place, according to described handover information, described allocated resource is switched back described origin node.

7. a fault treating apparatus for resource, is characterized in that, comprising:

Monitoring modular, whether the allocated resource for monitoring network storage cluster system interior joint breaks down, and wherein, described allocated resource is for specifying the resource corresponding to resource type in the resource type that divides in advance in described network storage group system;

Select module, for when described allocated resource breaks down, select the destination object of the described allocated resource of adapter according to preset strategy.

8. device according to claim 7, is characterized in that, described monitoring modular comprises:

Division unit, for carrying out the division of resource type to the resource of all nodes in described network storage group system;

Dispensing unit, for being a service group by resource distribution identical for resource type in described all nodes;

Whether judging unit, break down for allocated resource described in the condition adjudgement by detecting allocated resource described in described service group.

9. device according to claim 8, is characterized in that, described judging unit is used for when the physical internet ports state of described allocated resource transfers state for subsequent use to by run mode, determines that described allocated resource breaks down.

10. device according to claim 8, is characterized in that, described selection module, comprising:

Selected cell, for selecting the service unit of the described allocated resource of adapter in the service group at described allocated resource place;

Determining unit, for using the node at described service unit place as described destination object.