CN101079896B - A method for constructing a multi-availability mechanism coexistence architecture of a parallel storage system - Google Patents

Abstract

The invention discloses a multi-availability mechanism coexistence architecture of a parallel storage system, which includes a state detection and control framework, a data service framework, a metadata service framework, a data synchronization framework, a client framework, a system management framework, and a high-availability mechanism module 7 A part that supports online module loading and unloading. The high-availability mechanism module implements all the interface functions required by the framework in the form of plug-ins. The framework calls the interface implemented in the corresponding availability module according to the type of high-availability mechanism used by the logical data to complete specific functions. With the support of this architecture, users can choose the most suitable mechanism among the high-availability mechanisms provided by the system to ensure the reliability of logical data according to the availability requirements of logical data, read and write characteristics, and user quality of service requirements for data. And the availability of data services, so as to avoid unnecessary performance loss and disk redundancy caused by using a single high-availability mechanism.

Description

A method for constructing a multi-availability mechanism coexistence architecture of a parallel storage system

technical field

The invention relates to the field of computer application technology, and is a method for constructing a multi-availability mechanism coexistence framework of a parallel storage system, in particular a multi-availability mechanism coexistence framework of a distributed storage system based on a parallel file system and a distributed file system.

Background technique

A highly available system means that when a software or hardware failure occurs in the system, it will not cause the system to stop service, but allows the system to run with failures. In the parallel storage system, most of the existing technologies are implemented through data redundancy. If some data is not available, its backup data can provide services instead. A high-availability system usually consists of two or more nodes, which are connected to clients through the Internet, and each node has its own local storage space.

Most of the existing high-availability systems only provide a single high-availability mechanism, and all logical data use this high-availability mechanism to ensure data security. Since different logical data have different high-availability requirements, using a single high-availability mechanism will inevitably cause system performance loss and waste of storage space. Although some high-availability systems can dynamically configure high-availability mechanisms according to needs, they cannot dynamically determine which high-availability mechanism should be used according to the needs of logical data.

When realizing the high availability of the parallel storage system, it is necessary to provide a multi-availability mechanism coexistence architecture that can be applied to the system to support multiple high-availability mechanisms, and to enable different logical data to use different high-availability mechanisms. The multi-availability mechanism coexistence architecture proposed in this patent is to meet this requirement of logical data. With the support of this architecture, users can decide to use the appropriate available mechanism provided in the system to ensure the reliability and Availability of Data Services.

Contents of the invention

The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art, and provide a multi-availability mechanism coexistence architecture of a parallel storage system. The purpose is to enable users to select a high-availability mechanism for logical data, reducing unnecessary performance loss and disk redundancy.

The technical solution of the present invention is realized in this way: the framework is composed of the following seven parts, namely: state detection and control framework, data service framework, metadata service framework, data synchronization framework, client framework, system management framework, high-level The available mechanism module, the state detection and control framework is responsible for detecting and controlling the state of all entities on this node, and the data service framework is responsible for creating specific data service threads, distributing requests to data service threads, and completing data redundancy required by specific high-availability mechanisms. Yuhe service takeover function, the metadata service framework calls different functions to complete metadata operations according to the high availability mechanism of logical data, and the data synchronization framework supports the coexistence of data synchronization threads with multiple high availability mechanisms to complete mutually redundant data The client framework provides a complete set of functions for users to access the parallel storage system, supports multiple high-availability mechanism modules, and calls the corresponding high-availability mechanism functions according to the requested high-availability mechanism type. The system management framework provides an implementation Interfaces for system configuration, system monitoring, and system control functions. The high-availability mechanism module is used as a plug-in to realize the functional interfaces of the other 6 parts.

The workflow of the whole architecture is as follows:

a. When a user initiates read and write access to logical data, he first sends a request to the metadata service framework through a function in the client framework to obtain the metadata information of the logical data. High availability mechanism type;

b. Then, the client framework calls the interface function of the client framework implemented in the corresponding high availability mechanism module according to the high availability mechanism type of the logical data. This function completes the read and write operations by sending an access request to the data service framework on the data node ;

c. The data service framework distributes the data request to the corresponding data service thread according to the high-availability mechanism type of the logical data attached to the access request, and the data service thread processes the request and returns a response to the client framework to complete the data request The response operation;

d. If the data service framework or data service thread on a data node cannot be accessed, the client framework sends a status confirmation request to the status detection and control framework of the data node to confirm the current status of the data service thread, and the status detection After receiving the state confirmation request, the control framework calls the corresponding state query function to confirm the current state of the queried entity, and at the same time, calls the relevant state query function to communicate with the state monitoring framework on the node where the entity's related entity is located to obtain its current state , according to all obtained states, query the state transition table in the configuration information of the high-availability mechanism. When a previous item in the state transition table matches all the current states, set the state of the local entity as the subsequent state of the state transition entry The state specified in the item, if there is no matching entry, do nothing, and then return the converted entity state to the client;

e. When a data node fails, and the data service threads on other data nodes related to the data service thread of the node are running normally, the data service framework of the relevant data node records the redundant data related to the data of the failed node the modification log of

f. After a fault occurs, the system administrator can know that a data service thread has failed through the system management framework. After manual intervention, the system administrator starts the data synchronization process through the system management framework. The synchronization function in the high-availability mechanism module starts the data synchronization thread, accesses the data synchronization framework of other related nodes, and synchronizes the data of the faulty node according to the modification log. After the data synchronization is completed, the state detection and control framework on the data node is notified to adjust The state of the data service thread.

The parallel storage system is composed of multiple nodes, and the nodes are interconnected through a network. The nodes can be divided into four types according to their functions: metadata nodes, data nodes, client nodes, system management nodes, and metadata node storage logic The description information of the data, and in response to the access request for this information, the file data itself is divided into logical data blocks and stored on multiple data nodes, the client node is the user of logical data, and the management node provides System configuration and management functions.

The state detection and control framework has a state detection and control framework on each data node and metadata node of the parallel storage system, which is responsible for detecting and controlling the state of all entities on this node. The state of the entity on the node, the state detection and control framework will communicate with the state detection and control framework on other nodes, and the state of the required entity can be obtained through the state detection and control framework on other nodes. Entities are all services related to high availability mechanisms Programs, including data service threads and data synchronization threads in each high-availability mechanism, the parallel storage system needs to detect the running conditions of these entities in real time, and obtain the status of each entity. The status of entities is different due to different types of high-availability mechanisms. However, at least two states are included to indicate whether it is active or inactive.

The data service framework creates a data service thread at startup, loads a specific data service function, and then distributes access requests for the logical data to corresponding The data service thread responds to the request and returns the data required by the client. At the same time, the data service thread completes the data redundancy and service takeover functions required by a specific high-availability mechanism. The data service framework supports multiple high-availability mechanism modules. Coexistence, dynamic loading and unloading of various high-availability mechanism modules.

As for the data service thread, each high-availability mechanism has one or more corresponding data service threads, and each data service thread is an entity for the state detection and control framework, and the entity state includes an active state and a backup state , Synchronization state, and Stop state represent different operating conditions. The data service thread and data service framework provide the access interface of the entity state, which only allows access to the state detection and control framework of the same node, and the entity can also change its state independently.

The metadata service framework supports multiple high-availability mechanisms. Different interface functions are called to complete metadata operations according to different high-availability mechanisms of logical data. Metadata operations are implemented in different high-availability mechanisms according to different high-availability mechanisms. In the mechanism module, it can be loaded and unloaded dynamically.

The data synchronization framework can support the coexistence of data synchronization threads of multiple high-availability mechanisms, provide fault recovery under high-availability mechanisms, and data synchronization operations between mutually redundant data. There are two types of synchronization server threads and data synchronization client threads as synchronization data requesters. The data synchronization framework distributes data synchronization requests to corresponding data synchronization service threads according to the type of high-availability mechanism for data synchronization and the type of synchronization request thread.

The client framework provides a complete set of functional interfaces for users to access the parallel storage system. Users access the data in the parallel storage system by calling these interfaces. At the same time, the client framework supports the coexistence of multiple high-availability mechanisms and provides high-availability mechanism module loading And unloading operations, according to the high-availability mechanism type of the logical data accessed by the client, call the corresponding function to complete the data access. Each high-availability mechanism provides a set of client access functions, corresponding to the function interface provided by the client framework , with the support of these functions, the client completes data redundancy, failover, and failover operations specific to each high-availability mechanism.

The system management framework provides a friendly interface for managing the parallel storage system, which implements system configuration, system monitoring, and system control functions. The system configuration includes the configuration for the parallel storage system itself, including system scale, node type, node Information configuration also includes the configuration of high-availability mechanism modules. In system monitoring, the monitoring of the overall information of the system and the operation information of each node is realized. It also includes the detection of entity information and entity status corresponding to all high-availability mechanism modules. , providing the system administrator with a complete system view. The system control function realizes real-time control of all nodes in the system and state control of entities corresponding to all high-availability mechanisms. Administrators can manually turn on and off the system by using these control functions Data service and data synchronization operations within.

The high-availability mechanism module is implemented as different plug-ins, and the plug-ins implement the functional interfaces of the other six frameworks. These frameworks select and call the interface functions implemented in the corresponding availability modules according to the type of high-availability mechanism used by the logical data to complete the data access, failure recovery, and system administration.

Due to the adoption of the above technical solution, the present invention can dynamically accommodate multiple high-availability mechanisms coexisting, and call appropriate high-availability mechanism threads provided in the system according to the high-availability requirements of logic data for processing. Using the most suitable high-availability mechanism for different types of logical data can avoid unnecessary performance loss and waste of storage space caused by using a single high-availability mechanism. Guaranteed.

Description of drawings

FIG. 1 is a structural diagram of a parallel storage system applying the multi-availability mechanism coexistence architecture.

Fig. 2 is a flow chart of parallel file system file access when there is no fault.

Figure 3 is a diagram of the data node software structure.

Fig. 4 is a diagram of the software configuration of the metadata node.

Fig. 5 is a diagram of client node software configuration.

Fig. 6 is a flow chart of data synchronization during fault recovery.

Fig. 7 is a flow chart of client access when a data node fails.

The content of the present invention will be described in further detail below in conjunction with the accompanying drawings.

Detailed ways

Referring to FIG. 1 , a parallel storage system applying this architecture is composed of multiple nodes, and the nodes are interconnected through a network. Nodes can be divided into four types according to their functions: metadata nodes, data nodes, client nodes, and system management nodes. Metadata nodes store the description information of logical data and respond to access requests for this information. Logical data is divided into data blocks and stored on multiple data nodes. Client nodes are consumers of logical data. The management node provides system configuration and management functions for system administrators.

Referring to FIG. 2 , the file access process of the parallel storage system when there is no fault is: firstly, the metadata service module is accessed through the function in the client framework to obtain the metadata information of the logical data. The client framework calls the function corresponding to the high-availability mechanism to access the data service framework on the data node according to the type of the high-availability mechanism. After network communication, the data service framework gets the request and obtains the logical data according to the logical data metadata information attached to the request The high-availability mechanism type, and then distribute the data request to the data service thread of the high-availability mechanism corresponding to this type, and the data service thread processes the request and returns a response to the client to complete the data request.

Referring to Figure 3, the data service node software composition diagram includes a state detection and control framework, a data service framework, various high-availability mechanism modules, a management agent and a data synchronization framework. State detection and transfer to the corresponding detection and control function interfaces in various high-availability mechanism modules, responsible for the state detection and setting operations of various entities. The data service framework transfers the corresponding data service function to be responsible for the data service on this node. The management agent receives the request from the management node and performs the operations required in the request. The data synchronization framework is called into the corresponding data synchronization function interface, which is responsible for the balance operation of data in the case of fault recovery.

Referring to Figure 4, the metadata node software composition diagram includes state detection and control framework, metadata service framework, various high-availability mechanism modules and management agent, the state detection and control framework calls specific functions of high-availability mechanism modules, To obtain and set the status of entities related to the high availability mechanism, the metadata framework also needs to call the functions in the specific high availability mechanism module to complete the metadata operations related to the high availability mechanism.

Referring to Figure 5, the client node software composition diagram includes various lib functions, client framework, various high-availability mechanism modules and management agent. When an operation calls a lib function, the lib function calls the corresponding function in the corresponding high availability mechanism module through the client framework to complete the operation. Each high-availability mechanism module should implement the interface required by the client framework.

Referring to Figure 6, the flow chart of data synchronization during fault recovery is: the system management framework sends a request to start data synchronization to the state detection and control framework of the faulty data node, and the request is forwarded to the data synchronization framework of the data node, and the data According to the type of the high-availability mechanism, the synchronization framework requests the state detection and control framework to set the service state of the current high-availability mechanism to the blocked state, and starts all relevant data synchronization client threads. Then the state detection and control framework sends a request to open the data synchronization server thread to other related data nodes. When the request is completed, the data service thread corresponding to the high availability mechanism of the current node is also blocked. After the data synchronization server thread is opened, it analyzes the recorded modification log, reorganizes the log, generates a synchronization list, and waits for the access of the data synchronization client thread. After the data synchronization client thread is started, it directly requests the data synchronization item from the data synchronization server thread of the relevant node until the data synchronization is completed. Finally, the log information is deleted, and the client and server of the data synchronization thread send notifications of synchronization completion to their respective state detection and control frameworks, and the state detection and control framework adjusts the state of the data service program to normal and terminates data synchronization thread.

As shown in Figure 7, the client node includes user process, client framework and various high-availability mechanism modules, and the data service node includes state detection and control module, data service framework, data synchronization framework, management agent and various high-availability mechanism modules. When a data node fails, the flow chart of client access is as follows: the client first fails to connect to the data service framework, and then queries the state detection and control module for the state of the current high availability mechanism thread. The state detection and control framework calls the state acquisition function in the current high-availability mechanism module to obtain the state of the entity and its related entities, and returns it to the client. The management agent reports current status changes to the system management module.

Building a high-availability mechanism coexistence architecture suitable for parallel storage systems includes building seven parts of the framework: state detection and control framework, data service framework, metadata service framework, client framework, data synchronization framework, system management module and high-availability mechanism module.

The state detection and control framework is the agent of the state request operation of the entity on the node, which is responsible for detecting and controlling the state of all entities on the node, so as to meet the request for the state of the entity in the failover and recovery part of the high availability mechanism. In implementation, it is a resident program that exists on each data node and metadata node of the system, and loads all high-availability mechanism modules at startup. Its work includes receiving external request operations on entity status, sending status requests to monitoring entities, controlling the status of entities as needed, and responding to responses. The state detection and control framework can dynamically load and unload various high-availability mechanism modules.

The data service framework is also a resident program that exists on each data node. It loads all high-availability mechanism modules at startup and generates at least one data service thread for each module. According to the high-availability mechanism type of the logical data requested by the client, the data service framework distributes the access request for the logical data to the corresponding data service thread, and the thread responds to the request and returns the data required by the client. At the same time, The data service thread completes functions such as data redundancy and service takeover required by a specific high availability mechanism. The data service framework supports the coexistence of multiple high-availability mechanism modules, and can dynamically load and unload the data service threads corresponding to various high-availability mechanism modules, and correctly distribute client requests according to the client's request content, and simultaneously process all mechanisms. Common operations, such as restarting services and other operations.

The metadata framework is deployed on the metadata node, which is also a resident program. It loads all high-availability mechanism modules at startup and obtains pointers to processing functions associated with the high-availability mechanism, and provides metadata services to the outside. It receives requests from other nodes for metadata, calls different functions according to different high-availability mechanisms to complete mechanism-related operations, and returns responses.

The client provides a set of functional interfaces for users to access the parallel storage system, which can be the function library interface or the VFS interface, and the user can access the data in the parallel storage system by calling this interface. The specific access interface varies with different parallel storage systems. In order to support the coexistence of multiple high-availability mechanisms, the client framework provides the operation of loading and unloading high-availability mechanism modules, and can call corresponding functions according to the high-availability mechanism type of the logical data accessed by the client to complete data access. In specific operations, the customer specifies the required high-availability mechanism for the logical data, and the user's access function interface will load the high-availability mechanism module according to the configuration information and obtain a pointer to a specific function related to the mechanism, and call the function to complete the specific operation . Each high availability mechanism provides a set of client access functions corresponding to the interface functions provided by the client framework. With the support of these functions, the client completes related operations such as data redundancy, failover, and failover unique to each high-availability mechanism.

The data synchronization framework provides data synchronization operations between mutually redundant data in the fault recovery phase under the high-availability mechanism. The data synchronization framework can support the coexistence of data synchronization threads of multiple high-availability mechanisms. In actual implementation, it is a resident program that exists in every data node. When the resident program is started, all modules are loaded according to the configuration information, but threads are not generated, but specific threads are created when a synchronization request is received. There are two types of data synchronization threads: the data synchronization server thread as a synchronization data provider, and the data synchronization client thread as a synchronization data requester. The data synchronization framework can distribute the requests of the data synchronization client thread to the corresponding data synchronization server thread according to the high availability mechanism type and the synchronization thread type of the data synchronization. As a monitored entity of the state detection and control framework, the data synchronization thread has multiple running states. The external control of its state is completed through the state detection and monitoring framework on the node where it resides.

The system management module provides a friendly interface for managing the parallel storage system, which implements system configuration, system monitoring, and system control functions. The system configuration includes the configuration for the parallel storage system itself, including the system scale, node type, node information configuration, and also includes the configuration for the high-availability mechanism module. These configurations include: the number of supported high-availability mechanisms, and the corresponding high-availability The basic information of the available mechanism modules, and the specific configuration information of each high-availability mechanism. System monitoring realizes the monitoring of the overall information of the system and the operation information of each node, and also includes the detection of entity information and entity status corresponding to all high-availability mechanism modules, providing system administrators with a complete system view. The system control function realizes real-time control of all nodes in the system and state control of entities corresponding to all high-availability mechanisms. Administrators apply these controls to manually enable and disable data services and data synchronization operations within the system. During specific implementation, all service nodes, including metadata service nodes and data service nodes, have corresponding management agent modules, and the system management module communicates with these agents to obtain the required information on the corresponding nodes.

Different high-availability mechanism modules are implemented as different plug-ins, and the plug-ins implement the functional interfaces of the other six parts. Other frameworks in the architecture choose to call the interface implemented in the corresponding availability module according to the type of high availability mechanism used by the logical data, and cooperate with each other to complete data access, fault recovery and system management.

To access a piece of logical data, the user first accesses the metadata service framework through a function in the client framework to obtain the metadata information of the logical data, including the type of high-availability mechanism used by the logical data. If the logical data is created, The type of high-availability mechanism to be used for this segment of logical data should be indicated in the created operation. After the metadata information is obtained, it is saved in the client program. In the subsequent data access, the client framework calls the function corresponding to the high availability mechanism according to the type of high availability mechanism to access the data service framework on the data node. Through network communication, the data The service framework gets the request, learns the high-availability mechanism type of the logical data according to the logical data metadata information attached to the request, and then distributes the data request to the data service thread of the high-availability mechanism corresponding to the type, and the data service thread Process the request and return a response to the client, completing the data request. If a failure occurs when the data request corresponds to the data service framework of the data node, and the data service framework or data service thread of a certain data data node cannot be accessed normally, the client function will send a status confirmation request to access the status detection and control framework of the node, Confirm the current status of the data service thread. After receiving the status confirmation request, the status detection and control framework calls the status query function of the entity to confirm the current status of the entity according to the high availability mechanism type of the requested entity and the identifier of the entity. At the same time, for all related entities of the entity, call the status query function of the related entity to communicate with the status monitoring framework on other nodes to obtain its current status. When a previous item of the table matches all current states, the status of the local entity is set to the state specified in the latter item according to the latter item of the state conversion entry. If there are no matching entries, do nothing. The transformed entity state is then returned to the client. The client judges whether it can be accessed according to the state of the latest data service thread. If the entity can no longer be accessed, it can select other nodes to access according to the high-availability mechanism used, and repeat the above process until it gets a correct response or returns error. The above entity state requests are distinguished according to the initiator of the request. If the state is exchanged between state monitoring frameworks, the entity state before the state transition is returned, and the other initiators return the entity state after the state transition.

In the data storage process, the usual high availability mechanism will generate some data redundancy to ensure the availability of the storage system when a node failure occurs. Operations that generate data redundancy can be implemented in the data service thread or in the client function. When the data service thread is running normally, the redundant data is normally written into the storage medium of the data node. If the node fails, the data to be written to the faulty node cannot be written into the storage medium. Changes of redundant data related to the data of the faulty node are recorded on other related data nodes. Therefore, the data service framework sets a modification log related to logical data, and each data service thread sets a modification log file to record the modification of related data during the failure stage of other nodes. In order to provide the basis for data synchronization when the failed node recovers.

When there is a faulty node, the storage system automatically enters the faulty operation stage and automatically records the modification log. The system administrator can know that the node is faulty through the system management module. After manual intervention, if the node returns to normal, before the system fails After the data is not lost and added to the system in the initial state, the system administrator sends a request to start data synchronization to the status detection and control framework of the faulty node through the system control function in the system management module, and the request is forwarded to the node’s On the data synchronization framework, the data synchronization framework opens all relevant data synchronization client threads according to the type of high availability mechanism. Then the state detection and control framework sends a request to open the data synchronization server thread to other related nodes. After the data synchronization server thread is opened, it first analyzes the recorded modification log, reorganizes the log, generates a synchronization list, and waits for the access of the data synchronization client thread. After the data synchronization client thread is started, it directly requests data synchronization items from the data synchronization server thread of the relevant node, and processes the requested synchronization items one by one. When the data synchronization server thread finds that the number of synchronizations in the current log is less than a threshold , send a request to the state detection and control framework to block the data service thread. After being confirmed, the data service thread stops serving. In a short period of time, the data synchronization client can quickly synchronize all the data, and the data synchronization server When the synchronization linked list in the thread is empty, the client and server of the data synchronization thread send notifications of synchronization completion to their respective state detection and control frameworks, and the state detection and control framework adjusts the state of the data service program to normal, and Terminates the data synchronization thread. After the high-availability mechanisms on all failed nodes have completed the above operations, the data synchronization operation of the failed nodes is completed.

Claims (10)

1. A method for constructing a multi-availability mechanism coexistence architecture of a parallel storage system, characterized in that: the architecture is composed of the following seven parts, namely: state detection and control framework, data service framework, metadata service framework, and data synchronization framework , client framework, system management framework, high-availability mechanism module, the status detection and control framework is responsible for detecting and controlling the status of all entities on the node, and the data service framework is responsible for creating specific data service threads, distributing requests to data service threads and To complete the data redundancy and service takeover functions required by a specific high-availability mechanism, the metadata service framework calls different functions to complete metadata operations according to the high-availability mechanism of logical data, and the data synchronization framework supports data from multiple high-availability mechanisms Synchronization threads coexist to complete the data synchronization operation between mutually redundant data. The client framework provides a set of functions for users to access the parallel storage system, supports multiple high-availability mechanism modules, and calls the corresponding high-availability mechanism according to the requested high-availability mechanism type. Available mechanism functions, the system management framework provides an interface for system configuration, system monitoring, and system control functions, and the high-availability mechanism module is used as a plug-in to realize the functional interfaces of the other 6 parts. The workflow of the whole architecture is as follows: a. When a user initiates read and write access to logical data, he first sends a request to the metadata service framework through a function in the client framework to obtain the metadata information of the logical data. High availability mechanism type; b. Then, the client framework calls the interface function of the client framework implemented in the corresponding high availability mechanism module according to the high availability mechanism type of the logical data. This function completes the read and write operations by sending an access request to the data service framework on the data node ; c. The data service framework distributes the data request to the corresponding data service thread according to the high-availability mechanism type of the logical data attached to the access request, and the data service thread processes the request and returns a response to the client framework to complete the data request The response operation; d. If the data service framework or data service thread on a data node cannot be accessed, the client framework sends a status confirmation request to the status detection and control framework of the data node to confirm the current status of the data service thread, and the status detection After receiving the state confirmation request, the control framework calls the corresponding state query function to confirm the current state of the queried entity, and at the same time, calls the relevant state query function to communicate with the state monitoring framework on the node where the entity's related entity is located to obtain its current state , according to all obtained states, query the state transition table in the configuration information of the high-availability mechanism. When a previous item in the state transition table matches all the current states, set the state of the local entity as the subsequent state of the state transition entry The state specified in the item, if there is no matching entry, do nothing, and then return the converted entity state to the client; e. When a data node fails, and the data service threads on other data nodes related to the data service thread of the node are running normally, the data service framework of the relevant data node records the redundant data related to the data of the failed node the modification log of f. After a fault occurs, the system administrator can know that a data service thread has failed through the system management framework. After manual intervention, the system administrator starts the data synchronization process through the system management framework. The synchronization function in the high-availability mechanism module starts the data synchronization thread, accesses the data synchronization framework of other related nodes, and synchronizes the data of the faulty node according to the modification log. After the data synchronization is completed, the state detection and control framework on the data node is notified to adjust The state of the data service thread. 2. A method for constructing a multi-availability mechanism coexistence architecture of a parallel storage system according to claim 1, characterized in that: the parallel storage system is composed of a plurality of nodes, and the nodes are interconnected through a network, and the nodes are connected according to their functions It can be divided into four types: metadata nodes, data nodes, client nodes, and system management nodes. Metadata nodes store the description information of logical data and respond to access requests for this information. The file data itself is divided into logical data blocks And stored on multiple data nodes, the client node is the user of logical data, and the management node provides system configuration and management functions for system administrators. 3. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: the state detection and control framework is on each data node and metadata node of the parallel storage system There is a state detection and control framework, which is responsible for detecting and controlling the state of all entities on this node. If it is necessary to obtain the state of entities on other nodes, the state detection and control framework will communicate with the state detection and control framework on other nodes. The state of the required entity is obtained through the state detection and control framework on other nodes. Entities are all service programs related to high availability mechanisms, including data service threads and data synchronization threads in each high availability mechanism. Parallel storage systems require real-time Detect the running conditions of these entities to obtain the status of each entity. The status of the entity is different due to the different types of high availability mechanisms, but at least it contains two states indicating whether it is active or stopped. 4. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: the data service framework creates a data service thread when starting, loads a specific data service function, Then, according to the high-availability mechanism type of the logical data requested by the client, the access request for the logical data is distributed to the corresponding data service thread, and the thread responds to the request and returns the data required by the client. At the same time, the data service thread To complete the data redundancy and service takeover functions required by a specific high-availability mechanism, the data service framework supports the coexistence of multiple high-availability mechanism modules, and can dynamically load and unload various high-availability mechanism modules. 5. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1 or 3, characterized in that: for the data service thread, each high-availability mechanism has one or more corresponding data Service thread, each data service thread is an entity for the state detection and control framework. The entity state includes active state, backup state, synchronization state, and stop state, which represent different operating conditions, data service thread and data service framework Provide the access interface of the entity state, only allow the state detection and control framework access of the same node, and the entity can also change its state autonomously. 6. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: the metadata service framework supports multiple high-availability mechanisms, and according to the high-availability mechanism of logical data Different interface functions are called to complete metadata operations. Metadata operations can be dynamically loaded and unloaded in different high-availability mechanism modules according to different high-availability mechanisms. 7. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: the data synchronization framework can support the coexistence of multiple high-availability mechanism data synchronization threads, providing a high-availability mechanism Under fault recovery, the data synchronization operation between mutually redundant data, the data synchronization thread is divided into two types: the data synchronization server thread as the synchronization data provider and the data synchronization client thread as the synchronization data request end. The data synchronization framework is based on The high-availability mechanism type and synchronization request thread type for data synchronization to distribute data synchronization requests to the corresponding data synchronization service threads. 8. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: the client framework provides a set of function interfaces for users to access the parallel storage system, and users call these interfaces Access the data in the parallel storage system. At the same time, the client framework supports the coexistence of multiple high-availability mechanisms, provides high-availability mechanism module loading and unloading operations, and calls the corresponding function according to the high-availability mechanism type of the logical data accessed by the client to complete the data. Access, each high-availability mechanism provides a set of client access functions, which correspond to the function interface provided by the client framework. With the support of these functions, the client completes the unique data redundancy and failure of each high-availability mechanism. Switchover, failback operations. 9. A kind of method for constructing parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: said system management framework provides a friendly interface for managing parallel storage systems, and this interface realizes system configuration, System monitoring and system control functions. The system configuration includes the configuration for the parallel storage system itself, including the system scale, node type, node information configuration, and also includes the configuration for the high-availability mechanism module. The overall information of the system is realized in the system monitoring. And the monitoring of the operation information of each node, including the detection of the entity information and entity status corresponding to all high-availability mechanism modules, providing the system administrator with a complete system view. The system control function realizes real-time monitoring of all nodes in the system Control, and the state control of entities corresponding to all high-availability mechanisms, administrators can manually enable and disable data services and data synchronization operations in the system by applying these control functions. 10. A method for constructing a parallel storage system multi-availability mechanism coexistence architecture according to claim 1, characterized in that: said high-availability mechanism modules are implemented as different plug-ins, and the plug-ins implement the functions of the other six frameworks Interface, these frameworks choose to call the interface function implemented in the corresponding availability module according to the type of high availability mechanism used by the logical data to complete data access, fault recovery and system management.

Images