CN112751693B - Data processing method and device of distributed storage system and electronic equipment - Google Patents

Abstract

The invention relates to a data processing method and device of a distributed storage system and electronic equipment. The method comprises the following steps: respectively sending a first request for acquiring data copy information to each storage server; and updating mapping data for reflecting the corresponding relation between the data copy information and the storage servers according to the data copy information returned by each storage server in response to the first request.

Description

Data processing method, device and electronic device of distributed storage system

technical field

The present invention relates to the technical field of distributed storage, and more particularly, to a data processing method of a distributed storage system, a data processing device of a distributed storage system, an electronic device and a distributed storage system.

Background technique

Distributed storage is a storage solution in which data is distributed and stored on multiple independent devices. The distributed network storage system adopts an extensible system structure and uses multiple storage servers to share the storage load. It not only improves the reliability, availability and access efficiency of the system, but also is easy to expand.

In a distributed storage system, a dedicated control server usually coordinates the storage of data on multiple data servers. The control server stores metadata for describing data attributes, which can realize functions such as storage location records, historical data records, and resource search. In order to improve the reliability of the system, the number of control servers is usually more than one, the master control server among them provides control services, and the slave control server acts as a backup.

The master control server and the slave control server can maintain the consistency of metadata by means of data synchronization. However, if the main control server crashes during the data synchronization process, some metadata update information will be lost, affecting the operation of the distributed storage system.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a new technical solution for data processing in a distributed storage system.

According to a first aspect of the present invention, a data processing method for a distributed storage system is provided, which is implemented by a slave control server of the distributed storage system, including:

Send a first request for obtaining data copy information to each storage server respectively;

According to the data copy information returned by each of the storage servers in response to the first request, the mapping data used to reflect the corresponding relationship between the data copy information and the storage server is updated.

Optionally, the method further includes:

Detecting whether the main control server is in an unavailable state;

The sending a request for obtaining data copy information to each storage server respectively includes:

In the case that the master control server is in an unavailable state, a first request for acquiring data copy information is respectively sent to each storage server.

Optionally, the method further includes:

Based on the updated mapping data, the control service is provided for the distributed storage system in place of the main control server.

Optionally, the sending the first request for obtaining the data copy information to each storage server, further comprising:

According to the set time, a first request for obtaining data copy information is periodically sent to each storage server.

Optionally, according to the data copy information returned by each of the storage servers in response to the first request, updating the mapping data used to reflect the correspondence between the data copy information and the storage server includes:

According to the data copy information returned by each of the storage servers in response to the first request, the current mapping data is modified to obtain the updated mapping data.

According to a second aspect of the present invention, a data processing method for a distributed storage system is provided, which is implemented by a storage server in the distributed storage system, including:

receiving a first request for obtaining data copy information sent from the control server;

In response to the first request, the self-stored data copy information is sent to the slave control server.

According to a third aspect of the present invention, a data processing device of a distributed storage system is provided, which is arranged on a slave control server of the distributed storage system, including:

a request sending module, configured to send a first request for obtaining data copy information to each storage server respectively;

A data update module, configured to update the mapping data used to reflect the corresponding relationship between the data copy information and the storage server according to the data copy information returned by each of the storage servers in response to the first request.

Optionally, the device further includes a detection module, and the detection module is used for:

Detecting whether the main control server is in an unavailable state;

When the request sending module sends a request for obtaining data copy information to each storage server respectively, it is used for:

In the case that the master control server is in an unavailable state, a first request for acquiring data copy information is respectively sent to each storage server.

Optionally, the apparatus further includes a service providing module, and the service providing module is used for:

Based on the updated mapping data, the control service is provided for the distributed storage system in place of the main control server.

Optionally, when the request sending module sends the first request for obtaining the data copy information to each storage server, it is further configured to:

According to the set time, a first request for obtaining data copy information is periodically sent to each storage server.

Optionally, when the data update module updates the mapping data used to reflect the corresponding relationship between the data copy information and the storage server according to the data copy information returned by each of the storage servers in response to the first request , for:

According to the data copy information returned by each of the storage servers in response to the first request, the current mapping data is modified to obtain the updated mapping data.

According to a fourth aspect of the present invention, a data processing device of a distributed storage system is provided, and the storage server disposed in the distributed storage system includes:

a request receiving module, configured to receive a first request for obtaining data copy information sent from the control server;

A data sending module, configured to send the self-stored data copy information to the slave control server in response to the first request.

According to a fifth aspect of the present invention, there is provided an electronic device comprising a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions In order to realize the data processing method of the distributed storage system described in the first aspect or the second aspect of the present invention.

According to a sixth aspect of the present invention, a distributed storage system is provided, comprising a master control server, a slave control server implementing the method of the first aspect of the present invention, and a storage server implementing the method of the second aspect of the present invention, wherein , the storage server is respectively connected to the master control server and the slave control server.

The data processing method of a distributed storage system in an embodiment of the present invention does not require metadata synchronization between the master and slave control servers, and avoids failure to synchronize the master and slave control servers due to metadata loss caused by the failure of the master control server during the synchronization process The metadata in the system is beneficial to improve the reliability of the distributed storage system.

Other features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

FIG. 1 shows a schematic diagram of a hardware configuration of a distributed storage system that can be used to implement embodiments of the present invention.

FIG. 2 shows a schematic structural diagram of a server that can be used to implement an embodiment of the present invention.

FIG. 3 shows a flowchart of a data processing method of a distributed storage system according to an embodiment of the present invention.

FIG. 4 shows a flowchart of a specific example of the implementation of the data processing method of the distributed storage system according to the embodiment of the present invention.

Detailed ways

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, techniques, methods, and apparatus should be considered part of the specification.

In all examples shown and discussed herein, any specific values should be construed as illustrative only and not limiting. Accordingly, other instances of the exemplary embodiment may have different values.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further discussion in subsequent figures.

<Hardware configuration>

FIG. 1 shows a schematic structural diagram of a distributed storage system that can be used to implement an embodiment of the present invention.

As shown in FIG. 1 , the distributed storage system 100 includes a user agent server 1000 , a storage server 2001 , a storage server 2002 , a storage server 2003 , and a control server 3001 and a control server 3002 . Wherein, the number of storage servers and control servers may be multiple (more than two). In this embodiment of the present invention, the numbers of storage servers, control servers, and user agent servers are all exemplary, which are not limited in the present invention.

In this embodiment, the distributed storage system uses a data block (Block) as a basic unit to store target data. For example, the target file is decomposed into multiple data blocks, and multiple copies of each data block are stored in multiple independent storage servers. In one example, each data block corresponds to three data block replicas, one of which is the master replica, and the other two data block replicas are slave replicas. The storage server where the primary replica is located preferentially handles read and write requests for target files, and the storage server where the secondary replica is located plays a backup role.

The storage server 2001, the storage server 2002, and the storage server 2003 are used for storing target data, for example, storing copies of data blocks corresponding to the target data.

The control server 3001 and the control server 3002 are used to manage metadata, for example, to manage mapping data reflecting the corresponding relationship between the copy of the data block corresponding to the target file and the storage server. One of the control servers serves as the master control server, providing control services for the distributed storage system, and the other control server serves as the slave control server for backing up data in the master control server. In this embodiment, the control server 3001 is the master control server, and the control server 3002 is the slave control server. The roles of the master and slave control servers can be converted to each other. When the master control server fails, the slave control server can replace the master control server to provide control services for the distributed storage system.

The user agent server 1000 is configured to receive a data read/write request for target data sent by the client, and forward the data read/write request to the control server.

After receiving the data read/write request sent by the user agent server 1000, the main control server 3001 can query the storage server where the copy of the data block corresponding to the target data is located according to the metadata stored by itself, and return to the user agent server to process the data read/write Identification information of the requested storage server. The user agent server 1000 interacts with the corresponding storage server according to the identification information, and completes the read and write operations on the target data.

The user agent server 1000, the storage server 2001, the storage server 2002, the storage server 2003, the control server 3001, and the control server 3002 may communicate with each other through a wired network or a wireless network.

The user agent server 1000 , the storage server 2001 , the storage server 2002 , the storage server 2003 , the control server 3001 , and the control server 3002 all have the hardware configuration of the server 1100 shown in FIG. 2 . As shown in FIG. 2 , the server 1100 may include a processor 1110 , a memory 1120 , an interface device 1130 , a communication device 1140 , a display device 1150 and an input device 1160 . The processor 1110 may be, for example, a central processing unit (CPU) or the like. The memory 1120 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1130 includes, for example, a USB interface, a serial interface, and the like. The communication device 1140 is capable of wired or wireless communication, for example. The display device 1150 is, for example, a liquid crystal display. The input device 1160 may include, for example, a touch screen, a keyboard, and the like.

Applied to the embodiments of this specification, the memory 1120 of the server 1100 is used to store instructions for controlling the processor 1110 to operate to support implementing the method according to any embodiment of this specification. A skilled person can design instructions according to the solutions disclosed in this specification. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.

Those skilled in the art should understand that although a plurality of devices of the server 1100 are shown in FIG. 2 , the server 1100 in this embodiment of the present specification may only involve some of the devices, for example, only the processor 1110 , the memory 1120 and the Communication device 1140.

The distributed storage system 100 shown in FIG. 1 is illustrative only and is in no way intended to limit this specification, its application, or uses.

<Method Example>

This embodiment provides a data processing method for a distributed storage system, and the method is implemented by, for example, the slave control server 3002 in FIG. 1 . As shown in FIG. 2, the method includes the following steps S1100-S1200.

In step S1100, a first request for obtaining data copy information is respectively sent to each storage server.

In this embodiment, the control server is used to manage metadata, for example, to manage mapping data reflecting the correspondence between the copy of the data block corresponding to the target file and the storage server. Among them, the master control server provides control services for the distributed storage system, and the slave control server plays a backup role.

In this embodiment, the distributed storage system uses data blocks as basic units to store target data. For example, the target file is decomposed into multiple data blocks, and multiple copies of each data block are stored in multiple independent storage servers. In an example, each data block corresponds to three data block copies, one of which is the master copy, and the other two data block copies are slave copies. For example, the three copies are divided into A, B, and C. They are stored in the storage servers numbered 1, 2, and 3 respectively.

In this embodiment, a first request for obtaining data copy information is sent from the control server to each storage server, so as to obtain the information of the data block copy stored in each storage server. In an implementation manner of this step, the storage server where the primary copy is located may respond to the first request preferentially.

In one embodiment, the method of the embodiment of the present invention may further include the step of detecting whether the main control server is in an unavailable state. Exemplarily, the slave control server may send a status query request to the master control server (for example, through heartbeat interaction), and if the master control server does not respond, it may be determined that the master control server is currently in an unavailable state. On this basis, step S1100 further includes: in the case that the main control server is in an unavailable state, sending a first request for acquiring data copy information to each storage server respectively.

In this embodiment, the main control server is in an unavailable state, for example, the main control server crashes, and the communication between the main control server and the outside world is interrupted. It is easy to understand that the main control server is in an unavailable state, which means that the main control server cannot normally provide control services for the distributed storage system.

In one embodiment, the step of detecting from the control server whether the master control server is in an unavailable state may include: receiving a notification sent by a monitoring device (eg, zookeeper) about whether the master control server is in an unavailable state; according to the notification, determining Whether the master control server is in an unavailable state. In one example, the monitoring device obtains the active state of the master control server by receiving heartbeat packets periodically sent by the master control server. Exemplarily, the monitoring device (for example, zookeeper) may periodically perform heartbeat with the control server (master control server, slave control server), and if the heartbeat from the master control server is not received within a preset time period, the master control server may be determined. is in an unavailable state, which in turn sends a notification to the slave control server. The slave control server may determine that the master control server is in an unavailable state according to the notification of the monitoring device.

In this embodiment, the monitoring device is used to monitor the activity state of the main control server. The monitoring device may be an additional physical device, or may be the slave control server itself, for example, the slave control server monitors the active state of the master control server through its own special process.

In this embodiment, the monitoring device may send a corresponding notification to the slave control server when detecting that the master control server changes from an available state to an unavailable state. The slave control server determines that the master control server is in an unavailable state according to the notification.

In another embodiment, the step of detecting whether the master control server is in an unavailable state from the slave control server includes: the slave control server may perform a heartbeat to the master control server, and if no master control server is received within a preset time period (for example, 1 min) The heartbeat of the control server can determine that the main control server is in an unavailable state.

In step S1200, according to the data copy information returned by each storage server in response to the first request, the mapping data used to reflect the correspondence between the data copy information and the storage server is updated.

In this embodiment, the slave control server stores mapping data reflecting the corresponding relationship between the data copy and the storage server.

In this embodiment, in response to the first request sent by the slave control server, each storage server sends the data copy information stored by itself to the slave control server. Received from the control server each storage server sends its own stored data copy information. And according to the received data copy information, a corresponding relationship (mapping relationship) between the data copy information and the storage server where it is located is established, and the mapping data (or corresponding relationship) table is updated. The mapping data table or the corresponding relationship table may include the corresponding relationship (or mapping data) between more than one piece of data copy information and the storage server where it is located.

By executing step S1200, the latest correspondence between the data copy and the storage server can be obtained from the control server.

The data processing method of the distributed storage system in this embodiment does not require metadata synchronization between the master and slave control servers, and avoids failure to synchronize the master and slave control servers due to metadata loss caused by the failure of the master control server during the synchronization process. Metadata is beneficial to improve the reliability of distributed storage systems.

In one embodiment, the data processing method of the distributed storage system further includes: when it is determined that the master control server is in an unavailable state, the slave control server may replace the master control server to provide control for the distributed storage system based on the updated mapping data Serve. The main control server may be unavailable due to power failure, downtime, or unexpected restart. When the slave control server detects that the master control server is unavailable, after updating its stored mapping data, it replaces the master control server in time to provide control services for the distributed storage system. Ensure the normal operation of the distributed storage system and provide guarantee for the data storage security of the distributed storage system.

In this embodiment, the process for the main control server to provide control services for the distributed storage system is, for example: the main control server receives a data read request for the target data sent by the user agent server, and queries the corresponding target data according to the mapping data stored by itself. The storage server where the copy of the data block is located, and returns the identification information of the storage server that processes the data read and write request to the user agent server, so that the user agent server can interact with the corresponding storage server according to the identification information to complete the reading of the target data. write operation.

In this embodiment, the roles of the master and slave control servers can be converted to each other, and when the master control server is in an unavailable state, the slave control server can replace the master control server to provide control services for the distributed storage system.

In this embodiment, the slave control server can automatically replace the master control server to provide control services for the distributed storage system (ie, automatic switching), or can replace the master control server to provide control services for the distributed storage system in response to manual operations (ie, manual switching) ). In the embodiments of the present application, automatic switching is the preferred embodiment, so as to ensure the security and timeliness of data storage and avoid data loss.

In this embodiment, the slave control server provides a control service for the distributed storage system according to the updated mapping data, which can ensure the normal operation of the distributed storage system.

According to an embodiment of the present invention, the foregoing step S1100 may be implemented as periodically sending a first request (hereinafter referred to as a periodic request) for obtaining data copy information to each storage server according to a set time.

In one embodiment, the slave control server periodically sends a first request for obtaining data copy information to each storage server at a set time. In other embodiments, when the master control server is unavailable, the slave control server periodically sends the first request to each storage server according to a set time. Exemplarily, the set time is any length from 1s to 1min, for example, it can be 5s, 10s, 15s, 20s, 25s, etc., which is not limited in this application, and can be preset according to the experience value of technicians.

By regularly obtaining data copy information and regularly updating mapping data, the amount of data that needs to be processed for each update can be reduced, which is especially beneficial for the slave control server to quickly update the mapping data when the master control server is unavailable, improving the master-slave switching speed.

In one embodiment, the foregoing step S1100 may be implemented as, in the initial startup phase, sending a request for obtaining data copy information (hereinafter referred to as an initial request) to each storage server respectively.

In this embodiment, the slave control server sends a first request (which may be referred to as an initial request in the following description) to each storage server to obtain data copy information in the initial startup phase. It is easy to understand that the initial request is different from the request sent by the slave control server when the master control server is unavailable, and the request sent by the slave control server periodically.

Then, step S1200 is executed, that is, according to the data copy information returned by each storage server in response to the initial request, mapping data reflecting the corresponding relationship between the data copy and the storage server is established.

In this embodiment, in response to the initial request, each storage server sends the data copy information stored by itself to the slave control server. The data copy information sent by each storage server to be stored by itself is received from the control server, and the mapping data reflecting the corresponding relationship between the data copy and the storage server is established.

The mapping data reflecting the corresponding relationship between the data copy and the storage server is newly created from the control server in the initial startup phase, which can establish a basis for the subsequent updating of the mapping data.

In one embodiment, the process of updating the self-stored mapping data from the control server includes: modifying the self-stored current mapping data to obtain updated mapping data.

In this embodiment, the slave control service performs data update based on the current mapping data stored by itself, which can reduce the amount of data that needs to be processed for each update, which is beneficial to improve the update speed.

In one embodiment, the mapping data reflecting the corresponding relationship between the data copy and the storage server is stored in the memory of the slave control server. In this way, it is beneficial to improve the reading and writing speed of the mapped data, thereby improving the data updating speed.

In the foregoing method embodiments, the slave control server actively sends the first request to the storage server, when the master control server is in an unavailable state, sends the first request to the storage server, and at the initial startup stage of the slave control server (that is, when the operation is started) The implementation method and corresponding technical effect in three states of the first request are sent to the storage server. In the actual operation process of the distributed storage system, the full cycle may include the initial startup phase of the slave control server, the normal working phase of the slave control server, and the detection that the master control server is in an unavailable state, and the like. And, in different states in the whole cycle, the role of the slave control server in data processing may be different to some extent.

Another embodiment provided by the present invention will be described below based on the system states that may occur during the actual operation of the distributed storage system:

Stage (1): In the initial startup stage from the control server, a second request is sent to each storage server to obtain data copy information, and initial mapping data is created based on the data copy information obtained at this stage;

Stage (2): In the normal working stage of the slave control server, a third request can be sent periodically to obtain data copy information, which is used to update the mapping data regularly;

Stage (3): When the main control server is in an unavailable state, a fourth request is sent from the control server to obtain the current data copy information and update the mapping data, and then the main control server can be replaced based on the latest mapping data to provide services.

Among them, the first request, the second request, the third request and the fourth request in the present invention are to distinguish the read requests sent from the control server in different stages (or different states), but the first request, the second request and the second request The operation methods and implementation principles of the request, the third request, and the fourth request are the same. In other words, the first request is respectively expressed as the second request, the third request and the fourth request in different states of the control server.

In general, from the start of the control server, the mapping data stored in the control server can be continuously updated through the interaction with the storage server in its entire life cycle, and the above-mentioned embodiment is used in stages. Compared with the embodiment in which the mapping data is updated every time, the data to be updated is relatively less, so the data update speed is faster.

FIG. 4 shows a specific example of the implementation of the data processing method of the distributed storage system in this embodiment. As shown in FIG. 4 , the slave control server monitors the heartbeat of the master control server through its own monitoring process, so as to detect the active state of the master control server, that is, step S101 is performed. In addition, the slave control server also periodically sends a third request for obtaining data copy information to the storage server, and the storage server sends the data copy information stored by itself to the slave control server in response to the third request, that is, step S102 is performed. According to the data copy information sent by the storage server, the slave control server periodically updates the mapping data stored in its own memory reflecting the corresponding relationship between the data copy and the storage server, that is, step S103 is executed. Assuming that the master control server crashes during operation, and the slave control server may not be able to monitor the master control server within the preset time, it can be determined that the master control server is in an unavailable state, that is, step S104 is performed. After that, the control server sends a fourth request for acquiring replica information to each storage server respectively, that is, step S105 is executed. In response to the fourth request, each storage server sends the information of the data copy stored by itself to the slave control server, that is, step S106 is executed. According to the data copy information sent by the storage server, the slave control server corrects the data obtained by the latest regular update, and obtains the updated mapping data, that is, step S107 is executed. Finally, the slave control server automatically replaces the master control server to provide control services for the distributed storage system, that is, step S108 is performed.

This embodiment also provides another data processing method for a distributed storage system, and the method is implemented by, for example, any storage server shown in FIG. 1 . The method includes: receiving a request for acquiring data copy information sent from a control server; and in response to the request for acquiring data copy information, sending self-corresponding data copy information to the slave control server.

For the specific implementation of the method, reference may be made to the foregoing description of the data processing method of the distributed storage system, which will not be repeated here.

<Apparatus Example>

This embodiment provides a data processing device of a distributed storage system, which is applied to a slave control server of a distributed storage system, and includes a request sending module and a data updating module.

The request sending module is configured to send a request for obtaining data copy information to each storage server respectively.

The data update module is configured to update the mapping data used to reflect the corresponding relationship between the data copy information and the storage server according to the data copy information returned by each storage server in response to the first request.

Optionally, the data processing apparatus of the distributed storage system of the present invention may further include: a detection module: used to detect whether the main control server is in an unavailable state. Further, the request sending module is configured to send a first request for acquiring data copy information to each storage server respectively when the main control server is in an unavailable state.

Optionally, the apparatus further includes a service providing module, and the service providing module is configured to: replace the main control server with a control service for the distributed storage system based on the updated mapping data.

Optionally, the request sending module may also be configured to: periodically send a request for obtaining data copy information to each storage server according to a set time.

Optionally, the data updating module may be configured to modify the existing mapping data according to the data copy information returned by each storage server in response to the first request to obtain updated mapping data.

This embodiment also provides another data processing device of a distributed storage system, which is applied to a storage server in a distributed storage system, and includes a request receiving module and a data sending module.

The request receiving module is used for receiving the request for obtaining the data copy information sent from the control server.

The data sending module is used for sending the corresponding data copy information to the slave control server in response to the request for obtaining the data copy information.

<Example of electronic device>

This embodiment provides an electronic device, including a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the distributed storage system described in the method embodiments of the present invention. data processing method.

<System Example>

This embodiment provides a distributed storage system, including a master control server, a slave control server that implements the first method described in the embodiments of the present invention, and a storage server that implements the second method described in the method embodiments of the present invention, wherein , the storage server is respectively connected to the master control server and the slave control server.

<Computer-readable storage medium embodiment>

This embodiment provides a machine-readable storage medium. The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the method embodiments of the present invention. A data processing method for a distributed storage system is described.

The present invention may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

The computer program instructions for carrying out the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code written in any combination, programming languages including object-oriented programming languages - such as Smalltalk, C++, etc., and conventional procedural programming languages - such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executables for implementing the specified logical function(s) instruction. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Various embodiments of the present invention have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims (9)

1. A data processing method for a distributed storage system, implemented by a slave control server of the distributed storage system, comprising: In the initial startup phase, send a first request for obtaining data copy information to each storage server respectively; According to the set time, periodically send a first request for obtaining data copy information to each storage server; According to the data copy information returned by each of the storage servers in response to the first request, the mapping data used to reflect the corresponding relationship between the data copy information and the storage server is updated. 2. The method of claim 1, wherein the method further comprises: Detect whether the main control server is in an unavailable state; According to the set time, the request for obtaining data copy information is periodically sent to each storage server, including: In the case that the master control server is in an unavailable state, a first request for acquiring data copy information is respectively sent to each storage server. 3. The method of claim 2, wherein the method further comprises: Based on the updated mapping data, the control service is provided for the distributed storage system in place of the main control server. 4 . The method according to claim 1 , wherein, according to the data copy information returned by each of the storage servers in response to the first request, the update is used to reflect that the data copy information corresponds to the storage server. 5 . Mapping data for relationships, including: According to the data copy information returned by each of the storage servers in response to the first request, the current mapping data is modified to obtain the updated mapping data. 5. A data processing method for a distributed storage system, implemented by a storage server in the distributed storage system, comprising: In the initial startup phase, receiving the first request sent from the control server to obtain the data copy information; According to the set time, regularly receive the first request for obtaining the data copy information sent from the control server; In response to the first request, the self-stored data copy information is sent to the slave control server. 6. A data processing device of a distributed storage system, arranged on a slave control server of the distributed storage system, comprising: The request sending module is used to send the first request for obtaining the data copy information to each storage server respectively during the initial startup stage, and periodically send the first request for obtaining the data copy information to each storage server according to the set time; A data update module, configured to update the mapping data used to reflect the corresponding relationship between the data copy information and the storage server according to the data copy information returned by each of the storage servers in response to the first request. 7. A data processing device of a distributed storage system, arranged in a storage server in the distributed storage system, comprising: The request receiving module is used to receive, in the initial startup stage, the first request sent from the control server to obtain the data copy information, and periodically receive the first request sent from the control server to obtain the data copy information according to the set time; A data sending module, configured to send the self-stored data copy information to the slave control server in response to the first request. 8. An electronic device comprising a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement any of claims 1 to 4 The data processing method of any one of the distributed storage systems. 9. A distributed storage system, comprising a master control server, a slave control server implementing the method of any one of claims 1-4, and a storage server implementing the method of claim 5, wherein the storage servers are respectively Communication connection with the master control server and the slave control server.

Images