CN102646127A - Distributed file system copy selection method and device - Google Patents

Abstract

The invention provides a replica selection method and device for distributed file systems, and relates to the field of cloud computing, and the method and the device are used for solving the problem of lacking a mechanism for quickly selecting the latest replica to operate. The method comprises the following steps: according to the frequency of replica updating, updating the version number of a corresponding replica, wherein the more the frequency of replica updating is, the larger the version number corresponding to the replica is; and when a read operation request or a write operation request is received, returning a replica with the largest version number. According to the technical scheme provided by the invention, the method and the device provided by the invention are applicable to cloud computing systems, and an effect of quickly and efficiently obtaining the latest replica from a plurality of replicas is achieved.

Description

Distributed file system copy selection method and device

technical field

The invention relates to the field of cloud computing, in particular to a method for selecting a copy of a distributed file system.

Background technique

Cloud storage is a new concept extended and developed on the basis of cloud computing. It refers to the integration and utilization of a large number of different types of storage devices in the network through technologies such as cluster applications, network technologies, and distributed file systems. A unified storage pool is formed to provide external storage services.

As the core component of cloud storage, distributed file system has made great progress in recent years. How to select replicas in a distributed file system is a very important issue. In order to improve data reliability, data redundancy can be increased by establishing multiple copies of object files in the distributed file system. When performing file read or file write operations, a mechanism is required to quickly select the latest copy from many existing copies for operation, and such a mechanism is currently lacking.

Contents of the invention

The invention provides a method and device for selecting a copy of a distributed file system, which solves the problem of lack of a mechanism for quickly selecting the latest copy for operation.

A distributed file system copy selection method, comprising:

The version number of the corresponding copy is updated according to the number of copy updates, and the copy with more updates has a larger value of the corresponding version number;

When receiving a read operation request or a write operation request, return the copy with the largest version number.

Preferably, the above-mentioned distributed file system copy selection method also includes:

A copy link list is maintained, the copy link list includes at least one copy, and the storage of each copy in the copy link list corresponds to a version number.

Preferably, said updating the version number of the corresponding copy according to the number of copy updates includes:

Increment the value of the version number of the replica being written by 1; and,

Periodically check the version number of each copy in the copy linked list, update other copies in the copy linked list according to the copy with the largest version number, and modify the version numbers of other copies to the largest version number value; and,

When deleting a copy in the copy linked list, the content of the copy is cleared, and the version number corresponding to the position of the copy in the copy linked list is set to 0.

Preferably, the above-mentioned distributed file system copy selection method also includes:

When creating a copy, create a version number corresponding to the copy, and set an initial value for the version number.

Preferably, the initial value of the version number is 1.

Preferably, when a read operation request or a write operation request is received, the copy with the largest version number is returned.

The present invention also provides a distributed file system copy selection device, including:

The version number update module is used to update the version number of the corresponding copy according to the number of times the copy is updated, and the copy with more update times has a larger value of the corresponding version number;

The copy operation module is configured to return the copy with the largest version number when receiving a read operation request or a write operation request.

Preferably, the above-mentioned distributed file system copy selection device also includes:

The link list maintenance module is used to maintain a copy link list, the copy link list includes at least one copy, and the storage of each copy in the copy link list corresponds to a version number.

Preferably, the version number update module includes:

The first update unit is used to increase the value of the version number of the copy to be written by 1; and,

The second update unit is used to periodically check the version number of each copy in the copy linked list, update other copies in the copy linked list according to the copy with the largest value of the version number, and modify the version numbers of other copies to the maximum version number value; and,

The third updating unit is configured to clear the content of the copy when deleting the copy in the copy linked list, and set the version number corresponding to the position of the copy in the copy linked list to 0.

The present invention provides a method and device for selecting a copy of a distributed file system. The version number of the corresponding copy is updated according to the number of times the copy is updated. When an operation request is made, the copy with the largest version number value is returned, which realizes the fast and efficient acquisition of the latest copy from multiple copies, and solves the problem of lack of a mechanism for quickly selecting the latest copy for operation.

Description of drawings

FIG. 1 is a flowchart of a distributed file system copy selection method provided by Embodiment 1 of the present invention;

FIG. 2 is a schematic structural diagram of an apparatus for selecting a copy of a distributed file system according to Embodiment 2 of the present invention.

Detailed ways

In order to improve data reliability, data redundancy can be increased by establishing multiple copies of object files in the distributed file system. When performing file read or file write operations, a mechanism is required to quickly select the latest copy from many existing copies for operation, and such a mechanism is currently lacking. The copy selection method of the present invention achieves the goal of copy selection based on the detection and update of the version number.

Embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

First, Embodiment 1 of the present invention will be described with reference to the accompanying drawings.

The embodiment of the present invention provides a method for selecting a copy of a distributed file system, using the method to update the version of the copy and selecting the latest version of the copy during operation as shown in Figure 1, including:

Step 101, maintaining a copy link list, the copy link list includes at least one copy, and the storage of each copy in the copy link list corresponds to a version number;

In this step, the value of the version number (Version) of the copy is initialized to 1 when the copy is created, indicating that the copy is a new copy.

Step 102, update the version number of the corresponding copy according to the number of times the copy has been updated, and the copy with more update times has a larger value of the corresponding version number;

In this step, specifically, the following four situations are included:

1. When there is a file read operation, the corresponding block chunk is found through the given index, and a linked list of copies is maintained under the chunk. Scan the Versions of these copies in turn, find out the copy with the largest Version value (there may be more than one), and send its corresponding storage node ip and port to the client for it to read the file.

2. When there is a file write operation, if the corresponding block chunk is found through the given index, select the version with the largest version from the copy chain below the chunk and return it to the client for writing, and add one to their Version at the same time, indicating This copy was updated.

3. Periodically randomly check the copy list under a certain chunk, and if the Version of each copy is found to be inconsistent, use the copy with the largest Version as the source to update other "old" copies, and then reset the Version of these copies.

4. When a copy needs to be deleted (such as resetting the number of copies), setting Version to 0 means that the copy is unavailable. Later in the cycle detection, it is found that the copy with a Version value of 0 is actually deleted.

Step 103: When a read operation request or a write operation request is received, return the copy with the largest version number.

Embodiment 2 of the present invention will be described below with reference to the accompanying drawings.

An embodiment of the present invention provides a device for selecting a copy of a distributed file system, the structure of which is shown in Figure 2, including:

A version number update module 201, configured to update the version number of the corresponding copy according to the number of times the copy has been updated, and the version number corresponding to the copy with more update times is greater;

The copy operation module 202 is configured to return the copy with the largest version number when receiving a read operation request or a write operation request.

Preferably, the above-mentioned distributed file system copy selection device also includes:

The link list maintenance module 203 is configured to maintain a copy link list, the copy link list includes at least one copy, and the storage of each copy in the copy link list corresponds to a version number.

Preferably, the version number update module 201 includes:

The first update unit is used to increase the value of the version number of the copy to be written by 1; and,

The second update unit is used to periodically check the version number of each copy in the copy linked list, update other copies in the copy linked list according to the copy with the largest value of the version number, and modify the version numbers of other copies to the maximum version number value; and,

The third updating unit is configured to clear the content of the copy when deleting the copy in the copy linked list, and set the version number corresponding to the position of the copy in the copy linked list to 0.

The present invention provides a method and device for selecting a copy of a distributed file system. The version number of the corresponding copy is updated according to the number of times the copy is updated. When an operation request is made, the copy with the largest version number value is returned, which realizes the fast and efficient acquisition of the latest copy from multiple copies, and solves the problem of lack of a mechanism for quickly selecting the latest copy for operation.

Select the latest copy from many copies through the version number of the copy to operate, and then update the version number of the copy. Other copies maintain data consistency according to the version number. The change of the value of the version number reflects the number of updates of the copy, and the Version value The largest replica is the most recently updated replica, and all other replicas are based on it to update themselves. By introducing the concept of version numbers, it is easier to perform data update control and copy selection operations in the case of multiple copies, reducing system complexity and response time.

Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be implemented using a computer program flow, the computer program can be stored in a computer-readable storage medium, and the computer program can be run on a corresponding hardware platform (such as system, device, device, device, etc.), and when executed, includes one or a combination of the steps of the method embodiment.

Optionally, all or part of the steps in the above embodiments can also be implemented using integrated circuits, and these steps can be fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module accomplish. As such, the present invention is not limited to any specific combination of hardware and software.

The devices/functional modules/functional units in the above embodiments can be realized by general-purpose computing devices, and they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices.

When each device/functional module/functional unit in the above-mentioned embodiments is realized in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like.

Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present invention, and all should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope described in the claims.

Claims (9)

1. A method for selecting a copy of a distributed file system, comprising: The version number of the corresponding copy is updated according to the number of copy updates, and the copy with more updates has a larger value of the corresponding version number; When receiving a read operation request or a write operation request, return the copy with the largest version number. 2. The distributed file system copy selection method according to claim 1, wherein the method further comprises: A copy link list is maintained, the copy link list includes at least one copy, and the storage of each copy in the copy link list corresponds to a version number. 3. The distributed file system copy selection method according to claim 2, wherein said updating the version number of the corresponding copy according to the copy update times comprises: Increment the value of the version number of the replica being written by 1; and, Periodically check the version number of each copy in the copy linked list, update other copies in the copy linked list according to the copy with the largest version number, and modify the version numbers of other copies to the largest version number value; and, When deleting a copy in the copy linked list, the content of the copy is cleared, and the version number corresponding to the position of the copy in the copy linked list is set to 0. 4. The distributed file system copy selection method according to claim 1 or 3, wherein the method further comprises: When creating a copy, create a version number corresponding to the copy, and set an initial value for the version number. 5. The method for selecting a copy of a distributed file system according to claim 4, wherein the initial value of the version number is 1. 6. The method for selecting a copy of a distributed file system according to claim 1, wherein when a read operation request or a write operation request is received, the copy with the largest version number value is returned. 7. A distributed file system copy selection device, characterized in that it comprises: The version number update module is used to update the version number of the corresponding copy according to the number of times the copy is updated, and the copy with more update times has a larger value of the corresponding version number; The copy operation module is configured to return the copy with the largest version number when receiving a read operation request or a write operation request. 8. The distributed file system copy selection device according to claim 7, wherein the device further comprises: The link list maintenance module is used to maintain a copy link list, the copy link list includes at least one copy, and the storage of each copy in the copy link list corresponds to a version number. 9. The distributed file system copy selection device according to claim 7, wherein the version number updating module comprises: The first update unit is used to increase the value of the version number of the copy to be written by 1; and, The second update unit is used to periodically check the version number of each copy in the copy linked list, update other copies in the copy linked list according to the copy with the largest value of the version number, and modify the version numbers of other copies to the maximum version number value; and, The third updating unit is configured to clear the content of the copy when deleting the copy in the copy linked list, and set the version number corresponding to the position of the copy in the copy linked list to 0.

Images