CN115687250A - A storage method, device, system and computer storage medium - Google Patents

Abstract

The invention discloses a storage method, which is applied to an access device. The method includes: when receiving a storage request for a first file, assigning an identification code to the first file, using the existing There is a parent directory and an identification code, generate the index number of the first file, store the association between the name of the first file and the index number of the first file, and send the first file and the index number of the first file to Index device. The embodiment of the invention also discloses a device, a system and a computer storage medium at the same time, which improves the reliability of file storage and further improves the storage efficiency of file storage.

Description

A storage method, device, system and computer storage medium

technical field

The invention relates to computer file storage technology, in particular to a storage method, device, system and computer storage medium.

Background technique

A distributed file system (DFS, Distributed File System) can be used for data storage and management. At present, the common distributed file system is usually based on the stand-alone file system, and realizes the distributed storage of data across servers by adding additional modules, and manages the data by sharing the same namespace.

Among various stand-alone file systems, compared with other stand-alone file systems, the dynamic file system (ZFS, Zettabyte File System) can span the physical location of data due to the combination of file system and device management.

However, in the distributed file system, ZFS cannot break through the limitations of its own stand-alone file system, and still has many shortcomings in terms of service availability. For example, it is often difficult to provide service when a ZFS server is down or fails.

Contents of the invention

In view of this, the present invention provides a storage method, device, system and computer storage medium to solve the technical problem in the prior art that it is usually difficult to provide services when the ZFS server is down or faulty.

Technical scheme of the present invention is realized like this:

In the first aspect, an embodiment of the present invention provides a storage method, and the method is applied to an access device, including:

assigning an identification code to the first file in case a storage request for the first file is received;

generating an index number of the first file by using the existing parent directory of the first file and the identification code;

storing the association relationship between the name of the first file and the index number of the first file;

sending the first file and the index number of the first file to the indexing device;

Wherein, the indexing device is configured to, after determining the identifier of the storage node used to store the first file in the storage device, compare the index number of the first file with the identifier of the storage node of the first file The corresponding relationship is stored in the first device and the second device of the indexing device.

In a second aspect, an embodiment of the present invention provides a storage method, and the method is applied to an index device, including:

Receiving the first file and the index number of the first file sent by the access device; wherein, the index number of the first file is an existing parent directory of the first file used by the access device, and the generating an identification code assigned by the access device to the first file;

determining an identifier of a storage node used to store the first file in the storage device;

The corresponding relationship between the index number of the first file and the identifier of the storage node of the first file is stored in the first device and the second device of the indexing device.

In a third aspect, an embodiment of the present invention provides an access device, including:

An allocation module, configured to allocate an identification code to the first file when a storage request for the first file is received;

A generating module, configured to generate an index number of the first file by using the existing parent directory of the first file and the identification code;

A first storage module, configured to store the association relationship between the name of the first file and the index number of the first file;

a sending module, configured to send the first file and the index number of the first file to an indexing device;

Wherein, the indexing device is configured to, after determining the identifier of the storage node used to store the first file in the storage device, compare the index number of the first file with the identifier of the storage node of the first file The corresponding relationship is stored in the first device and the second device of the indexing device.

In a fourth aspect, an embodiment of the present invention provides an indexing device, including:

The receiving module is configured to receive the first file and the index number of the first file sent from the access device; wherein, the index number of the first file is obtained by the access device using the existing generating a parent directory and an identification code assigned by the access device to the first file;

A determining module, configured to determine an identifier of a storage node used to store the first file in the storage device;

The second storage module is configured to store the corresponding relationship between the index number of the first file and the identifier of the storage node of the first file in the first device and the second device of the indexing devices.

In the fifth aspect, the embodiment of the present invention also provides an access device, the access device includes: a processor and a storage medium storing instructions executable by the processor, and the storage medium relies on the The processor executes operations, and when the instructions are executed by the processor, the storage method described in one or more embodiments above is executed.

In a sixth aspect, an embodiment of the present invention also provides an indexing device, the indexing device includes: a processor and a storage medium storing instructions executable by the processor, and the storage medium relies on the processor through a communication bus The execution operation is to execute the storage method described in one or more embodiments above when the instruction is executed by the processor.

In a seventh aspect, an embodiment of the present invention provides a computer storage medium storing executable instructions, and when the executable instructions are executed by one or more processors, the processors perform one or more of the above implementations The storage method described in the example.

A storage method, device, system, and computer storage medium provided by the present invention, the method includes: when the access device receives a storage request for the first file, assigning an identification code to the first file, using the first The existing parent directory and identification code of the file, generate the index number of the first file, store the association relationship between the name of the first file and the index number of the first file, and store the first file and the index number of the first file , sent to the indexing device, where the indexing device is used to determine the correspondence between the index number of the first file and the storage node's identifier of the first file after determining the identifier of the storage node used to store the first file in the storage device The relationship is stored in the first device and the second device of the index device; that is, in the present invention, when the access device obtains the storage request and assigns an identification code to the first file, it passes the existing parent directory of the first file and the identification code to generate the index number of the first file, so that the association relationship between the name of each file and the index number of the file can be stored in the access device, which is conducive to reading the file and modifying the file, and will The first file and the index number of the first file are sent to the index device, then the index device can determine the identification of the storage node used to store the first file, and compare the index number of the first file with the storage node identification of the first file Correspond and store in the first device and the second device of the index device, so that when the first device fails, the second device can also reset the route to ensure the normal storage and reading of files, thereby improving the file storage efficiency. Reliability, thereby improving the storage efficiency of file storage.

Description of drawings

FIG. 1 is a schematic structural diagram of an optional file system in an embodiment of the present invention;

FIG. 2 is a schematic flow diagram of an optional storage method in an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an example of an optional file system in an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an example of an optional file system in an embodiment of the present invention;

FIG. 5 is a schematic diagram of an optional directory in the embodiment of the present invention;

FIG. 6 is a schematic flow diagram of an optional Raft algorithm in an embodiment of the present invention;

FIG. 7 is a schematic flowchart of an optional storage method in an embodiment of the present invention;

FIG. 8 is a schematic flowchart of another optional storage method in an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an optional access device in an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of an optional indexing device in an embodiment of the present invention;

FIG. 11 is a schematic structural diagram of another optional access device in an embodiment of the present invention;

FIG. 12 is a schematic structural diagram of another optional indexing device in the embodiment of the present invention;

FIG. 13 is a schematic structural diagram of an optional file system in an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention.

Embodiment one

An embodiment of the present invention provides a storage method, which is applied to a file system. FIG. 1 is a schematic structural diagram of an optional file system in an embodiment of the present invention. As shown in FIG. 1 , the file system may include : an access device 11, an index device 12 and a storage device 13; wherein, the index device 12 includes two types of devices, namely a first device 121 and a second device 122; wherein,

The access device 11 has a communication connection with the index device 12, and the access device 11 can be a user client or a gateway server, playing the role of a client in the file system; the first device 121 and the second device 122 of the index device 12 Both are computing servers, the first device 121 can be a ZFS server, the second device 122 can be an Index server, and the Index server is used to manage the routing of the ZFS server; the storage device 13 is usually a storage server, used to store files.

Based on the above-mentioned file system in FIG. 1, an embodiment of the present invention provides a storage method. FIG. 2 is a schematic diagram of an optional storage process interaction in the embodiment of the present invention. As shown in FIG. 2, the storage method may include:

S201: When the access device 11 receives a storage request for the first file, allocate an identification code to the first file;

At present, mainstream distributed file systems have good performance in reading and writing large files. The performance bottleneck is usually limited to reading and writing of massive small files and index access, and the problem of reading and writing small files can also be attributed to index design problems. Mainstream distributed file system indexes are roughly divided into two directions. Among them, there is a bottleneck in the scalability of the centralized index, and the basic performance of the decentralized index is insufficient. Mainstream distributed file systems usually can only promote small file aggregation at the upper application level or at the protocol access level to alleviate bottleneck restrictions in some models.

At the same time, these distributed systems are hindered by traditional consensus algorithms (for example, PAXOS algorithm or Multi-PAXOS algorithm), and it is impossible to simultaneously design an efficient system when implementing PAXOS semantics, and ultimately cannot prove the actual security of the designed system. In the production environment, various mainstream distributed file systems have split brains, and data loss is not uncommon, and fault self-repair to ensure final consistency is even more normal. The principle of the PAXOS algorithm itself has complete security provability and derivation, but its biggest flaw is that it is difficult to understand and implement in practice. It often requires a lot of extra overhead when used in distributed file storage, and ultimately cannot accurately guarantee security. And again to maintain the final consistency.

In order to improve file storage efficiency, in the embodiment of the present invention, after receiving the storage request of the first file, the access device 11 first assigns an identification code to the first file. When the first file needs to be stored, an identification code can be generated for the first file according to the preset rules. For example, it can be coded according to the serial number of the stored file, or the files to be stored can be numbered according to letters and numbers. Here , which is not specifically limited in this embodiment of the present invention.

S202: The access device 11 generates an index number of the first file by using the existing parent directory and the identification code of the first file;

Specifically, after the access device 11 assigns the identification code to the first file, it can use the parent directory of the first file and the identification code of the first file to generate the index number of the first file, for example, the identification code assigned to the first file code is 345, and the parent directory of the first file is disk F, then the index number of the first file can be F/345, and thus, the index number of the first file can be obtained.

S203: The access device 11 stores the association relationship between the name of the first file and the index number of the first file;

After the index number of the first file is generated, the name of the first file can be associated with the index number of the first file and stored, so that when the file is read or lost, the index number of the file can be found by naming the file , which is beneficial for reading or modifying files from storage devices.

S204: The access device 11 sends the name of the first file and the index number of the first file to the indexing device;

Wherein, indexing devices generally include multiple first devices and multiple second devices, and each second device manages multiple first devices, then, in order to determine the first The device corresponding to the file, in an optional embodiment, the above method may also include:

The access device 11 sends an acquisition request for the routing information of the indexing device to the second device 122;

The second device 122 determines the routing information of the first device managed by the second device 122 as the routing information of the indexing device, and sends it to the access device 11;

The access device 11 determines the received routing information of the indexing device as the determined routing information of the indexing device.

The access device 11 determines the device corresponding to the first file from the first device of the index device based on the determined routing information of the index device and according to the existing parent directory of the first file;

Specifically, the access device 11 sends to the second device 122 a request for obtaining the routing information of the indexing device. Here, it should be noted that the second device 122 here represents all the second devices in the indexing device, and each There are at least two first devices connected to each of the two devices, and they are managed. Then, after receiving the acquisition request, the second device 122 uses the routing information of the first device under each second device as the routing information of the index device. The routing information is sent to the access device 11, so that the access device 11 will receive the routing information of the indexing device and determine it as the determined routing information of the indexing device.

In this way, the routing information of the indexing device is determined by sending an acquisition request, so that the determined routing information of the indexing device is effective routing information, so as to ensure normal reading and writing of the file system.

The routing information of the determined index device is stored in the access device 11. Since the first device corresponding to the parent directory is stored in the routing information of the index device, the access device 11 can use the determined routing information of the index device In the process, the first device corresponding to the parent directory of the first file is determined, and the first device corresponding to the parent directory of the first file is determined as the device corresponding to the first file.

It should be noted that the determined routing information of the indexing device stored in the access device 11 may be valid routing information or invalid routing information. In order to make the determined routing information of the indexing device valid routing information , to ensure the normal reading and writing and modification of the storage device, any of the following methods can be adopted. For example, it can be determined whether it is valid routing information by comparing the version number of the routing information. In an optional embodiment, the above method Can also include:

When the version number of the received routing information of the indexing device is the same as the version number of the stored routing information of the indexing device, the access device 11 determines the stored routing information of the indexing device as the determined routing information of the indexing device ;

When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, the access device 11 determines the received routing information of the indexing device as the determined routing information of the indexing device .

That is to say, the access device 11 compares whether the version number of the routing information received from the indexing device is the same as the version number of the stored routing information of the indexing device. The routing information is valid routing information, so the access device 11 determines the stored routing information of the indexing device as the determined routing information of the indexing device.

If they are different, it means that the routing information of the indexing device stored in the access device 11 is invalid routing information, so the routing information of the indexing device received by the accessing device is determined as the determined routing information of the indexing device.

In this way, it can be ensured that the determined routing information of the indexing device is valid routing information, and problems caused by access devices using invalid routing information can be prevented.

After the access device 11 determines the index number of the first file, the first file and the index number of the first file are sent to the indexing device, so that the indexing device is used to store the first file in the determined storage device After the identification of the node, the corresponding relationship between the index number of the first file and the identification of the storage node of the first file is stored in the first device and the second device of the indexing device.

Wherein, it should be noted that, between the above S203 and S204, S203 may be executed first, S204 may be executed first, or S203 and S204 may be executed simultaneously, which is specifically limited in this embodiment of the present application.

S205: The indexing device 12 determines an identifier of a storage node used to store the first file in the storage device;

Wherein, the above-mentioned storage nodes may include a master node and a backup node.

Here, specifically, after the first device of the indexing device 12 obtains the first file and the index number of the first file, it uses the identifier of the storage node for storing the first file from the storage nodes that are free in the storage device; wherein , the storage node may include a master node and a backup node. Here, it should be noted that, generally, there is one master node, and there may be 2 or 3 backup nodes. Here, this embodiment of the present invention does not specifically limit this.

S206: The indexing device 12 stores the corresponding relationship between the index number of the first file and the identifier of the storage node of the first file in the first device and the second device of the indexing device 12;

Specifically, the first device of the index device, that is, the device that follows the first file determined from the first device of the index device, needs to combine the index number of the first file with the identification and backup of the primary node of the first file The corresponding relationship between the identifiers of the nodes is stored, so that when the user needs to read the stored file, the file can be read through the corresponding relationship.

In addition, the first device also needs to send the above correspondence to the second device that manages the first device, so that the second device stores the correspondence, so as to prevent the failure to read and write files when a certain first device fails , ensuring the reliability of the file system.

In order to improve the consistency of reading and writing files in the file system, in an optional embodiment, the above method further includes:

When the device corresponding to the first file receives a reply message that the storage device 13 writes successfully to the first file, the device corresponding to the first file sends a response message to the access device 11 that the write to the first file is successful;

When the access device 11 receives the response message that the writing of the first file is successful, it generates prompt information; wherein, the prompt information is used to prompt the user that the first file is successfully stored.

That is to say, when the device corresponding to the first file successfully stores the first file to the storage device 13, the storage device 13 will send a reply message of writing success for the first file to the device corresponding to the first file, and the first After the device corresponding to the file receives the reply message, it sends a write success response message to the access device 11, so that the access device 11 knows that the first file is stored successfully, and the access device 11 also generates a prompt after knowing that the storage is successful. information to prompt the user that the first file is stored successfully.

When the writing fails, the storage device 13 will not send a reply message for the writing success of the first file to the device corresponding to the first file, and the device corresponding to the first file will not send a message to the access device 11, so when When the access device 11 does not receive a write success response message within a period of time, it determines that the storage of the first file has failed. At this time, it prompts the user that the storage of the first file fails. Then, the file system does not store the first file. In this way, The consistency of data in the file system can be guaranteed through the response between the access device 11 and the index device 12 .

For the file system, in addition to storing the file, the stored file can also be read from the file system. In an optional embodiment, the above method may also include:

The access device 11 receives a read request of the stored second file;

The access device 11 determines the index number of the second file according to the association between the name of the stored file and the index number of the file;

The access device 11 determines the device corresponding to the second file from the first device of the index device based on the determined routing information of the index device and according to the existing parent directory of the second file;

The access device 11 sends the index number of the second file to the device corresponding to the second file;

The device corresponding to the second file determines the storage node of the second file according to the correspondence between the index number of the stored file and the identifier of the storage node of the file;

The device corresponding to the second file reads the second file from the storage node of the second file.

In order to read the stored second file, the user sends a read request of the second file to the access device 11 through the interface of the access device 11. Therefore, the access device 11 can determine the index number of the second file.

Then determine the first device corresponding to the existing parent directory of the second file from the determined routing information of the index device, determine it as the device corresponding to the second file, and send the index number of the second file to the second The device corresponding to the file.

Since the corresponding relationship between the index number of the file and the identifier of the storage node of the file is stored in the device corresponding to the second file, the device corresponding to the second file determines according to the corresponding relationship, the storage node of the second file, and The second file is read from the storage node of the second file.

For the file system, in addition to storing the file, the stored file can also be modified from the file system. In an optional embodiment, the above method can also include:

When the data of the master node in the storage node of the stored third file changes, the access device 11 determines the index number of the third file according to the association between the name of the stored file and the index number of the file;

The access device 11 determines the device corresponding to the third file from the first device of the index device according to the existing parent directory of the third file based on the determined routing information of the index device;

The access device 11 sends the modified third file and the index number of the third file to the device corresponding to the third file.

The device corresponding to the third file determines the backup node among the storage nodes of the third file according to the corresponding relationship between the index number of the stored file and the identifier of the storage node of the file.

Wherein, the device corresponding to the third file is used to determine the backup node among the storage nodes of the third file according to the index number of the third file, and the backup node of the third file is used to update the stored data to the modified third file .

In order to modify the stored third file, the user modifies the data of the master node of the third file through the access device 11. Since the access device 11 stores the association between the name of the file and the index number of the file, The access device 11 may determine the index number of the third file.

Then, determine the first device corresponding to the existing parent directory of the third file from the determined routing information of the index device, determine it as the device corresponding to the third file, and send the index number of the third file to the first device The three files correspond to the device.

Since the corresponding relationship between the index number of the file and the identifier of the storage node of the file is stored in the device corresponding to the third file, the device corresponding to the third file determines according to the corresponding relationship that the backup node of the third file, the The demarcated point of the third file uses the modified third file to update the data in the backup node of the third file.

When the line file system fails, in order to ensure the normal operation of the file system, in an optional embodiment, the above method may also include:

When one of the first devices managed by the second device fails, the second device selects a first device from other first devices managed by the second device;

The second device sends the corresponding relationship between the index number of the file stored in one of the first devices and the identifier of the storage node of the file to the selected first device for storage, and updates the routing information of the index device.

That is to say, for a certain second device, when one of the first devices managed by the second device fails, in order to ensure the normal operation of the file system, the second device first starts from other devices managed by the second device. Select a first device from the first devices, and send the pre-stored correspondence between the index number of the file stored in one of the first devices and the identifier of the storage node of the file to the selected first device device and store it.

In this way, when the first device fails, the file system can use other normal first devices to deal with it in time, ensuring that the file system can cope with emergencies, increasing the reliability of the file system, and improving the read and write of the file system efficiency.

Examples are given below to describe the storage method described in one or more embodiments above.

FIG. 3 is a schematic structural diagram of an example of an optional file system in an embodiment of the present invention. As shown in FIG. 3, the file system may include: an access layer, an index layer, and a storage layer;

Among them, the access layer can be a user client or a gateway server (Posix File System Client), the index layer is composed of index clusters, including two computing servers, namely ZFSServer and IndexServer, and the storage layer is composed of storage clusters, including at least two Storage server (ZFS zpool).

It should be pointed out that the above-mentioned three-tier architecture is not an absolute 1:1:1 relationship. The storage cluster is the storage resource provider of the index cluster, and the docking relationship depends entirely on the level of isolation of physical or logical fault domains. For example, multiple index clusters can share a set of storage cluster resources, and these index clusters can also provide services to the same access gateway.

A file write process:

In the first step, the device in the access layer first allocates an Inode number (equivalent to an identification code) in a distributed file system (DZFS, DistributeZFS) for writing a file.

In the second step, the device in the access layer converts the parent directory of the written file (including the DZFS root directory and the directory used to divide the namespace) into prefix_id, and converts its own Inode number in DZFS into file_id; The two ids are concatenated into the index number of the written file, and the written file is associated with the index number of the written file.

The third step, because the access layer caches the route, if the route is not invalid, the device in the access layer can directly send the written file to the ZFSServer corresponding to the written file; if the route is invalid, re-acquire the written file from the IndexServer The corresponding ZFSServer sends the written file to the ZFSServer.

In the fourth step, ZFSServer selects the primary node and the redundant node (equivalent to the backup node) from the idle nodes in the storage layer, and writes files to the primary node and the redundant node at the same time.

Step 5: After the master node and the redundant node write together, reply ZFSServer that the write is successful.

In the sixth step, the ZFSServer associates and stores the index number written into the file with the identification of the primary node and the identification of the redundant node writing the file.

In the seventh step, ZFSServer replies to the access layer that the writing is completed, and the access layer replies to the user that the writing is completed.

A file read process:

Step 1: The device in the access layer obtains the name of the read file, determines the index number of the read file, and sends the index number of the read file to the corresponding ZFSServer that reads the file;

In the second step, ZFSServer determines the identity of the primary node and/or the identity of the redundant node that reads the file according to the index number of the read file, and reads the file from the ZFS zpool.

Fig. 4 is a schematic diagram of the structure of an example of an optional file system in the embodiment of the present invention. As shown in Fig. 4, the access layer is responsible for analyzing the file semantics, and transforms the semantics of the application's file and directory operations into the DZFS Semantics recognizable by the backend index layer and storage layer. In the public cloud environment, it is also responsible for the isolation of multiple tenants. This layer is usually stateless, and the industry generally supports NFSv4 interfaces and SMB2.0.

The index layer is responsible for bionicing the directory and file concepts of the local file system on a distributed architecture, usually shared by multi-tenancy. The benefits of this architecture are:

The index data is separated, the index is more of an enhanced central processing unit (CPU-Intensive, Central Processing Unit-Intensive), while the data is of enhanced input/output (IO-Intensive, Input/Output-Intensive), the two are different The hardware requirements allow them to deploy the most cost-effective deployments on more targeted hardware, scale out separately, and facilitate independent architecture evolution. If the storage is mixed, there may be such a problem: the utilization rate of the storage pool occupied by the data is still very low, but the index consumes a lot and must be expanded, and the overall resource utilization rate drops.

Reduce indexing and IO interference: Metadata operations actually account for the bulk of the entire file system operation. For example, the HEAD request used by the content distribution network (CDN, Content Delivery Network) back-to-source does not need to access data in most cases. This is especially evident in multi-tenant scenarios. If indexes and data coexist, it is easy to cause index-intensive applications to interfere with IO, resulting in a decrease in the performance of both.

It is more suitable to support elastic architecture: for example, in the scenario where the index and data coexist, it is difficult to represent a very large file (for example, PB level), but in the separation architecture, it is more convenient to do data scheduling and other operations according to the data access model .

The storage layer is the guarantee of high reliability and high availability of data. It must also consider the requirements of the business system for IO access modes, and the random IO required for file storage needs to be implemented more efficiently. ZFS is already a stand-alone file storage, with overall disk management reliability and top-level availability file system, so the storage layer design more considers distributed availability and consistency.

Let's expand each layer as follows:

For the access layer, there are two main points: the protocol layer (Protocal Layer) will carry out the semantic conversion of the protocol, for example, the above-mentioned NFSv4 or smb2.0 protocol; the communication between the access layer, the index layer and the storage layer through the common The remote procedure call (RPC, Remote Procedure Call) framework is completed.

For the index layer, in terms of implementation, the two parts of directory and file index are implemented separately, called DirIdx and FileIdx. In the persistence of index data, the two parts are divided and stored.

DirIdx is in charge of IndexServer and relies on external database storage for persistence. Even in the scenario of a large number of small files, the scale of DirIdx does not appear to be huge, and it can be efficiently accessed on an external distributed database (Distribute DB, Distribute DataBase). At the same time, it will be explained below The indexing frequency of the access layer accessing IndexServer will not be very high.

FileIdx is in charge of ZFSServer, and is persisted by ZFS itself of each storage node. At the same time, IndexServer manages the first-level index from the file to the parent directory (including the root directory of the namespace).

Each file in DZFS will be assigned a unique number in the namespace to which it belongs (equivalent to the Inode number of traditional file storage). DZFS uses the number of the file's parent directory (including the root directory of the namespace) as prefix_id, and uses the number of the file itself as file_id, prefix_id /file_id as a unique key value indexed across the namespace.

DirIdx only describes the existence basis of the directory item and is used as the basis for real-time allocation of sub-files under the directory item. After the client creates the directory, DirIdx will be recorded by the database of IndexServer, where the primary key is the unique ID of the directory item, and the database entry will contain its sub-file routing real-time distribution. In layman's terms, the newly created directory will be assigned to different ZFSServers by the IndexServer. Once the ZFSServer is confirmed in the normal operation of the cluster, the client will directly communicate with the ZFSServer for operations on the directory itself and sub-files.

Figure 5 is a schematic diagram of an optional directory in the embodiment of the present invention, as shown in Figure 5, including directories /dir0/dirx/, dir0/diry/, /dir0/diry/dileα/, /dir1/, /dir2 /file0/; Multiple subdirectories are divided into multiple ZFSServers for processing. From the above routing design with the unique number in DirIdx as the key, it can be seen that the client can obtain the prefix_id for reading and writing files, and the route can be obtained through the prefix_id, so as to finally achieve the purpose of data distribution.

The index layer is described in more detail:

Node discovery: The lease heartbeat (heartbeat) between IndexServer and ZFSServer is initiated by IndexServer, so that IndexServer does not need to expose services to ZFSServer, and some external service discovery applications such as zookeeper can guarantee node discovery. Therefore, IndexServer needs to record the list of ZFSServers that are allowed to serve in the current cluster, and only the ZFSServers in this list are allowed to serve.

Routing maintenance: Under normal circumstances, IndexServer needs to ensure that a leaf directory is only served by one ZFSServer, and it needs to maintain the correctness of this routing table through, for example, a lease mechanism. There are only two situations for the route obtained by the client: one is that the route is correct: usually the route does not change, and the routing table maintained by the client is the same as that maintained by the IndexServer. The other is route expiration: IndexServer schedules the route of DirIdx, causing the routing table maintained by the client to expire.

After the client obtains the route of a certain subdirectory for the first time, it does not need to communicate with IndexServer regularly to obtain the latest route, because it will check whether the route is expired in the normal communication with ZFSServer, so that the pressure on IndexServer will not be great. At the same time, since a single subdirectory routing entry only identifies the routing of a single directory to ZFSServer, the pressure on the client cache is also extremely low. A lease is implemented between ZFSServer and IndexServer to prevent ZFSServer from voluntarily withdrawing from service after a network partition occurs.

ZFSServer is the core module on the key IO link, and writes data to the storage node through the interaction with the underlying storage. The architectural feature of ZFSServer is stateless. It realizes the persistence of client files and file attributes through the underlying storage layer module and ZFS, ensuring high data reliability.

Routing scheduling: IndexServer only occurs routing scheduling in the following cases: one case is directory splitting: if a single subdirectory is too hot and the number of sub-files is too high, the directory will be split, and the directory will be divided and changed through the same semantics as the client. The file key is prefix_id/num/file_id to achieve the purpose of distribution. Heat and usage are reported by ZFSServer through lease heartbeat. Another case is directory migration: if the overall ZFSServer heat usage is unbalanced, the directory routing is migrated to an idle ZFSServer (including the newly added ZFSServer). The above is reported by the lease heartbeat. Another situation is that the ZFSServer service is abnormal: ZFSServer's own lease heartbeat report exits, or the client schedules due to timeout. Other situations are network anomalies, network partitions, and so on.

High availability of IndexServer: The access pressure of the above-mentioned IndexServer is very small, so the main-standby structure is also realized through external service discovery such as zookeeper. Only the main IndexServer can update (update) the database, and the slave IndexServer only has the right to query (query) the database to maintain Its own cache space is convenient for rapid switching in case of exception.

Additional services: file system additional services such as QUOTA, QOS, snapshot and other tenant functions can be collectively attributed to setting the attributes of directory files in the distributed file system. The design of extracting directory indexes separately in the embodiment of the present invention is to realize these functions All aspects can be realized.

In other words, the index layer uses the unique id as the primary key of the database index and also realizes the efficiency of the list (size range list). In summary, this implementation satisfies the purpose of index separation and index IO interference reduction.

For the storage layer, the efficient random IO and high reliability required by file storage have been realized through the ZFS stand-alone file system. The storage layer also needs to use additional modules to achieve high availability. Usually, distributed file systems use redundancy to ensure that Raft is introduced as a consensus algorithm for redundancy.

The above-mentioned DirIdx indexes a single ZFSServer when indexing. When ZFSServer writes a subfile based on this DirIdx, it will select a stand-alone ZFS process in the storage layer. This directory is regarded as the master copy on this ZFS, and the copy to other nodes is controlled by the ZFS process. Write and sync. After the copy is written, the master ZFS stand-alone will reply to ZFSServer which nodes the file has been copied to, and ZFSServer will record the copy node for this file index in memory. In the absence of exceptions, file reads go through the primary replica only.

The most important implementation of the storage layer is to maintain consistency between replicas through Raft. Raft is an algorithm for maintaining the consistency of the replication state machine. By transforming the ZFS transaction level, the consistency between replicas is realized. Figure 6 is a schematic flow diagram of an optional Raft algorithm in the embodiment of the present invention, as shown in Figure 6 , relying on ZFS is a transactional file storage system, which has an entry for Raft access, and at the same time, copy-on-write (COW) storage can ensure the idempotence of transaction log (log) operations, so the storage layer ZFS It is relatively simple to transform transaction processing into distributed consistency.

Redundant synchronous writes:

As shown in Figure 6,

① ZFS submits the transaction to the Raft node.

②Raft records the wal log to persist the operations on the transaction space (for example, x=1). After most nodes are persisted, they can submit the state machine to apply these operations.

③Raft StateMachine transforms the submitted state application operation, and operates on the ZFS transaction log storage space (including cache). (The contents of the log space, such as: x:9; y:22; z:33, become: x:1; y:22; z:33 after applying this operation).

④⑤The transaction process of ZFS itself uses the transaction log to ensure the atomicity of file IO.

The asynchronous snapshot saves the transaction log of ZFS. This snapshot is generated and acquired by Raft control, and is used as Raftwal log compaction to avoid the infinite growth of Raft wal log.

It can be seen that the whole process achieves the final distributed consistency through the atomicity guaranteed by the transaction process of ZFS itself and the algorithm security provided by Raft.

Data Migration: As mentioned above in routing scheduling, no matter what happens when DirIdx scheduling of the index layer is triggered, data migration will be triggered. The ZFSServer that is newly scheduled to the directory can start data migration through the lease and routing expiration prompts, but the COW feature of ZFS mentioned above, the data Migration overhead has little impact on cluster performance.

File index: FileIdx, file information and attributes are maintained by ZFS zpool itself, and metadata storage flattened by prefix_id/file_id will exert extremely high performance. After prefix_id/file_id is used as the main identifier, the files in ZFSPool no longer need to distinguish directories through the directory tree, but use prefix_id as the list basis and prefix_id/file_id as the direct index basis, which is extremely efficient.

Additional services: Use the characteristics of ZFS to provide additional services to the prefix_id list, which is extremely efficient.

It can be seen that based on this three-layer architecture design, the DZFS system can shield single-module single-node failures.

For example, if a storage node ZFS zpool fails at the storage layer, it will first be detected by all ZFSServers of its consumers. ZFSServer can modify all file indexes it currently maintains to make other ZFS zpool nodes become master nodes to continue to provide normal services. Then ZFSServer will send a notification message to these new primary nodes, instructing to re-elect a secondary node for storage of the files involved. (For example, restore from the remaining two copies to three copies).

The failure of the index layer ZFSServer will be sensed by IndexServer, and the route (DirIdx) maintained by it will be redistributed to the non-failure ZFSServer node. ZFSServer only needs to simply visit all storage layer nodes, and can immediately collect the indexes of all files that should be maintained through the prefix_id method as mentioned above.

The failure of the Index Server at the index layer will be sensed by services such as zookeeper, and the subordinate Index Server will be notified to switch to become the master, and the latest routing allocation will be obtained from the database query to continue to maintain the routing allocation of the index layer.

The failure of the access layer client or gateway is actually managed by the user application itself. The DZFS index layer in this proposal obviously supports multi-client access, so the multi-path construction of the access layer is sufficient.

From the perspective of three-tier connection, the access layer can allocate files to different index clusters according to different namespaces, and the failure of one namespace will not affect the other namespace. Different index clusters can adopt different redundancy settings, which are maintained by IndexServer. For example, for two index clusters that use the same set of physical resources of the storage cluster, one can be set for two-copy redundancy, and the other can be set for erasure code redundancy, because redundancy setting information is included when writing to the storage layer.

A storage method provided by the present invention, the method includes: when the access device receives a storage request for the first file, assigning an identification code to the first file, using the existing parent directory and identification code of the first file code, generate the index number of the first file, store the association relationship between the name of the first file and the index number of the first file, and send the first file and the index number of the first file to the indexing device, wherein, The indexing device is configured to, after determining the identifier of the storage node used to store the first file in the storage device, store the corresponding relationship between the index number of the first file and the identifier of the storage node of the first file in the first device and the second device; that is to say, in the present invention, when the access device obtains the storage request and allocates the identification code for the first file, it generates the first file's ID through the existing parent directory and the identification code of the first file. index number, in this way, the association relationship between the name of each file and the index number of the file can be stored in the access device, which is conducive to reading the file and modifying the file, and the first file and the first file The index number is sent to the index device, then the index device can determine the ID of the storage node used to store the first file, and store the index number of the first file in correspondence with the storage node ID of the first file in the index device's In the first device and the second device, in this way, when the first device fails, the second device can also reset the route to ensure the normal storage and reading of files, thereby improving the reliability of file storage and further improving file storage. storage efficiency.

Embodiment two

The above storage method will be described below with reference to each device side deployed in the file system.

First, the storage method is described on the access device side.

An embodiment of the present invention provides a storage method, which is applied to an access device. FIG. 7 is a schematic flowchart of an optional storage method in an embodiment of the present invention. As shown in FIG. 7, the storage method may include :

S701: When receiving a storage request for the first file, assign an identification code to the first file;

S702: Generate an index number of the first file by using the existing parent directory and the identification code of the first file;

S703: Store the association relationship between the name of the first file and the index number of the first file;

S704: Send the first file and the index number of the first file to the first device of the index device;

Wherein, the indexing device is configured to store the corresponding relationship between the index number of the first file and the storage node's identifier of the first file in the indexing device after determining the identifier of the storage node used to store the first file in the storage device. first device and second device.

In an optional embodiment, the above method also includes:

sending an acquisition request for routing information of the indexing device to the second device;

receiving routing information from the indexing device sent by the second device;

When the version number of the received routing information of the indexing device is the same as the version number of the stored routing information of the indexing device, determining the stored routing information of the indexing device as the determined routing information of the indexing device;

When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, determining the received routing information of the indexing device as the determined routing information of the indexing device;

Based on the determined routing information of the indexing device, and according to the existing parent directory of the first file, the device corresponding to the first file is determined from the first devices of the indexing devices.

In an optional embodiment, the above method also includes:

When receiving a response message from the device corresponding to the first file that the writing of the first file is successful, a prompt message is generated; wherein the prompt message is used to prompt the user that the first file is successfully stored.

In an optional embodiment, the above method also includes:

receiving a read request of the stored second file;

Determine the index number of the second file according to the association between the stored file name and the index number of the file;

Based on the determined routing information of the index device, according to the existing parent directory of the second file, determine the device corresponding to the second file from the first device of the index device;

Send the index number of the second file to the device corresponding to the second file;

Wherein, the index number of the second file is used for: the device corresponding to the second file determines the storage node of the second file and reads the second file.

In an optional embodiment, the above method also includes:

When the data of the primary node in the storage node of the stored third file is changed, the index number of the third file is determined according to the association relationship between the name of the stored file and the index number of the file;

Based on the determined routing information of the index device, according to the existing parent directory of the third file, determine the device corresponding to the third file from the first device of the index device;

Sending the modified third file and the index number of the third file to the device corresponding to the third file;

Wherein, the device corresponding to the third file is used to determine the backup node among the storage nodes of the third file according to the index number of the third file, and the backup node of the third file is used to update the stored data to the modified third file.

Then, the storage method is described on the indexing device side.

An embodiment of the present invention provides a storage method, which is applied to an index device. FIG. 8 is a schematic flowchart of another optional storage method in an embodiment of the present invention. As shown in FIG. 8 , the method may include:

S801: Receive the first file and the index number of the first file sent from the access device;

Wherein, the index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned by the access device to the first file;

S802: Determine an identifier of a storage node used to store the first file in the storage device;

S803: Store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first device and the second device of the index devices.

In an optional embodiment, the above method also includes:

When receiving a response message from the storage device that the writing of the first file is successful, send a response message to the access device that the writing of the first file is successful; wherein, the response message is used for the access device to prompt the user that the first The file was saved successfully.

In an optional embodiment, the above method also includes:

When the access device receives the read request of the stored second file, the first device receives the index number of the second file;

The first device determines the storage node of the second file according to the corresponding relationship between the index number of the stored file and the identifier of the storage node of the file;

The first device reads the second file from the storage node of the second file.

In an optional embodiment, the above method also includes:

When the data of the master node in the stored storage node of the third file is modified, the first device receives the modified third file and the index number of the third file sent from the access device;

The first device determines the backup node among the storage nodes of the third file according to the corresponding relationship between the index number of the stored file and the identifier of the storage node of the file;

Wherein, the backup node of the third file is used for: updating the stored data to the modified third file.

In an optional embodiment, the method also includes:

The second device receives a request from the access device for obtaining routing information of the index device;

The second device determines the routing information of the first device managed by the second device as the routing information of the indexing device, and sends the routing information to the access device.

In an optional embodiment, the method also includes:

When one of the first devices managed by the second device fails, the second device selects a first device from other first devices managed by the second device;

The second device sends the corresponding relationship between the index number of the file stored in one of the first devices and the identifier of the storage node of the file to the selected first device for storage, and updates the routing information of the index device.

Based on the same inventive concept as the foregoing embodiments, the embodiments of the present invention provide an access device, which is consistent with the access device provided in one or more embodiments above.

FIG. 9 is a schematic structural diagram of an optional access device in an embodiment of the present invention. As shown in FIG. 9, the access device includes:

An allocation module 91, configured to allocate an identification code to the first file when a storage request for the first file is received;

A generating module 92, configured to generate an index number of the first file by utilizing the existing parent directory and the identification code of the first file;

The first storage module 93 is configured to store the association relationship between the name of the first file and the index number of the first file;

A sending module 94, configured to send the name of the first file and the index number of the first file to the indexing device;

Wherein, the indexing device is configured to store the corresponding relationship between the index number of the first file and the storage node's identifier of the first file in the indexing device after determining the identifier of the storage node used to store the first file in the storage device. first device and second device.

In an optional embodiment, the above access device is also used for:

sending an acquisition request for routing information of the indexing device to the second device;

receiving routing information from the indexing device sent by the second device;

When the version number of the received routing information of the indexing device is the same as the version number of the stored routing information of the indexing device, determining the stored routing information of the indexing device as the determined routing information of the indexing device;

When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, determining the received routing information of the indexing device as the determined routing information of the indexing device;

Based on the determined routing information of the index device, and according to the existing parent directory of the first file, the device corresponding to the first file is determined from the first devices of the index device.

In an optional embodiment, the above access device is also used for:

When receiving a response message from the device corresponding to the first file that the writing of the first file is successful, a prompt message is generated; wherein the prompt message is used to prompt the user that the first file is successfully stored.

In an optional embodiment, the above access device is also used for:

receiving a read request of the stored second file;

Determine the index number of the second file according to the association between the stored file name and the index number of the file;

Based on the determined routing information of the index device, according to the existing parent directory of the second file, determine the device corresponding to the second file from the first device of the index device;

Send the index number of the second file to the device corresponding to the second file;

Wherein, the index number of the second file is used for: the device corresponding to the second file determines the storage node of the second file and reads the second file.

In an optional embodiment, the above access device is also used for:

When the data of the primary node in the storage node of the stored third file is changed, the index number of the third file is determined according to the association relationship between the name of the stored file and the index number of the file;

Based on the determined routing information of the index device, according to the existing parent directory of the third file, determine the device corresponding to the third file from the first device of the index device;

Sending the modified third file and the index number of the third file to the device corresponding to the third file;

Wherein, the device corresponding to the third file is used to determine the backup node among the storage nodes of the third file according to the index number of the third file, and the backup node of the third file is used to update the stored data to the modified third file.

In practical applications, the above-mentioned distribution module 91, generation module 92, first storage module 93 and sending module 94 can be implemented by a processor located on the access device, specifically a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), digital signal processor (DSP, Digital Signal Processing) or field programmable gate array (FPGA, Field Programmable Gate Array) and other implementations.

An embodiment of the present invention provides a first device, which is consistent with the first device described in one or more embodiments above.

FIG. 10 is a schematic structural diagram of an optional indexing device in an embodiment of the present invention. As shown in FIG. 10, the indexing device includes:

The receiving module 101 is configured to receive the first file and the index number of the first file sent from the access device; wherein, the index number of the first file is obtained by the access device using the existing parent directory of the first file, and access generating an identification code assigned by the device to the first file;

A determining module 102, configured to determine an identifier of a storage node used to store the first file in the storage device;

The second storage module 103 is configured to store the corresponding relationship between the index number of the first file and the identifier of the storage node of the first file in the first device and the second device of the index devices.

In an optional embodiment, the above-mentioned first device is also used for:

When receiving a reply message from the storage device that the writing of the first file is successful, send a response message to the access device that the writing of the first file is successful; wherein, the response message is used by the access device to prompt the user to store the first file success.

In an optional embodiment, the above-mentioned first device is also used for:

When the access device receives the read request of the stored second file, the index number of the second file is received;

According to the corresponding relationship between the index number of the stored file and the identifier of the storage node of the file, determine the storage node of the second file;

The second file is read from the storage node of the second file.

In an optional embodiment, the above-mentioned first device is also used for:

When the data of the master node in the storage node of the stored third file changes, receiving the modified third file and the index number of the third file sent from the access device;

According to the corresponding relationship between the index number of the stored file and the identification of the storage node of the file, determine the backup node in the storage node of the third file;

Wherein, the backup node of the third file is used for: updating the stored data to the modified third file.

In an optional embodiment, the above-mentioned second device is also used for:

receiving a request from the access device for obtaining routing information of the index device;

The routing information of the first device managed by the second device is determined as the routing information of the indexing device, and sent to the access device.

In an optional embodiment, the above-mentioned second device is also used for:

When one of the first devices managed by the second device fails, select a first device from other first devices managed by the second device;

The corresponding relationship between the index number of the file stored in one of the first devices and the identifier of the storage node of the file is sent to the selected first device for storage, and the routing information of the index device is updated.

In practical applications, the above-mentioned receiving module 101, determining module 102 and second storage module 103 may be implemented by a processor located on the indexing device, specifically a CPU, MPU, DSP, or FPGA.

Fig. 11 is a schematic structural diagram of another optional access device in the embodiment of the present invention. As shown in Fig. 11, the embodiment of the present invention provides an access device 1100, including:

The processor 111 and the storage medium 112 storing instructions executable by the processor 111, the storage medium 112 relies on the processor 111 to perform operations through the communication bus 113, when the instructions are executed by the processor 111, Execute the storage method described in Embodiment 1 above.

It should be noted that, in actual application, various components in the terminal are coupled together through the communication bus 113 . It can be understood that the communication bus 113 is used to realize connection and communication between these components. In addition to the data bus, the communication bus 113 also includes a power bus, a control bus and a status signal bus. However, the various buses are labeled as communication bus 113 in FIG. 11 for clarity of illustration.

FIG. 12 is a schematic structural diagram of another optional indexing device in an embodiment of the present invention. As shown in FIG. 12 , an indexing device 1200 is provided in an embodiment of the present invention, including:

The processor 121 and the storage medium 122 storing instructions executable by the processor 121, the storage medium 122 relies on the processor 121 to perform operations through the communication bus 123, when the instructions are executed by the processor 121, Execute the storage method described in Embodiment 1 above.

It should be noted that, in actual application, various components in the terminal are coupled together through the communication bus 123 . It can be understood that the communication bus 123 is used to realize connection and communication between these components. In addition to the data bus, the communication bus 123 also includes a power bus, a control bus and a status signal bus. However, the various buses are labeled as communication bus 123 in FIG. 12 for clarity of illustration.

An embodiment of the present invention provides a file system. FIG. 13 is a schematic structural diagram of an optional file system in an embodiment of the present invention. As shown in FIG. 13 , the file system 1300 includes The access device, the index device described in one or more embodiments above, and the storage device.

An embodiment of the present invention provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the storage method described in Embodiment 1.

Wherein, the computer-readable storage medium may be a magnetic random access memory (ferromagnetic random access memory, FRAM), a read-only memory (Read Only Memory, ROM), a programmable read-only memory (Programmable Read-Only Memory, PROM), an erasable In addition to programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), flash memory (Flash Memory), magnetic surface memory, optical disc, Or a memory such as a Compact Disc Read-Only Memory (CD-ROM).

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention.

Claims (14)

1. A storage method is applied to access equipment and comprises the following steps:

in the event that a storage request for a first file is received, assigning an identification code to the first file;

generating an index number of the first file by using the existing parent directory of the first file and the identification code;

storing the incidence relation between the name of the first file and the index number of the first file;

sending the first file and the index number of the first file to index equipment;

the index device is used for storing the corresponding relation between the index number of the first file and the identifier of the storage node of the first file into the first device and the second device of the index device after the identifier of the storage node of the first file in the storage device is determined.

2. The method of claim 1, further comprising:

sending an acquisition request aiming at the routing information of the index device to the second device;

receiving routing information of the index device sent by the second device;

when the received version number of the routing information of the index device is the same as the stored version number of the routing information of the index device, determining the stored routing information of the index device as the determined routing information of the index device;

when the version number of the received routing information of the index device is different from the version number of the stored routing information of the index device, determining the received routing information of the index device as the determined routing information of the index device;

and determining equipment corresponding to the first file from the first equipment of the indexing equipment according to the determined routing information of the indexing equipment and the existing parent directory of the first file.

3. The method according to claim 1 or 2, characterized in that the method further comprises:

receiving a reading request of a stored second file;

determining the index number of the second file according to the association relationship between the name of the stored file and the index number of the file;

determining equipment corresponding to the second file from first equipment of the indexing equipment according to the determined routing information of the indexing equipment and the existing parent directory of the second file;

sending the index number of the second file to the equipment corresponding to the second file;

wherein the index number of the second file is used to: and the device corresponding to the second file determines the storage node of the second file and reads the second file.

4. The method according to claim 1 or 2, characterized in that the method further comprises:

when the data of a main node in a storage node of a stored third file is changed, determining the index number of the third file according to the association relationship between the name of the stored file and the index number of the file;

determining equipment corresponding to the third file from first equipment of the indexing equipment according to the determined routing information of the indexing equipment and the existing parent directory of the third file;

sending the modified third file and the index number of the third file to equipment corresponding to the third file;

the device corresponding to the third file is configured to determine a backup node in the storage node of the third file according to the index number of the third file, where the backup node of the third file is configured to update the stored data to the modified third file.

5. A storage method is applied to an index device and comprises the following steps:

receiving a first file sent by access equipment and an index number of the first file; the index number of the first file is generated by an access device by using an existing parent directory of the first file and an identification code allocated to the first file by the access device;

determining an identifier of a storage node in the storage device for storing the first file;

and storing the corresponding relation between the index number of the first file and the identifier of the storage node of the first file in first equipment and second equipment of the index equipment.

6. The method of claim 5, further comprising:

when the access equipment receives a reading request of a stored second file, the first equipment receives an index number of the second file;

the first equipment determines the storage node of the second file according to the corresponding relation between the index number of the stored file and the identifier of the storage node of the file;

and the first device reads the second file from the storage node of the second file.

7. The method of claim 5, further comprising:

when the data of the main node in the storage node of the stored third file is changed, the first device receives the modified third file sent by the access device and the index number of the third file;

the first equipment determines a backup node in the storage nodes of the third file according to the corresponding relation between the index number of the stored file and the identification of the storage node of the file;

wherein the backup node of the third file is configured to: updating the stored data to the modified third file.

8. The method of claim 5, further comprising:

the second equipment receives an acquisition request of the routing information of the index equipment from the access equipment;

and the second equipment determines the routing information of the first equipment managed by the second equipment as the routing information of the index equipment, and sends the routing information to the access equipment.

9. The method of claim 8, further comprising:

when one of the first devices managed by the second device fails, the second device selects one first device from the other first devices managed by the second device;

and the second equipment sends the corresponding relation between the index number of the file stored in one of the first equipment and the identifier of the storage node of the file to the selected one of the first equipment for storage, and updates the routing information of the index equipment.

10. An access device, comprising:

the device comprises an allocation module, a storage module and a processing module, wherein the allocation module is used for allocating an identification code to a first file under the condition of receiving a storage request aiming at the first file;

the generating module is used for generating an index number of the first file by utilizing the existing parent directory of the first file and the identification code;

the first storage module is used for storing the association relationship between the name of the first file and the index number of the first file;

the sending module is used for sending the first file and the index number of the first file to an index device;

the index device is used for storing the corresponding relation between the index number of the first file and the identifier of the storage node of the first file into the first device and the second device of the index device after the identifier of the storage node of the first file in the storage device is determined.

11. An indexing device, comprising:

the receiving module is used for receiving a first file sent by access equipment and an index number of the first file; the index number of the first file is generated by an access device by using an existing parent directory of the first file and an identification code allocated to the first file by the access device;

the determining module is used for determining the identification of a storage node in the storage equipment, which is used for storing the first file;

and the second storage module is used for storing the corresponding relation between the index number of the first file and the identifier of the storage node of the first file in the first equipment and the second equipment of the index equipment.

12. An access device, comprising:

a processor and a storage medium storing instructions executable by the processor to perform operations dependent on the processor via a communication bus, the instructions when executed by the processor performing the storage method of any of claims 1 to 4.

13. An indexing apparatus, comprising:

a processor and a storage medium storing processor-executable instructions that depend on the processor via a communication bus to perform operations, the instructions when executed by the processor performing the storage method of any of claims 5 to 9 above.

14. A computer storage medium having stored thereon executable instructions which, when executed by one or more processors, perform the storage method of any one of claims 1 to 4 or the storage method of any one of claims 5 to 9.

Images