CN119645948A

CN119645948A - Quota allocation method and device and other equipment

Info

Publication number: CN119645948A
Application number: CN202311839356.XA
Authority: CN
Inventors: 叶柳青; 黎红波; 赵宏宇
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2023-09-18
Filing date: 2023-12-27
Publication date: 2025-03-18

Abstract

The present application provides a quota allocation method, device and other equipment, the method comprising: a metadata management node allocates a target quota to a target client; if the target quota is invalid, the metadata management node updates the target quota according to a gradient allocation strategy, the gradient allocation strategy is used to instruct the metadata management node to allocate quotas to one or more clients based on a first remaining space and a preset quota level, the preset quota level is divided according to the first remaining space, and the first remaining space includes the storage space after the total quota space is removed from the allocation area; if the target quota is valid, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation strategy. In this way, a quota is allocated to a client according to the gradient allocation strategy, and the quota over-limit is constrained by limiting the size of the space that can be written by the client.

Description

Quota allocation method and device and other equipment

The present application claims priority from the chinese patent application filed at 2023, 9 and 18, filed with the national intellectual property office, application number 202311203612.6, entitled "method, apparatus, and other device for quota management", the entire contents of which are incorporated herein by reference.

Technical Field

The present application relates to the field of storage, and in particular, to a quota allocation method, apparatus, and other devices.

Background

A distributed file system (distributedfilesystem, DFS) refers to a complete, hierarchical file system formed by a combination of several different logical disk partitions or volumes, or by a combination of different logical disk partitions or volumes, in which the physical storage resources managed by the file system are not necessarily directly connected to a local node, but rather are connected to the node via a computer network. The file system can limit the quota of the whole disk space, so as to achieve the aim of managing and controlling the disk space.

In a multi-client scenario of a distributed file system, in order to avoid performance loss, most of the existing schemes manage through real-time quota update of a metadata server and passive quota update of a client, but the whole disk space quota accuracy cannot be controlled.

Disclosure of Invention

The application provides a quota allocation method, a quota allocation device and other equipment, which are used for allocating quota for a client according to a gradient allocation strategy, and limiting the size of writable space of the client so as to restrict the quota overrun.

In view of this, in a first aspect, the present application provides a quota allocation method, applied to a metadata management node in a distributed file system, where the distributed file system includes one or more directory entries and one or more clients, where the one or more directory entries are used to store data written by the one or more clients, the metadata management node is used to allocate a quota to the one or more clients, and the quota of each client indicates a size of a storage space allocated by the metadata management node for the client to store data, and the client is used to store data to be written in the directory entries according to the quota.

When the target quota of the target client fails, the metadata management node updates the target quota according to a gradient allocation policy, wherein the gradient allocation policy allocates the quota for one or more clients according to a first residual space and a preset quota level, the preset quota level can be divided according to the first residual space, the first residual space comprises a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of one or more directory items, and the allocation area is a storage space occupied by the allocated quota of one or more clients;

when the target quota of the target client is valid, after the target client writes the data to be written, the metadata management node updates the target quota of the target client according to the gradient allocation policy.

In the embodiment of the application, the metadata management node can allocate different quotas for the client step by step according to the residual space size of the total quota space and the preset quota level through the gradient allocation strategy so as to limit the size of the writable space of the client, thereby restricting the overrun limit and overrun duration.

In a possible implementation manner, if the target quota of the target client is valid, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation policy, which may include if the target quota of the target client is valid and the reserved quota of the target client is greater than or equal to the space to be written, the metadata management node determines whether the total writing space is smaller than the total quota space, the reserved quota is a quota space reserved by the target client from the quota and used for storing the data to be written, the space to be written is a storage space occupied by the data to be written, the total writing space is a sum of storage spaces occupied by one or more clients, and if the total writing space is smaller than the total quota space, the metadata management node updates the target quota according to the gradient allocation policy after the target client writes the data to be written.

In the embodiment of the application, when the reserved quota of the client is enough, before updating the quota for the client, the metadata management node can judge whether the metadata server is overrun or not, namely whether the total writing space is smaller than the total quota space, so that the metadata server can be restrained in time when overrun.

In a possible implementation manner, if the target quota of the target client is valid, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation policy, and may include if the target quota of the target client is valid and the reserved quota of the target client is smaller than the space to be written, the metadata management node determines whether the total remaining space is greater than or equal to the space to be written, the total remaining space includes a storage space in which the data has been written in the total quota space removal allocation area, if the total remaining space is greater than or equal to the space to be written, the metadata management node determines whether the total writing space is smaller than the total quota space, and if the total writing space is smaller than the total quota space, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation policy.

In the embodiment of the application, when the reserved quota of the client is insufficient, the metadata management node can update the quota of the client according to the total residual space, and when the total residual space is enough and the total writing space is smaller than the total quota space, the metadata management node can acquire the new quota to perform the related operation of reading and writing data.

In one possible implementation, the quota allocation method may further include the metadata management node sending stop information to the target client when the total write space is greater than or equal to the total quota space, the stop information being used to instruct the target client to stop writing data.

In the embodiment of the application, after the data is written in the client, the metadata management node can judge the total writing space and the total quota space, and when the total writing space is larger than the total quota space, stop information is timely sent to the client so that the client stops writing the data to restrict the overrun quota of the client.

In one possible implementation manner, if the reserved quota of the target client is smaller than the space to be written, the quota allocation method may further include that if the second remaining space is greater than or equal to the space to be written, the metadata management node allocates the second remaining space to the client, wherein the second remaining space comprises a remaining space of an allocation area and an overflow buffer area, the remaining space of the allocation area comprises a storage space of the allocation area from which the written data is removed, the overflow buffer area comprises a storage space reserved according to the number of one or more clients in the total quota space, if the second remaining space is smaller than the space to be written, the metadata management node judges whether the total remaining space is greater than or equal to the space to be written, if the total remaining space is greater than or equal to the space to be written, the metadata management node recalls the allocated quota of one or more clients, if the recall is successful, the metadata management node calculates the quota of the one or more clients according to a gradient allocation policy and allocates the quota, and if the recall fails, the metadata management node returns the quota to the target client to the insufficient.

In the embodiment of the application, when the reserved quota of the client is smaller than the space to be written, namely the reserved quota is insufficient, a centralized management mode can be adopted, and the second residual space and the total residual space are judged gradually, when the second residual space is insufficient and the total residual space is sufficient, one or more unused quota of the client is recalled first by adopting a quota recall mode, and then the quota is redistributed to all the clients, so that each client can acquire the quota according to the residual space, and further the total quota space overrun of the metadata server can be avoided.

In one possible implementation, the foregoing recall, by the metadata management node, the allocated quota in the one or more clients if the total remaining space is greater than or equal to the space to be written, may include deleting, by the metadata management node, metadata of the one or more clients if the total remaining space is greater than or equal to the space to be written.

In a possible implementation manner, if the target quota of the target client fails, the metadata management node updates the target quota according to the gradient allocation policy, which may include determining, by the metadata management node, the size of the first remaining space and the overflow buffer if the target quota of the target client fails, updating, by the metadata management node, the target quota according to the gradient allocation policy if the first remaining space is smaller than the overflow buffer, and allocating, by the metadata management node, the target quota to the target client if the first remaining space is greater than or equal to the overflow buffer.

In a second aspect, the application provides a quota allocation method applied to a target client in a distributed file system, wherein the distributed file system comprises one or more directory entries, one or more clients and a metadata management node, the one or more directory entries are used for storing data written by the one or more clients, and the metadata management node is used for allocating quota to the one or more clients, and the method comprises the steps that the target client sends a writing request, and the writing request carries data to be written; if the target quota of the target client fails, the target client acquires an updated target quota, the updated target quota is obtained by updating the target quota according to a gradient allocation policy by a metadata management node, the gradient allocation policy allocates the quota to one or more clients based on a first residual space and a preset quota level by the metadata management node, the first residual space comprises a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of one or more directory entries, and the allocation area is a storage space occupied by the quota allocated by one or more clients;

And if the target quota of the target client is valid, the target client writes the data to be written.

In the embodiment of the application, when the reserved quota is insufficient, the client can write data according to the total residual space of the directory entry, thereby fully utilizing the quota space and avoiding the condition that the total quota is enough but writing fails.

In a possible implementation manner, if the target quota of the target client is valid, the target client writing the data to be written may include that the target client determines whether the reserved quota is greater than or equal to a space to be written, the reserved quota reserves a quota space for storing the data to be written from the quota for the target client, the space to be written is a storage space occupied by the data to be written, if the reserved quota is greater than or equal to the space to be written, the target client writes the data to be written, if the reserved quota is less than the space to be written, and the total residual space is greater than or equal to the space to be written, the target client writes the data to be written, and the total residual space is a storage space of the written data in the total quota space removing allocation area.

In a possible implementation manner, when the reserved quota is smaller than the space to be written, the method further comprises the steps that if the second residual space is larger than or equal to the space to be written, the target client obtains updated target quota, the second residual space comprises residual space of the allocation area and an overflow buffer area, the residual space of the allocation area comprises storage space of the allocation area for removing written data, the overflow buffer area comprises storage space reserved according to the number of one or more clients in the total quota space, and if the second residual space is smaller than the space to be written and the total residual space is larger than or equal to the space to be written, the target client obtains updated target quota after the metadata management node recalls the allocated quota of the one or more clients.

In a third aspect, the present application provides a quota allocation apparatus, the apparatus being applied to a metadata management node in a distributed file system, the distributed file system further including one or more directory entries and one or more clients, the one or more directory entries being used for storing data written by the one or more clients, the metadata management node being used for allocating a quota to the one or more clients, including:

The allocation module is used for allocating a target quota for the target client by the metadata management node;

The system comprises an updating module, a metadata management node, a first storage space and a second storage space, wherein the updating module is used for updating a target quota according to a gradient allocation strategy when the target quota of a target client fails, the gradient allocation strategy is used for indicating the metadata management node to allocate the quota for one or more clients based on a first residual space and a preset quota level, the preset quota level is divided according to the first residual space, the first residual space comprises a storage space with a total quota space removed from an allocation area, the total quota space is a total storage space of one or more catalog items, and the allocation area is a storage space occupied by the allocated quota of one or more clients;

the updating module is further configured to update the target quota according to the gradient allocation policy by the metadata management node after the target client writes the data to be written when the target quota of the target client is valid.

In a possible implementation manner, the updating module is specifically configured to, if a target quota of the target client is valid and a reserved quota of the target client is greater than or equal to a to-be-written space, determine whether a total written space is smaller than the total quota space, reserve the quota for the target client to reserve a quota space storing data to be written from the quota, the to-be-written space is a storage space occupied by the data to be written, the total written space is a sum of storage spaces occupied by write data of one or more clients, and if the total written space is smaller than the total quota space, update the target quota by the metadata management node according to a gradient allocation policy after the target client writes the data to be written.

In a possible implementation manner, the updating module is specifically configured to, if a target quota of the target client is valid and a reserved quota of the target client is smaller than a space to be written, determine whether a total remaining space is greater than or equal to the space to be written, wherein the total remaining space is a storage space in which data has been written in a total quota space removal allocation area, determine whether a total writing space is smaller than the total quota space if the total remaining space is greater than or equal to the space to be written, and update the target quota according to a gradient allocation policy after the target client writes the data to be written if the total writing space is smaller than the total quota space.

In one possible implementation, the device further comprises a sending module, configured to send, when the total writing space is greater than or equal to the total quota space, stop information to the target client, where the stop information is used to instruct the target client to stop writing data.

In one possible implementation manner, if the reserved quota of the client is smaller than the space to be written, the apparatus further includes:

The metadata management node is used for allocating a second residual space to a target client when the second residual space is larger than or equal to the space to be written, wherein the second residual space comprises a residual space of an allocation area and an overflow buffer area, the residual space of the allocation area comprises a storage space of the allocation area for removing written data, and the overflow buffer area comprises a storage space reserved according to the number of one or more clients in the total quota space;

The judging module is used for judging whether the total residual space is larger than or equal to the space to be written when the second residual space is smaller than the space to be written;

a recall module, configured to recall, when the total remaining space is greater than or equal to the space to be written, the quota allocated by the one or more clients by the metadata management node;

The processing module is used for calculating and distributing quota of one or more clients by the metadata management node according to the gradient distribution strategy when recall is successful;

The processing module is further used for returning insufficient quota to the target client by the metadata management node when recall fails.

In one possible implementation, the recall module is specifically configured to delete metadata of one or more clients by the metadata management node if the total remaining space is greater than or equal to the space to be written.

In one possible implementation, the updating module is specifically configured to:

if the target quota of the target client fails, the metadata management node judges the sizes of the first residual space and the overflow buffer area;

If the first residual space is smaller than the overflow buffer zone, the metadata management node updates the target quota according to the gradient allocation strategy;

and if the first residual space is greater than or equal to the overflow buffer zone, the metadata management node allocates a target quota for the target client.

In a fourth aspect, an embodiment of the present application provides a quota allocation apparatus, where the apparatus is applied to a target client in a distributed file system, and the distributed file system includes one or more directory entries, one or more clients, and a metadata management node, where the one or more directory entries are used to store data written by the one or more clients, and the metadata management node is used to allocate a quota for the one or more clients, where the quota allocation apparatus includes:

the sending module is used for sending a writing request by the target client, wherein the writing request carries data to be written;

The system comprises an acquisition module, a gradient allocation policy and a metadata management node, wherein the acquisition module is used for acquiring an updated target quota when the target quota of a target client is invalid, the updated target quota is obtained by updating the target quota by the metadata management node according to the gradient allocation policy, the gradient allocation policy is used for indicating the metadata management node to allocate the quota for one or more clients based on a first residual space and a preset quota level, the first residual space comprises a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of one or more catalog items, and the allocation area is a storage space occupied by the quota allocated by one or more clients;

and the writing module is used for writing the data to be written into by the target client when the target quota of the target client is valid.

In a possible implementation manner, the writing module is specifically configured to determine whether the reserved quota is greater than or equal to a space to be written, where the reserved quota is a quota space reserved by the target client for storing data to be written from the quota, the space to be written is a storage space occupied by the data to be written, if the reserved quota is greater than or equal to the space to be written, the target client writes the data to be written, and if the reserved quota is less than the space to be written and the total remaining space is greater than or equal to the space to be written, the target client writes the data to be written, and the total remaining space is a storage space of the written data in the total quota space removal allocation area.

In a possible implementation manner, when the reserved quota is smaller than the space to be written, the device further comprises an acquisition module, a storage module and a storage module, wherein the acquisition module is used for acquiring the updated target quota by the target client when a second residual space is larger than or equal to the space to be written, the second residual space comprises a residual space of an allocation area and an overflow buffer area, the residual space of the allocation area comprises a storage space of the allocation area for removing written data, and the overflow buffer area comprises a storage space reserved according to the number of one or more clients in the total quota space;

The obtaining module is further configured to, when the second remaining space is smaller than the space to be written, and the total remaining space is greater than or equal to the space to be written, recall, at the metadata management node, the quota allocated by the one or more clients, and obtain, by the target client, the updated target quota.

In a fifth aspect, the present application provides a cluster of computing devices, the cluster of computing devices comprising at least one computing device, each computing device comprising a processor and a memory, the processor of the at least one computing device being operable to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform a method as described in the first aspect.

In a sixth aspect, the present application provides a cluster of computing devices comprising at least one computing device, each computing device comprising a processor and a memory, the processor of the at least one computing device being operable to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform a method as described in the second aspect.

In a seventh aspect, the application provides a computer program product comprising instructions which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method according to the first aspect.

In an eighth aspect, the application provides a computer program product comprising instructions which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method according to the second aspect.

In a ninth aspect, the application provides a computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform a method according to the first aspect.

In a tenth aspect, the application provides a computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform a method according to the second aspect.

Drawings

FIG. 1 is a schematic diagram of a system architecture according to the present application;

FIG. 2 is a flow chart of a quota allocation method provided by the application;

FIG. 3 is a flow chart of another quota allocation method provided by the present application;

FIG. 4 is a flow chart of another quota allocation method provided by the application;

FIG. 5 is a flow chart of another quota allocation method provided by the application;

FIG. 6 is a schematic structural diagram of a quota dispensing device provided by the present application;

FIG. 7 is a schematic diagram of another quota dispensing device provided by the present application;

FIG. 8 is a schematic diagram of a computing device according to the present application;

FIG. 9 is a schematic diagram of another computing device according to the present application;

FIG. 10 is a schematic diagram illustrating a computing device cluster in accordance with the present application;

FIG. 11 is a schematic diagram illustrating another architecture of a computing device cluster according to the present application;

FIG. 12 is a schematic diagram of another computing device cluster in accordance with the present application;

Fig. 13 is another schematic structural diagram of another computing device cluster according to the present application.

Detailed Description

The following description of the technical solutions according to the embodiments of the present application will be given with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.

The embodiment of the application can be applied to one or more schemes of disk quota, directory quota or user group quota, and the embodiment is not limited to the scheme.

The system framework on which the embodiments of the present application are based is described below.

Referring to fig. 1, an embodiment of the present application provides a system framework that is a distributed file system framework, where the distributed file system includes a metadata service cluster 101, at least one client 102, and a storage service cluster 103, where the metadata service cluster 101 includes at least one metadata management node, and the storage service cluster 103 includes at least one directory entry for storing data.

In the embodiment of the application, the total storage space of one directory entry can be divided into a pre-allocation area and an overflow buffer area, the pre-allocation area can be divided into an allocation area and an unallocated area, the allocation area is a storage space occupied by a quota which is allocated to a client in advance by a metadata management node, the metadata management node can allocate the quota to the client according to the data storage condition in the directory entry, can allocate the quota to the client according to the quota cache time of the client, or can send a stop message to the client according to the data storage condition in the directory entry so that the client stops writing data, and the client can finish reading and writing of the data based on the quota allocated by the metadata management node or stop writing of the data according to the stop message sent by the metadata management node.

The existing scheme mainly comprises the steps that the metadata management node distributes and updates the client quota periodically or distributes and updates the client quota in an equal division mode. The metadata management node periodically updates the quota of the client, the client passively updates at regular time, whether the total quota of the disk reaches the maximum quota cannot be perceived, and data writing is performed only according to whether the quota is overtime, so that the total quota of the disk is overrun, and the excess cannot be controlled. For the scheme of carrying out quota update by adopting the equal division mode, the remaining quota is evenly distributed to each client according to the number of the clients and the remaining quota, and in the scheme, the clients still cannot sense the total quota of the disk, overrun can be caused, and when the quota is enough, some clients cannot write data because the quota is not distributed according to the actual situation of the clients, so that the quota of some clients is insufficient.

In order to solve the existing problems, the embodiment of the application provides a quota allocation method, and a data management node can allocate quota to one or more clients by adopting a gradient allocation strategy so as to limit the size of a writing space of the client and restrict the overrun quota. In addition, when the reserved quota of the client is insufficient, the data to be written can be written according to the size of the total residual space, so that the quota space is fully utilized, and the situation that the total quota is enough but writing fails is avoided.

The method flow of the embodiment of the present application is described below in conjunction with the foregoing system framework.

Participating in fig. 2, a flow chart of a quota allocation method provided by the application is described below.

Step 201, a metadata management node allocates a target quota for a target client;

the metadata management node may allocate a target quota for one or more target clients in advance, so that the target clients may read and write data according to the obtained target quota.

Step 202, the target client judges whether the target quota is invalid;

After obtaining the target quota, the target client may determine whether the target quota is invalid, execute the next operation according to the determination result, execute step 204, step 205, and step 206 if the target quota is invalid, and execute step 207 and step 208 if the target quota is valid.

The quota failure is that the use and management of the quota exceeds the valid period, and the quota effective is that the use and management of the quota does not exceed the valid period. The effective time limit and the allocation time length of the quota can be obtained, the allocation time length is the use and management time length of the quota, and whether the quota is invalid or not is judged by comparing the allocation time length and the effective time limit.

For example, when the target client writes data, whether the target quota is invalid or not will be judged first, the allocation duration and the valid period of the target quota can be obtained, if the allocation duration is 130s and the valid period of the target quota is 120s, the target quota is judged to be invalid, and if the allocation duration is 80s and the valid period of the target quota is 120s, the target quota is judged to be valid.

Step 203, the target client sends information to the metadata management node;

After the target client obtains the judgment result, the judgment result can be carried in the information, and the information is sent to the metadata management node, so that the metadata management node can execute the next operation according to the information.

Step 204, if the target quota is invalid, the metadata management node updates the target quota according to the gradient allocation policy;

When the target quota is invalid, the metadata management node may update the target quota of the target client according to the gradient allocation policy after receiving the query request sent by the target client.

The gradient allocation policy allocates quota for one or more clients based on a first remaining space and a preset quota level, the preset quota level can be divided according to the first remaining space, the first remaining space comprises a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of one or more directory entries, and the allocation area is a storage space occupied by the quota allocated by one or more clients. For example, after the metadata management node allocates a quota for one or more clients, at this time, the first remaining space is 5G, the preset quota level may set different quotas, such as 2G, 200M, 20M, and 2M, according to the preset proportion and the size of the first remaining space, and the target quota may be updated to 2G according to the first remaining space and the preset quota level.

Step 205, the metadata management node sends the updated target quota to the target client;

after the updated target quota is obtained, the metadata management node sends the updated target quota to the target client, so that the target client can read and write data according to the updated target quota, and quota adjustment and allocation according to the size of the residual space are achieved.

Step 206, if the target quota is invalid, the target client acquires the updated target quota;

when the target quota is invalid, the target client can send a query request to the metadata management node, and the metadata management node updates the quota for the target client according to the query request and sends the updated target quota to the target client.

Step 207, if the target quota is valid, the target client writes the data to be written;

When the target quota is valid, the target client can directly write the data to be written under the condition that the reserved quota is enough, and if the reserved quota is insufficient, but the total residual space is enough, the data to be written can also be directly written, and the total residual space is the storage space of the written data in the total quota space removing allocation area.

Step 208, if the target quota is valid, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation policy.

After the target quota is valid and the target client writes data, the metadata management node can update the target quota for the target client according to the gradient allocation policy, and timely adjust the quota of the client according to the size of the first residual space and a preset quota level, so that the size of the writable space of the client can be limited, and the quota when the quota is over-limited can be constrained.

Based on the quota allocation method described in fig. 2, a specific implementation manner of the quota allocation method provided by the application is described below.

Referring to fig. 3, a flow chart of another quota allocation method provided by the present application is as follows.

Step 301, the client judges whether the allocation duration of the first quota of the client exceeds the first effective duration of the first quota;

After the client obtains the quota, the first effective duration (may also be referred to as a first effective duration) and the allocation duration, it may be determined whether the allocation duration of the quota is greater than the first effective duration (may also be determined whether the quota of the client is invalid), and the subsequent client and the metadata management node may execute the next operation according to the determination result, if the allocation duration of the first quota is greater than the first effective duration, that is, the first quota of the client is invalid, step 305 is executed, and if the allocation duration of the first quota is less than or equal to the first effective duration, that is, the first quota of the client is valid, step 306 is executed.

Typically, each client has a valid period of quota, if the allocated duration of the first quota of the client exceeds the first valid duration of the first quota, the metadata management node will reallocate the quota to the client, and the client will also reacquire the quota and the valid duration.

Step 302, the client sends first information to the metadata management node;

after obtaining the judgment result of the first allocation duration and the effective duration of the client, the client can carry the judgment result in first information and send the first information to the metadata management node, and the metadata management node can execute the next operation according to the judgment result.

Step 303, if the allocation duration of the first quota exceeds the first effective duration, the metadata management node updates the first quota according to the gradient allocation policy to obtain a second quota and a second effective duration of the second quota;

When the judgment result carried in the first information is that the allocation duration of the first quota exceeds the first effective duration, the metadata management node can update the first quota according to the gradient allocation policy to obtain an updated second quota and the second effective duration of the second quota.

Specifically, when the allocation duration of the first quota of the client exceeds the first effective duration, the metadata management node may determine the size of the first remaining space and the overflow buffer area first, where the first remaining space may include a total quota space that removes remaining storage space of the allocated area, where the total quota space includes a total storage space of directory entries, the allocation area includes storage space occupied by a quota allocated by the metadata management node in advance for each client, and the overflow buffer area may include storage space reserved in the total quota space according to the number of clients, that is, the storage space of the overflow buffer area may be divided according to the size of the total quota space and a preset quota level. The preset quota level is a quota set of different quota space sizes set according to the first residual space and the preset proportion, and the preset quota level can be an arithmetic sequence, an arithmetic sequence or a combination meeting other preset proportions. For example, the number of clients is 3, the preset quota level is divided into 1G, 100M, 10M and 1M, and if the total quota space is greater than 1G, the storage space size of the overflow buffer may be divided into 3×1g=3g.

If the first remaining space is greater than or equal to the overflow buffer, the metadata management node may allocate a first quota for the client as a quota for the client, i.e., without recalculating the quota size, may allocate the original quota size to the client; if the first remaining space is smaller than the overflow buffer, the metadata management node may update the first quota according to a gradient allocation policy to obtain a second quota and a second effective duration.

The metadata management node updates the first quota according to the gradient allocation policy, and may reallocate the quota for the client according to the size of the first remaining space and a preset quota level, for example, when the first remaining space includes a 3G storage space, a 1G quota may be allocated for the client according to the preset quota level (1G, 100M, 10M, and 1M), and when the first remaining space includes a 0.1G storage space, a 100M quota may be allocated for the client.

Step 304, the metadata management node sends a second quota to the client for a second effective duration;

After the second quota and the second effective duration are obtained, the metadata management node may send the second quota and the second effective duration to the client, so that the quota of the client may be updated immediately, and the overrun duration of the client may be constrained by setting the second effective duration.

Step 305, if the allocation duration of the first quota exceeds the first effective duration, the client acquires a second quota and a second effective duration;

After judging that the allocation duration exceeds the first effective duration, the client can acquire the quota, namely the second quota and the second effective duration of the second quota, from the metadata management node again, and update the original quota so as to facilitate subsequent data writing.

Step 306, if the allocation duration of the first quota does not exceed the first effective duration, the client judges whether the reserved quota is greater than or equal to the space to be written;

when the allocation duration of the first quota does not exceed the first effective duration, the client may determine the reserved quota and the size of the space to be written, so as to determine whether the data to be written can be directly written, where the reserved quota includes a quota space reserved by the client from the allocated quota to store the data to be written, the space to be written includes a storage space occupied by the data to be written, if the reserved quota is greater than or equal to the space to be written, step 307 is executed, and if the reserved quota is less than the space to be written, step 308 is executed.

Step 307, if the allocation duration of the first quota does not exceed the first effective duration and the reserved quota is greater than or equal to the space to be written, the client writes the data to be written and updates the metadata;

When the allocation duration of the first quota of the client does not exceed the first effective duration, the client can judge whether the reserved quota is enough or not, and when the reserved quota is greater than or equal to the space to be written, the client can directly write the data to be written and update the metadata.

Step 308, if the reserved quota is smaller than the space to be written, the metadata management node judges whether the total remaining space is larger than or equal to the space to be written;

on the premise that the allocation duration of the first quota does not exceed the first effective duration, when the reserved quota of the client is insufficient, the metadata management node may determine whether the total remaining space is sufficient, that is, determine whether the total remaining space is greater than or equal to the space to be written, where the total remaining space includes a storage space in which data has been written in the allocation region is removed from the total quota space, if the total remaining space is less than the space to be written, step 309 is executed, and if the total remaining space is greater than or equal to the space to be written, step 310 is executed.

Step 309, if the total remaining space is smaller than the space to be written, the metadata management node returns insufficient quota;

when the total remaining space is smaller than the space to be written, the metadata management node returns insufficient quota, namely the remaining space of the directory entry is insufficient to store the data to be written, so that the client can stop writing the data.

Step 310, if the allocation duration of the first quota does not exceed the first effective duration, the reserved quota is smaller than the space to be written, and the total remaining space is greater than or equal to the space to be written, the client writes the data to be written, and updates the metadata.

When the total remaining space is sufficient, the client may write the data to be written and update the metadata. Under the condition that the reserved quota is insufficient, the client can write the data to be written by judging whether the total residual space is enough, so that the situation that the total quota is enough and writing fails is avoided.

In the embodiment of the application, a gradient allocation strategy can be adopted, and different preset quota levels are selected to allocate quota for the client according to the size of the storage space after the allocated area is removed from the first residual space, namely the total quota space, so that the size of writable space of the client is limited, and the overrun quota is restrained. In addition, when the reserved quota is insufficient, the client can write the data to be written when the total residual space is enough, so that the quota space is fully utilized, and the situation that the total quota is enough but writing fails is avoided.

Referring to fig. 4, a flow chart of another quota allocation method provided by the present application is as follows.

Step 401, a client sends a quota query request to a metadata management node;

the client can query the quota size, the effective duration of the quota and the allocation duration of the quota by sending a quota query request to the metadata management node.

Step 402, the metadata management node sends a query response to the client;

after receiving the query request of the client, the metadata management node may send a query response to the client, where the response carries a quota size of the client, an effective duration of the quota, and an allocation duration, where the allocation duration may be an accumulated duration of the quota obtained by accumulating the allocation of the client by a timer.

Step 403, the client determines whether the allocation duration of the first quota of the client exceeds the first effective duration of the first quota;

step 404, the client sends first information to the metadata management node;

Step 405, if the allocation duration of the first quota exceeds the first effective duration, the metadata management node updates the first quota according to the gradient allocation policy to obtain a second quota and a second effective duration of the second quota;

step 406, the metadata management node sends a second quota to the client for a second effective duration;

Step 407, if the allocation duration of the first quota exceeds the first effective duration, the client acquires a second quota and a second effective duration;

Step 408, if the allocation duration of the first quota does not exceed the first effective duration, the client judges whether the reserved quota is greater than or equal to the space to be written;

Step 409, if the allocation duration of the first quota does not exceed the first effective duration and the reserved quota is greater than or equal to the space to be written, writing the data to be written into by the client, and updating the metadata;

step 410, if the reserved quota is smaller than the space to be written, the metadata management node determines whether the total remaining space is greater than or equal to the space to be written;

step 411, if the total remaining space is smaller than the space to be written, the metadata management node returns insufficient quota;

Step 412, if the allocation duration of the first quota does not exceed the first effective duration, the reserved quota is smaller than the space to be written and the total remaining space is greater than or equal to the space to be written, the client writes the data to be written, and updates the metadata;

In this embodiment, steps 403 to 412 are similar to steps 301 to 310 in the embodiment shown in fig. 3, and detailed descriptions thereof are omitted herein.

Step 413, the metadata management node judges whether the total writing space is smaller than the total quota space;

After the client writes the data to be written, the metadata management node may determine whether the total writing space is overrun or not, that is, whether the total writing space is greater than or equal to the total quota space, by determining the size of the total writing space and the total quota space, where the total writing space includes a sum of storage spaces occupied by each client for writing the data to be written, if the total writing space is greater than or equal to the total quota space, step 414 is executed, and if the total writing space is less than the total quota space, step 415 is executed.

Step 414, if the total writing space is greater than or equal to the total quota space, the metadata management node sends information for stopping writing to the client;

If the total writing space is greater than or equal to the total quota space, the metadata management node may send information of stopping writing to the client, so that the client stops writing the data to be written, thereby limiting the overrun quota.

Step 415, if the total writing space is smaller than the total quota space, the metadata management node updates the first quota according to the gradient allocation policy;

on the premise that the allocation duration of the first quota does not exceed the first effective duration, the total writing space is smaller than the total quota space, that is, the total writing space is not exceeded, the metadata management node can reallocate the quota for the client according to the gradient allocation policy, and a specific process of allocating the quota for the client by adopting the gradient allocation policy is described in step 405, which is not repeated herein.

Step 416, the metadata management node sends the updated first quota to the client.

After obtaining the updated first quota, the metadata management node may send the first quota to the corresponding client, so that the client may perform a subsequent data reading related operation according to the first quota.

In the embodiment of the application, after the client writes the data, the metadata management node can also judge the overrun of the metadata server by judging the sizes of the total writing space and the total quota space, and when the total writing space is larger than or equal to the total quota space after the client writes the data, the metadata management node can send the stop information to stop the client from writing the data, so that the client can be stopped in time when overrun occurs.

The embodiment of the application can be applied to quota management in an approximately accurate scene, can refer to the steps in the foregoing fig. 4, can be applied to quota management in an absolute accurate scene on the premise that the reserved quota of the client is insufficient, can be switched between an approximately accurate quota management scheme and an absolute accurate quota management scheme through an out-of-band Application Program Interface (API), and is exemplarily introduced below by taking the flow of the absolute accurate quota management as an example.

Referring to fig. 5, a flow chart of another quota allocation method provided by the present application is as follows.

Step 501, a client sends a quota query request to a metadata management node;

step 502, the metadata management node sends a query response to the client;

step 503, the client judges whether the allocation duration of the first quota of the client exceeds the first effective duration of the first quota;

step 504, the client sends first information to the metadata management node;

step 505, if the allocation duration of the first quota exceeds the first effective duration, the metadata management node updates the first quota according to the gradient allocation policy to obtain a second quota and a second effective duration of the second quota;

step 506, the metadata management node sends a second quota to the client for a second effective duration;

step 507, if the allocation duration of the first quota exceeds the first effective duration, the client acquires a second quota and the second effective duration;

step 508, if the allocation duration of the first quota does not exceed the first effective duration, the client judges whether the reserved quota is greater than or equal to the space to be written;

In this embodiment, steps 501 to 508 are similar to steps 401 to 408 in the embodiment shown in fig. 4, and are not repeated here.

Step 509, if the allocation duration of the first quota does not exceed the first effective duration and the reserved quota is greater than or equal to the space to be written, the metadata management node updates the first quota according to the gradient allocation policy;

When the first quota buffer time of the client is not overtime and the reserved quota of the client is enough, the metadata management node may update the first quota according to the gradient allocation policy, and a specific process of allocating the quota for the client by adopting the gradient allocation policy is described in step 405, which is not repeated herein.

Step 510, the metadata management node sends the updated first quota to the client;

After obtaining the updated first quota, the metadata management node may send the updated quota to a corresponding client, and the client may perform a subsequent data reading related operation according to the updated first quota.

Step 511, if the allocation duration of the first quota does not exceed the first effective duration and the reserved quota is greater than or equal to the space to be written, writing the data to be written by the client and updating the metadata;

step 512, if the allocation duration of the first quota does not exceed the first effective duration and the reserved quota is smaller than the space to be written, the metadata management node determines whether the second remaining space is greater than or equal to the space to be written;

Optionally, when the allocation duration of the first quota does not exceed the first effective duration and the reserved quota is smaller than the space to be written, it may be queried whether the second remaining space is sufficient, where the second remaining space includes a remaining space of the allocation region and an overflow buffer region, and the remaining space of the allocation region includes a remaining storage space after the allocation region removes the storage space of the written data, if the second remaining space is greater than or equal to the space to be written, step 513 is executed, and if the second remaining space is smaller than the space to be written, step 514 is executed.

Step 513, if the second remaining space is greater than or equal to the space to be written, the metadata management node allocates the second remaining space to the client;

alternatively, when the second remaining space is greater than or equal to the space to be written, the metadata management node may allocate the second remaining space to the client, and the client may write the data to be written into the second remaining space.

Step 514, if the second remaining space is smaller than the space to be written, the metadata management node determines whether the total remaining space is greater than or equal to the space to be written;

Optionally, when the second remaining space is smaller than the space to be written, the metadata management node may query whether the total remaining space is sufficient, that is, determine whether the total remaining space is greater than or equal to the space to be written, where the total remaining space includes a remaining storage space after the storage space of the data written in the total quota space is removed from the allocation area, execute step 515 if the total remaining space is smaller than the space to be written, and execute step 516 if the total remaining space is greater than or equal to the space to be written.

Step 515, if the total remaining space is smaller than the space to be written, the metadata management node returns insufficient quota;

Step 516, if the total remaining space is greater than or equal to the space to be written, the metadata management node recalls the allocated quota in each client;

when the total remaining space is greater than or equal to the space to be written, the metadata management node may recall the unoccupied portion of the quota allocated to each client first, execute step 517 if the recall is successful, and execute step 519 if the recall is failed.

Specifically, the original metadata management node may be invalidated by writing code such that the quota assigned to the client is invalidated, thereby recalling the quota for each client.

Step 517, if recall is successful, the metadata management node calculates the quota of each client;

When recall is successful, the metadata management node may recalculate the quota for each client according to the gradient allocation policy.

According to the embodiment of the application, when the total residual space is enough, unused quota of each client can be recalled, and the quota is redistributed for each client by adopting a gradient distribution strategy, so that the quota of each client can be updated according to the total residual space, and therefore, the condition that the quota is overrun due to the fact that each client writes data according to the updated quota is ensured.

518, The metadata management node distributes the calculated quota of the client to the corresponding client;

after recalculating the quota of each client, the metadata management node may allocate the quota corresponding to each client, so that the client may successfully write the data to be written.

In step 519, if the recall fails, the metadata management node returns a quota insufficient.

Wherein, when recalling the quota of each client, the metadata management node, if there is a client writing data at this time, then the recall will fail and when the recall fails, the metadata management node may return a quota deficit such that the client stops writing data.

In the embodiment of the application, the client can gradually judge whether the second residual space (comprising the available space of the allocation area and the available space of the overflow buffer area) and the total residual space (comprising the available space of the allocation area, the unallocated area and the available space of the overflow buffer area) are enough or not under the condition of insufficient reserved quota, allocate the quota for the client, when the second residual space is enough, the space can be allocated to the client, when the second residual space is insufficient but the total residual space is enough, the unused quota of each client can be recalled first, if the recall is successful, the metadata management node can recalculate and allocate the quota for each client according to a gradient allocation strategy, if the recall is failed, namely, the client is writing data at the moment, the returned quota is insufficient, and the overrun caused by writing data by a plurality of clients at the same time is avoided.

The foregoing describes a method flow provided by the present application, and the following describes an apparatus provided by the present application based on the foregoing method flow.

Referring to fig. 6, a schematic structural diagram of a quota allocation apparatus 600 provided in the present application includes:

An allocation module 601, configured to allocate a target quota for a target client by using a metadata management node;

The updating module 602 is configured to update, when a target quota of a target client fails, the target quota according to a gradient allocation policy, where the gradient allocation policy is used to instruct the metadata management node to allocate a quota for one or more clients based on a first remaining space and a preset quota level, the preset quota level is divided according to the first remaining space, the first remaining space includes a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of one or more directory entries, and the allocation area is a storage space occupied by one or more allocated quotas of the clients;

The updating module 602 is further configured to update, when the target quota of the target client is valid, the target quota according to the gradient allocation policy by the metadata management node after the target client writes the data to be written.

In a possible implementation manner, the updating module 602 is specifically configured to, if the target quota of the target client is valid and the reserved quota of the target client is greater than or equal to the to-be-written space, determine whether the total written space is smaller than the total quota space, reserve the quota space for storing the to-be-written data from the quota by the target client, where the to-be-written space is a storage space occupied by the to-be-written data, and the total written space is a sum of storage spaces occupied by the to-be-written data of one or more clients, and if the total written space is smaller than the total quota space, update the target quota by the metadata management node according to the gradient allocation policy after the to-be-written data is written by the target client.

In a possible implementation manner, the updating module 602 is specifically configured to, if the target quota of the target client is valid and the reserved quota of the target client is smaller than the space to be written, determine whether the total remaining space is greater than or equal to the space to be written, where the total remaining space is a storage space in which data has been written in the total quota space removal allocation area, determine whether the total writing space is smaller than the total quota space if the total remaining space is greater than or equal to the space to be written, and update the target quota according to the gradient allocation policy after the target client writes the data to be written if the total writing space is smaller than the total quota space.

In a possible implementation manner, the device further comprises a sending module 603, configured to send, when the total writing space is greater than or equal to the total quota space, stop information to the target client, where the stop information is used to instruct the target client to stop writing data.

the allocation module 601 is further configured to allocate, when the second remaining space is greater than or equal to the space to be written, the second remaining space to the target client, where the second remaining space includes a remaining space of the allocation area and an overflow buffer area, where the remaining space of the allocation area includes a storage space of the allocation area from which the written data is removed, and the overflow buffer area includes a storage space reserved according to the number of one or more clients in the total quota space;

a judging module 604, configured to judge whether the total remaining space is greater than or equal to the space to be written when the second remaining space is less than the space to be written;

a recall module 605, configured to recall, when the total remaining space is greater than or equal to the space to be written, the quota allocated by the one or more clients by the metadata management node;

The processing module 606 is configured to, when the recall is successful, calculate a quota of one or more clients according to the gradient allocation policy by the metadata management node and allocate the quota;

The processing module 606 is further configured to, when the recall fails, return a quota deficient to the target client by the metadata management node.

In one possible implementation, the recall module 605 is specifically configured to delete metadata of one or more clients by the metadata management node if the total remaining space is greater than or equal to the space to be written.

In one possible implementation, the updating module 602 is specifically configured to:

The distribution module, the updating module, the sending module, the judging module, the recall module and the processing module can be realized by software or hardware. Illustratively, the implementation of the allocation module is described next as an example of an allocation module. Similarly, the implementation manners of the updating module, the sending module, the judging module, the recall module and the processing module can refer to the implementation manner of the distributing module.

Module as an example of a software functional unit, the allocation module may comprise code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container, among others. Further, the above-described computing examples may be one or more. For example, the allocation module may include code running on multiple hosts/virtual machines/containers. It should be noted that, multiple hosts/virtual machines/containers for running the code may be distributed in the same region (region), or may be distributed in different regions. Further, multiple hosts/virtual machines/containers for running the code may be distributed in the same availability zone (availabilityzone, AZ) or may be distributed in different AZs, each AZ comprising one data center or multiple geographically close data centers. Wherein typically a region may comprise a plurality of AZs.

Also, multiple hosts/virtual machines/containers for running the code may be distributed in the same virtual private cloud (virtualprivate cloud, VPC) or may be distributed in multiple VPCs. In general, one VPC is disposed in one region, and a communication gateway is disposed in each VPC for implementing inter-connection between VPCs in the same region and between VPCs in different regions.

Modules as an example of hardware functional units, the allocation module may include at least one computing device, such as a server or the like. Or the distribution module may be a device implemented with an application-specific integrated circuit (ASIC) or a programmable logic device (programmablelogicdevice, PLD), etc. The PLD may be implemented as a complex program logic device (complexprogrammablelogicaldevice, CPLD), a field-programmable gate array (FPGA), general-purpose array logic (genericarraylogic, GAL), or any combination thereof.

The multiple computing devices included in the allocation module may be distributed in the same region or may be distributed in different regions. The multiple computing devices included in the a module may be distributed in the same AZ or may be distributed in different AZ. Likewise, multiple computing devices included in the a module may be distributed in the same VPC or may be distributed among multiple VPCs. Wherein the plurality of computing devices may be any combination of computing devices such as servers, ASIC, PLD, CPLD, FPGA, and GAL.

It should be noted that, in other embodiments, the allocation module may be used to execute any step in the quota allocation method, the update module may be used to execute any step in the quota allocation method, the send module may be used to execute any step in the quota allocation method, the recall module may be used to execute any step in the quota allocation method, the processing module may be used to execute any step in the quota allocation method, and the steps that the allocation module, the update module, the send module, the judge module, the recall module, and the processing module are responsible for implementing may be specified as needed, and all functions of the quota allocation device are implemented by implementing different steps in the quota allocation method respectively.

Referring to fig. 7, a schematic structural diagram of a quota allocation apparatus 700 provided in the present application includes:

a sending module 701, configured to send a write request by a target client, where the write request carries data to be written;

The obtaining module 702 is configured to, when a target quota of a target client fails, obtain an updated target quota by the target client, where the updated target quota is obtained by updating the target quota by a metadata management node according to a gradient allocation policy, where the gradient allocation policy is used to instruct the metadata management node to allocate the quota to one or more clients based on a first remaining space and a preset quota level, the first remaining space includes a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of one or more directory entries, and the allocation area is a storage space occupied by a quota allocated by one or more clients;

The writing module 703 is configured to write the data to be written by the target client when the target quota of the target client is valid.

In a possible implementation manner, the writing module 703 is specifically configured to determine whether the reserved quota is greater than or equal to a space to be written, where the reserved quota is a quota space reserved by the target client for storing data to be written from the quota, the space to be written is a storage space occupied by the data to be written, if the reserved quota is greater than or equal to the space to be written, the target client writes the data to be written, and if the reserved quota is less than the space to be written and the total remaining space is greater than or equal to the space to be written, the target client writes the data to be written, and the total remaining space is a storage space of the written data in the total quota space removal allocation area.

In a possible implementation manner, when the reserved quota is smaller than the space to be written, the device further comprises an acquisition module 702, configured to acquire the updated target quota by the target client when the second remaining space is greater than or equal to the space to be written, where the second remaining space includes a remaining space of the allocation region and an overflow buffer, where the remaining space of the allocation region includes a storage space of the allocation region from which the written data is removed, and where the overflow buffer includes a storage space reserved according to the number of one or more clients in the total quota space;

the obtaining module 702 is further configured to, when the second remaining space is smaller than the space to be written, and the total remaining space is greater than or equal to the space to be written, obtain, by the target client, the updated target quota after the metadata management node recalls the quota allocated by the one or more clients.

The sending module, the obtaining module and the writing module can be realized by software or hardware. By way of example, the implementation of the transmission module is described next. Similarly, the implementation of the acquisition module and the writing module may refer to the implementation of the sending module.

Module as an example of a software functional unit, the sending module may comprise code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container, among others. Further, the above-described computing examples may be one or more. For example, the sending module may include code running on multiple hosts/virtual machines/containers. It should be noted that, multiple hosts/virtual machines/containers for running the code may be distributed in the same region (region), or may be distributed in different regions. Further, multiple hosts/virtual machines/containers for running the code may be distributed in the same availability zone (availabilityzone, AZ) or may be distributed in different AZs, each AZ comprising one data center or multiple geographically close data centers. Wherein typically a region may comprise a plurality of AZs.

Module as an example of a hardware functional unit, the sending module may comprise at least one computing device, such as a server or the like. Or the transmit module may be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (programmablelogicdevice, PLD), etc. The PLD may be implemented as a complex program logic device (complexprogrammablelogicaldevice, CPLD), a field-programmable gate array (FPGA), general-purpose array logic (genericarraylogic, GAL), or any combination thereof.

Multiple computing devices included in the sending module may be distributed in the same region or may be distributed in different regions. The multiple computing devices included in the sending module may be distributed in the same AZ or may be distributed in different AZ. Also, the multiple computing devices included in the sending module may be distributed in the same VPC, or may be distributed in multiple VPCs. Wherein the plurality of computing devices may be any combination of computing devices such as servers, ASIC, PLD, CPLD, FPGA, and GAL.

It should be noted that, in other embodiments, the sending module may be configured to execute any step in the quota allocation method, the obtaining module may be configured to execute any step in the quota allocation method, the writing module may be configured to execute any step in the quota allocation method, the steps that the sending module, the obtaining module, and the writing module are responsible for implementing may be specified according to needs, and all functions of the quota allocation device are implemented by implementing different steps in the quota allocation method by the sending module, the obtaining module, and the writing module respectively.

The present application also provides a computing device 800. As shown in fig. 8, computing device 800 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. Communication between processor 804, memory 806, and communication interface 808 is via bus 802. Computing device 800 may be a server or a terminal device. It should be understood that the present application is not limited to the number of processors, memories in computing device 800.

Bus 802 may be a peripheral component interconnect (peripheralcomponentinterconnect, PCI) bus, or an extended industry standard architecture (extendedindustrystandardarchitecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one line is shown in fig. 8, but not only one bus or one type of bus. Bus 804 may include a pathway for transferring information among the various components of computing device 800 (e.g., memory 806, processor 804, communication interface 808).

The processor 804 may include any one or more of a central processing unit (centralprocessingunit, CPU), a graphics processor (graphics processingunit, GPU), a Microprocessor (MP), or a digital signal processor (DIGITALSIGNAL PROCESSOR, DSP).

Memory 806 may include volatile memory (volatilememory), such as random access memory (randomaccess memory, RAM). The processor 804 may also include non-volatile memory (non-volatilememory), such as read-only memory (ROM), flash memory, mechanical hard disk (HDD), or solid state disk (solidstate drive, SSD).

The memory 806 stores executable program codes, and the processor 804 executes the executable program codes to implement the functions of the foregoing allocating module, updating module, sending module, judging module, recall module, and processing module, respectively, so as to implement a quota allocating method. That is, the memory 806 has instructions stored thereon for performing the quota allocation method.

The present application also provides another computing device 900. As shown in fig. 9, computing device 900 includes a bus 902, a processor 904, a memory 906, and a communication interface 908. Communication between the processor 904, the memory 906, and the communication interface 908 is via the bus 902. Computing device 900 may be a server or a terminal device. It should be understood that the present application is not limited to the number of processors, memories in computing device 900.

Bus 902 may be a peripheral component interconnect standard (peripheralcomponentinterconnect, PCI) bus, or an extended industry standard architecture (extendedindustrystandardarchitecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one line is shown in fig. 9, but not only one bus or one type of bus. Bus 904 may include a path for transferring information between various components of computing device 900 (e.g., memory 906, processor 904, communication interface 908).

The processor 904 may include any one or more of a central processing unit (centralprocessingunit, CPU), a graphics processor (graphics processingunit, GPU), a Microprocessor (MP), or a digital signal processor (DIGITALSIGNAL PROCESSOR, DSP).

Memory 906 may include volatile memory (volatilememory), such as random access memory (randomaccess memory, RAM). The processor 904 may also include non-volatile memory (non-volatilememory), such as read-only memory (ROM), flash memory, mechanical hard disk (HDD), or solid state disk (solidstate drive, SSD).

The memory 906 has stored therein executable program code that is executed by the processor 904 to implement the functions of the foregoing transmission module, acquisition module, and writing module, respectively, to implement a quota allocation method. That is, the memory 906 has instructions stored thereon for performing the quota allocation method.

The embodiment of the application also provides a computing device cluster. The cluster of computing devices includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop, notebook, or smart phone.

As shown in fig. 10, the cluster of computing devices includes at least one computing device 800. The same instructions for performing the quota allocation method may be stored in memory 806 in one or more computing devices 800 in the cluster of computing devices.

In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also each have stored therein a portion of instructions for performing the quota allocation test method. In other words, a combination of one or more computing devices 800 may collectively execute instructions for performing a quota allocation method.

It should be noted that the memory 806 in different computing devices 800 in the computing device cluster may store different instructions for performing part of the functions of the quota allocation apparatus, respectively. That is, the instructions stored in the memory 806 of the different computing devices 800 may implement the functionality of one or more of an allocation module, an update module, a send module, a judge module, a recall module, and a process module.

In some possible implementations, one or more computing devices in a cluster of computing devices may be connected through a network. Wherein the network may be a wide area network or a local area network, etc. Fig. 11 shows one possible implementation. As shown in fig. 11, two computing devices 800A and 800B are connected by a network. Specifically, the connection to the network is made through a communication interface in each computing device. In this type of possible implementation, instructions to perform the functions of the identification module are stored in memory 806 in computing device 800A. Meanwhile, the memory 806 in the computing device 800B stores instructions for performing the functions of the allocation module, the update module, the transmission module, the judgment module, the recall module, and the processing module.

It should be appreciated that the functionality of computing device 800A shown in fig. 11 may also be performed by multiple computing devices 800. Likewise, the functionality of computing device 800B may also be performed by multiple computing devices 800.

The embodiment of the application also provides another computing device cluster. The cluster of computing devices includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop, notebook, or smart phone.

As shown in fig. 12, the cluster of computing devices includes at least one computing device 900. The same instructions for performing the quota allocation method may be stored in memory 906 in one or more computing devices 900 in the cluster of computing devices.

In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also have stored therein respective portions of instructions for performing the quota allocation test method. In other words, a combination of one or more computing devices 900 may collectively execute instructions for performing a quota allocation method.

It should be noted that the memory 906 in different computing devices 900 in the computing device cluster may store different instructions for performing part of the functions of the quota allocation apparatus. That is, the instructions stored by the memory 906 in the different computing devices 900 may implement the functionality of one or more of the sending module, the obtaining module, and the writing module.

In some possible implementations, one or more computing devices in a cluster of computing devices may be connected through a network. Wherein the network may be a wide area network or a local area network, etc. Fig. 13 shows one possible implementation. As shown in fig. 13, two computing devices 900A and 900B are connected by a network. Specifically, the connection to the network is made through a communication interface in each computing device. In this type of possible implementation, instructions to perform the functions of the identification module are stored in memory 906 in computing device 900A. Meanwhile, instructions to perform the functions of the sending module, the obtaining module, and the writing module are stored in the memory 906 in the computing device 900B.

It should be appreciated that the functionality of computing device 900A shown in fig. 13 may also be performed by multiple computing devices 900. Likewise, the functionality of computing device 900B may also be performed by multiple computing devices 900.

Embodiments of the present application also provide a computer program product comprising instructions. The computer program product may be software or a program product containing instructions capable of running on a computing device or stored in any useful medium. The computer program product, when run on at least one computing device, causes the at least one computing device to perform a quota allocation method.

The embodiment of the application also provides a computer readable storage medium. The computer readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc. The computer-readable storage medium includes instructions that instruct a computing device to perform a quota allocation method.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that the above is only a specific embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes or substitutions are covered by the scope of the present application.

Claims

1. A quota allocation method, wherein the method is applied to a metadata management node in a distributed file system, the distributed file system further comprising one or more directory entries and one or more clients, the one or more directory entries for storing data written by the one or more clients, the metadata management node for allocating quota to the one or more clients, the method comprising:

the metadata management node allocates a target quota for the target client;

If the target quota is invalid, the metadata management node updates the target quota according to a gradient allocation policy, wherein the gradient allocation policy is used for indicating the metadata management node to allocate the quota for the one or more clients based on a first residual space and a preset quota level, the preset quota level is divided according to the first residual space, the first residual space comprises a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of the one or more directory entries, and the allocation area is a storage space occupied by the allocated quota of the one or more clients;

And if the target quota is valid, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation strategy.

2. The method of claim 1, wherein the metadata management node updating the target quota according to the gradient allocation policy after the target client writes data to be written if the target quota is valid, comprising:

if the target quota is valid and the reserved quota of the target client is greater than or equal to a space to be written, the metadata management node judges whether a total writing space is smaller than the total quota space, the reserved quota is a quota space reserved by the target client from the quota and used for storing the data to be written, the space to be written is a storage space occupied by the data to be written, and the total writing space is the sum of storage spaces occupied by the data written by the one or more clients;

And if the total writing space is smaller than the total quota space, after the target client writes the data to be written, the metadata management node updates the target quota according to the gradient allocation strategy.

3. The method of claim 1, wherein the metadata management node updating the target quota according to the gradient allocation policy after the target client writes data to be written if the target quota is valid, comprising:

If the target quota is valid and the reserved quota of the target client is smaller than the space to be written, the metadata management node judges whether the total residual space is larger than or equal to the space to be written, and the total residual space is a storage space in which the data written in the allocation area is removed by the total quota space;

if the total remaining space is greater than or equal to the space to be written, the metadata management node judges whether the total writing space is smaller than a total quota space;

4. A method according to any one of claims 1 to 3, further comprising:

And when the total writing space is larger than or equal to the total quota space, the metadata management node sends stop information to the target client, wherein the stop information is used for indicating the target client to stop writing data.

5. The method of claim 1, wherein the reserved quota of the target client is less than the space to be written, the method further comprising:

If the second remaining space is greater than or equal to the space to be written, the metadata management node allocates the second remaining space to the target client, where the second remaining space includes a remaining space of the allocation area and an overflow buffer area, the remaining space of the allocation area includes a storage space of the allocation area from which the written data is removed, and the overflow buffer area includes a storage space reserved according to the number of the one or more clients in the total quota space;

If the second remaining space is smaller than the space to be written, the metadata management node judges whether the total remaining space is larger than or equal to the space to be written;

if the total remaining space is greater than or equal to the space to be written, the metadata management node recalls the allocated quota of the one or more clients;

if recall is successful, the metadata management node calculates quota of the one or more clients according to the gradient allocation strategy and allocates the quota;

If the recall fails, the metadata management node returns insufficient quota to the target client.

6. The method of claim 5, wherein the metadata management node recalls an allocated quota in the one or more clients if the total remaining space is greater than or equal to the space to be written, comprising:

and if the total remaining space is greater than or equal to the space to be written, deleting the metadata of the one or more clients by the metadata management node.

7. The method of any of claims 1 to 6, wherein the metadata management node updating the target quota according to a gradient allocation policy if the target quota is stale, comprising:

if the target quota is invalid, the metadata management node judges the sizes of the first residual space and the overflow buffer area;

If the first residual space is smaller than the overflow buffer, the metadata management node updates the target quota according to the gradient allocation strategy;

and if the first residual space is larger than or equal to the overflow buffer zone, the metadata management node allocates the target quota for the target client.

8. A quota allocation method, wherein the method is applied to a target client in a distributed file system, the distributed file system comprising one or more directory entries, one or more clients, and a metadata management node, the one or more directory entries for storing data written by the one or more clients, the metadata management node for allocating quota to the one or more clients, the method comprising:

The target client sends a writing request, wherein the writing request carries data to be written;

If the target quota is invalid, the target client acquires the updated target quota, wherein the updated target quota is obtained by updating the target quota by the metadata management node according to a gradient allocation policy, the gradient allocation policy is used for indicating the metadata management node to allocate the quota to the one or more clients based on a first residual space and a preset quota level, the first residual space comprises a storage space after a total quota space is removed from an allocation area, the total quota space is a total storage space of the one or more directory entries, and the allocation area is a storage space occupied by the quota allocated by the one or more clients;

and if the target quota is valid, the target client writes the data to be written.

9. The method of claim 8, wherein the target client writing the data to be written if the target quota is valid, comprises:

The target client judges whether the reserved quota is larger than or equal to a space to be written in, wherein the reserved quota is a quota space reserved by the target client from the quota and used for storing the data to be written in, and the space to be written in is a storage space occupied by the data to be written in;

if the reserved quota is greater than or equal to the space to be written, the target client writes the data to be written;

And if the reserved quota is smaller than the space to be written, and the total residual space is larger than or equal to the space to be written, the target client writes the data to be written, and the total residual space is the total quota space and is the storage space in which the data is written in the allocation area is removed.

10. The method of claim 9, wherein if the reserved quota is less than the space to be written, the method further comprises:

if the second remaining space is greater than or equal to the space to be written, the target client acquires the updated target quota, wherein the second remaining space comprises the remaining space of the allocation region and an overflow buffer region, the remaining space of the allocation region comprises a storage space of the allocation region from which the written data is removed, and the overflow buffer region comprises a storage space reserved according to the number of the one or more clients in the total quota space;

And if the second remaining space is smaller than the space to be written, and the total remaining space is larger than or equal to the space to be written, after the metadata management node recalls the allocated quota of the one or more clients, the target client acquires the updated target quota.

11. A quota allocation apparatus, the apparatus being applied to a metadata management node in a distributed file system, the distributed file system further comprising one or more directory entries and one or more clients, the one or more directory entries for storing data written by the one or more clients, the metadata management node for allocating quota to the one or more clients, the apparatus comprising:

The updating module is used for updating the target quota according to a gradient allocation strategy by the metadata management node when the target quota is invalid, wherein the gradient allocation strategy is used for indicating the metadata management node to allocate the quota for the one or more clients based on a first residual space and a preset quota level, the preset quota level is divided according to the first residual space, the first residual space comprises a storage space after a total quota space is removed from an allocation zone, the total quota space is a total storage space of the one or more directory entries, and the allocation zone is a storage space occupied by the quota allocated by the one or more clients;

and the updating module is further used for updating the target quota according to the gradient allocation strategy by the metadata management node after the target client writes the data to be written when the target quota is valid.

12. The apparatus of claim 11, wherein the updating module is specifically configured to:

If the target quota is valid and the reserved quota of the target client is greater than or equal to a space to be written, the metadata management node judges whether the total writing space is smaller than the total quota space, the reserved quota is a quota space reserved by the target client from the quota for storing the data to be written, the space to be written is a storage space occupied by the data to be written, and the total writing space is a sum of storage spaces occupied by the data written by the one or more clients;

13. The apparatus of claim 11, wherein the updating module is specifically configured to:

14. The apparatus according to any one of claims 11 to 13, further comprising:

and the sending module is used for sending stop information to the target client when the total writing space is larger than or equal to the total quota space, wherein the stop information is used for indicating the target client to stop writing data.

15. The apparatus of claim 11, wherein if the reserved quota of the client is less than the space to be written, the apparatus further comprises:

The metadata management node is configured to allocate a second remaining space to the target client when the second remaining space is greater than or equal to the space to be written, where the second remaining space includes a remaining space of the allocation area and an overflow buffer area, the remaining space of the allocation area includes a storage space of the allocation area from which written data is removed, and the overflow buffer area includes a storage space reserved according to the number of the one or more clients in the total quota space;

The processing module is used for calculating and distributing quota of the one or more clients by the metadata management node according to the gradient distribution strategy when recall is successful;

and the processing module is also used for returning insufficient quota to the target client by the metadata management node when the recall fails.

16. The apparatus of claim 15, wherein the recall module is specifically configured to delete metadata of the one or more clients if the total remaining space is greater than or equal to the space to be written.

17. The apparatus according to any one of claims 11 to 16, wherein the updating module is specifically configured to:

18. A quota allocation apparatus for application to a target client in a distributed file system, the distributed file system comprising one or more directory entries for storing data written by one or more clients, and a metadata management node for allocating quota to the one or more clients, the apparatus comprising:

The acquiring module is configured to, when the target quota is invalid, acquire an updated target quota, where the updated target quota is obtained by updating the target quota by the metadata management node according to a gradient allocation policy, where the gradient allocation policy is used to instruct the metadata management node to allocate the quota to the one or more clients based on a first remaining space and a preset quota level, the first remaining space includes a storage space after a total quota space is removed from an allocation area, and the total quota space is a total storage space of the one or more directory entries, and the allocation area is a storage space occupied by a quota allocated by the one or more clients;

And the writing module is used for writing the data to be written into by the target client when the target quota is valid.

19. The apparatus according to claim 18, wherein the writing module is specifically configured to:

20. The apparatus of claim 19, wherein when the reserved quota is less than the space to be written, the apparatus further comprises:

21. A cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory, the processor of the at least one computing device to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any of claims 1-7.

22. A cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory, the processor of the at least one computing device to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any of claims 8-10.

23. A computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1-7.

24. A computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 8-10.

25. A computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform the method of any of claims 1-7.

26. A computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform the method of any of claims 8-10.