CN117579562A - Distributed file system-based current limiting solution - Google Patents

Abstract

The invention discloses a current limiting solution method based on a distributed file system, which specifically comprises the following steps: A. limiting the flow of mds resource service and oss resource service; B. resource access restrictions are performed with mds ops, including all operation types; C. resource limitation is performed in two dimensions, IOPS and BPS. The invention can dynamically adjust according to the actual access condition, can better adapt to peak-valley fluctuation and burst access, set different token adding rates and thresholds, can realize the fine control of different interfaces or different users, has simple realization of the current limiting scheme, is easy to integrate into the existing system, is stable and controllable, can process burst flow, and provides reliable protection for the distributed file system.

Description

Distributed file system-based current limiting solution

Technical Field

The invention relates to the technical field of distributed file systems, in particular to a current limiting solution method based on a distributed file system.

Background

Distributed file systems are typically composed of multiple nodes that share limited resources such as network bandwidth, disk I/O, memory, etc. When multiple clients request large amounts of data at the same time, resource contention may result, and throttling may help balance this contention, preventing a node from being overwhelmed by too many requests. When the file system receives too many requests, it may cause overload of the system. The restriction can help control the flow of requests, avoiding the system from becoming overwhelmed, resulting in increased response time, unstable service, and even crashing. In a distributed environment, consistency of data is very important. If there are too many requests, the system may not be able to process all requests in time, which may lead to delays in data updates, thereby affecting data consistency. Through the current limiting, the system can be ensured to process requests within the bearing capacity range, and the consistency of data is protected. The throttling may help prevent malicious attacks, such as DDoS attacks. An attacker may send a large number of requests in an attempt to consume system resources, and the impact of the attack can be slowed down by limiting the flow, thereby improving the security of the system. In high concurrency situations, the system may crash due to resource exhaustion or mishandling of the request. Through the current limiting, the system can be prevented from being crashed due to sudden large flow, and the stability of the system is improved. In general, current limiting is an important mechanism in distributed file systems that helps maintain the stability, reliability and performance of the system, ensuring that the system can operate properly under reasonable loads.

Conventional solutions have drawbacks such as:

1. token bucket algorithm (Token Bucket Algorithm), principle: the system places tokens into a "token bucket" at a constant rate, each token representing the processing authority of a request. When a request comes, a token needs to be consumed. If there are not enough tokens in the bucket, the request is denied or waiting. But cannot handle bursty traffic.

2. Leak bucket algorithm (Leaky Bucket Algorithm), principle: requests enter the "leaky bucket" at a constant rate, and the leaky bucket processes requests at a fixed rate. If the leaky bucket is full at the time of the request, the request is discarded. But cannot cope with instantaneous high flows.

3. Queue-Based Throttling (Queue-Based Throttling), principle: queue buffer requests are used to control the length of the queue or the rate of requests in the processing queue, thereby controlling the processing capacity of the system. However, an appropriate queue management mechanism is required and excessive length of the queue may cause delays.

4. Based on the current limit (Sliding Window Algorithm) of the sliding window, a fixed size time window is maintained, the number of requests within the window is counted, and if the threshold is exceeded, the current limit is maintained. But is relatively complex to implement and requires maintenance of the request records within the time window.

5. Token bucket leak-off algorithm (Token Bucket with Leaky Bucket), principle: the characteristics of the token bucket and the leakage bucket are combined, the token bucket is used for limiting the average speed, and the leakage bucket is used for limiting the burst flow. But require fine tuning of parameters to accommodate different flow patterns.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a current limiting solution based on a distributed file system.

In order to solve the technical problems, the invention provides the following technical scheme:

the invention relates to a current limiting solution method based on a distributed file system, which specifically comprises the following steps:

A. limiting the flow of mds resource service and oss resource service;

B. resource access restrictions are performed with mds ops, including all operation types;

C. resource limitation is performed in two dimensions, IOPS and BPS.

As a preferable technical scheme of the invention, the implementation mode of the step C is as follows:

s1, setting read iops, write iops, read bps, write bps, total iops, total bps and mops, total=read+write, and mops=mds+ops for any directory existing in the file system, wherein when a certain directory is set, the quotas are quotas of the whole cluster, namely the total access amount of all clients.

S2, the clients share and distribute relevant quota through mgr services, the clients report the ios, bps and mops used of the time period of the clients on a regular basis, and the mgr classifies the clients according to the data reported by the clients, the last distributed resources and the average data of the clients in the last 10S time period, and calculates the quota of each client distributed in the next round.

As a preferable technical scheme of the invention, the implementation mode of S2 is as follows:

step 1, calculating an average avg according to quota and client number of a share directory;

step 2, counting the number of clients of the non-free and using the used amount of the token in the clients of the free;

step 3, using expected= (total-used)/not_free as the expected value of the first stage;

step 4, counting the number of clients of busy and the number of tokens which are not consumed, but the number of tokens consumed is token_free which is larger than avg and smaller than expect;

step 5, using expect + =token_free/busy as the expected value of the second stage;

step 6, distributing expected values to all clients, wherein the clients of busy are expected, the clients with the token consumption number larger than the expected are expected, and the rest expected values are the token consumption number of the previous cycle;

step 7, counting all client expected values total_used;

step 8, token_res= (total-total_used)/client_size;

and 9, compensating token_res number tokens for all clients again.

Compared with the prior art, the invention has the following beneficial effects:

the invention can dynamically adjust according to the actual access condition, can better adapt to peak-valley fluctuation and burst access, set different token adding rates and thresholds, can realize the fine control of different interfaces or different users, has simple realization of the current limiting scheme, is easy to integrate into the existing system, is stable and controllable, can process burst flow, and provides reliable protection for the distributed file system.

Detailed Description

The following description of the preferred embodiments of the present invention is provided for the purpose of illustration and explanation only and is not intended to limit the present invention.

Example 1

The invention provides a current limiting solution method based on a distributed file system, which specifically comprises the following steps:

A. limiting the flow of mds resource service and oss resource service;

B. resource access restrictions are performed with mds ops, including all operation types;

C. resource limitation is performed in two dimensions, IOPS and BPS.

Further, the implementation manner of the step C is as follows:

s1, setting read iops, write iops, read bps, write bps, total iops, total bps and mops, total=read+write, and mops=mds+ops for any directory existing in the file system, wherein when a certain directory is set, the quotas are quotas of the whole cluster, namely the total access amount of all clients.

S2, the clients share and distribute relevant quota through mgr services, the clients report the ios, bps and mops used of the time period of the clients on a regular basis, and the mgr classifies the clients according to the data reported by the clients, the last distributed resources and the average data of the clients in the last 10S time period, and calculates the quota of each client distributed in the next round.

Further, the implementation manner of S2 is as follows:

step 1, calculating an average avg according to quota and client number of a share directory;

step 2, counting the number of clients of the non-free and using the used amount of the token in the clients of the free;

step 3, using expected= (total-used)/not_free as the expected value of the first stage;

step 4, counting the number of clients of busy and the number of tokens which are not consumed, but the number of tokens consumed is token_free which is larger than avg and smaller than expect;

step 5, using expect + =token_free/busy as the expected value of the second stage;

step 6, distributing expected values to all clients, wherein the clients of busy are expected, the clients with the token consumption number larger than the expected are expected, and the rest expected values are the token consumption number of the previous cycle;

step 7, counting all client expected values total_used;

step 8, token_res= (total-total_used)/client_size;

and 9, compensating token_res number tokens for all clients again.

In particular, the method comprises the steps of,

resource limitations on mds services:

the mds operation type is counted, such as create, delete, unlink.

An access limit is set for each operation type, including a maximum number or frequency of accesses.

When the number or frequency of operations exceeds a limit, the operation is denied or delayed.

Resource limitations on oss services:

resource limitation is performed in two dimensions, IOPS (input/output operands per second) and BPS (byte transmission rate per second).

A read iops, a write iops, a read bps, a write bps, a total iops, a total bps, and a mops quota are set for each directory.

total＝read+write，mops＝mds+ops。

The quota is a cluster level quota, i.e., a total access volume applicable to all clients.

Client resource quota allocation:

the client reports its own iops, bps, and mops used for some time through mgr service (cluster management service).

mgr classifies clients according to the data reported by clients, the resources allocated in the last round and other condition variables.

The quota for the next round is calculated for each client according to the following algorithm:

calculating an average avg according to the quota of the share directory and the number of clients;

counting the number of clients of non free and the used token in the clients of free;

calculating an expected value expected= (total-used)/not_free of the first stage;

counting the number of clients of busy and the number of tokens token_free which are not used up but consume more tokens than avg and less than expect;

calculating an expected value of the second stage expect+ =token_free/busy;

distributing expected values to all clients, wherein the busy clients are expected, the clients with the token consumption number larger than the expected are expected, and the rest clients are token consumption numbers of the previous cycle;

counting expected values total_used of all clients;

token_res＝(total-total_used)/client_size；

compensating for the number of tokens of all client token res.

Taking an e-commerce platform as an example

Determining a current limit threshold: first, a current limit threshold for each interface needs to be determined, which may be determined according to factors such as an average access amount, a peak access amount, etc. of the interfaces, assuming that an access threshold of 100 times per minute is set for the interface a.

Creating a token bucket: next, a token bucket needs to be created for limiting the access frequency of interface a, which can be set to a fixed capacity, adding a certain number of tokens per minute to the bucket, in this embodiment assuming 100 tokens per minute,

implementing the current limiting logic: when a user requests to access the interface A, a token needs to be acquired from a token bucket, and if enough tokens exist in the bucket, the request passes and a result is returned; otherwise, the request is denied or queued for processing.

Monitoring and adjusting: the access condition of the interface A and the use condition of the token bucket are required to be monitored in real time so as to adjust the current limiting threshold value and the token adding rate in time, and if the access amount is found to continuously exceed the threshold value, the token adding rate can be increased; if the amount of accesses is found to be too low, the token addition rate may be reduced.

Compared with the prior art, the invention has the following beneficial effects:

the invention can dynamically adjust according to the actual access condition, can better adapt to peak-valley fluctuation and burst access, set different token adding rates and thresholds, can realize the fine control of different interfaces or different users, has simple realization of the current limiting scheme, is easy to integrate into the existing system, is stable and controllable, can process burst flow, and provides reliable protection for the distributed file system.

Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (3)

1. The current limiting solution based on the distributed file system is characterized by comprising the following steps:

A. limiting the flow of mds resource service and oss resource service;

B. resource access restrictions are performed with mds ops, including all operation types;

C. resource limitation is performed in two dimensions, IOPS and BPS.

2. The distributed file system based current limiting solution according to claim 1, wherein step C is implemented as follows:

s1, setting read iops, write iops, read bps, write bps, total iops, total bps and mops, total=read+write, and mops=mds+ops for any directory existing in the file system, wherein when a certain directory is set, the quotas are quotas of the whole cluster, namely the total access amount of all clients.

S2, the clients share and distribute relevant quota through mgr services, the clients report the ios, bps and mops used of the time period of the clients on a regular basis, and the mgr classifies the clients according to the data reported by the clients, the last distributed resources and the average data of the clients in the last 10S time period, and calculates the quota of each client distributed in the next round.

3. The distributed file system-based current limiting solution according to claim 2, wherein S2 is implemented as follows:

step 1, calculating an average avg according to quota and client number of a share directory;

step 2, counting the number of clients of the non-free and using the used amount of the token in the clients of the free;

step 3, using expected= (total-used)/not_free as the expected value of the first stage;

step 4, counting the number of clients of busy and the number of tokens which are not consumed, but the number of tokens consumed is token_free which is larger than avg and smaller than expect;

step 5, using expect + =token_free/busy as the expected value of the second stage;

step 6, distributing expected values to all clients, wherein the clients of busy are expected, the clients with the token consumption number larger than the expected are expected, and the rest expected values are the token consumption number of the previous cycle;

step 7, counting all client expected values total_used;

step 8, token_res= (total-total_used)/client_size;

and 9, compensating token_res number tokens for all clients again.