CN114765592A - Flow control method, device, storage medium, service equipment and service cluster - Google Patents

Abstract

The present invention provides a flow control method, device, storage medium, service device and service cluster. The method includes: a first service device in the service cluster receives an access request sent by a user, and if it is determined that the token bucket of the first service device is in the token bucket of the first service device If there is no token for processing the access request, the access request is stored in the buffer queue of the first service device, and the first service device and the second service device in the service cluster are switched to the centralized mode. Mode refers to the mode in which the management device in the service cluster participates in flow control. When the token bucket of the second service device is full, the second service device sends its redundant tokens to the management device for storage. The first service device uses the token obtained from the management device to process the access request stored in the buffer queue of the first service device. In the centralized mode, tokens are borrowed and lent between different service devices through management devices to achieve low-cost and accurate distributed flow control.

Description

Flow control method, apparatus, storage medium, service device and service cluster

technical field

The present invention relates to the field of Internet technologies, and in particular, to a flow control method, device, storage medium, service device and service cluster.

Background technique

In many application scenarios, there is a need to control the flow of the user's access request (usually also referred to as speed-limiting the user's access traffic). For example, in the scenario where the server provides services for the user, the gateway is In scenarios where users provide network forwarding services.

For high availability, services are generally provided to users in the form of service clusters, such as clusters composed of multiple servers and clusters composed of multiple gateways. At this time, how to ensure that the user's access traffic is accurately limited in a low-cost (lower resource consumption cost) manner in the entire cluster is an urgent problem to be solved.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a flow control method, apparatus, storage medium, service device and service cluster, which can realize low-cost and accurate flow control.

In a first aspect, an embodiment of the present invention provides a flow control method, which is applied to a first service device in a service cluster, where the first service device is any service device in the service cluster, and the method includes:

Receive access requests sent by users;

If it is determined that there is no token for processing the access request in the token bucket of the first service device, storing the access request in the buffer queue of the first service device;

switching the first service device and the second service device in the service cluster to a centralized mode, where the centralized mode refers to a mode in which the management device in the service cluster participates in flow control;

Using the token obtained from the management device, the access request stored in the buffer queue of the first service device is processed, wherein when the token bucket of the second service device is replenished, the second service device Send its own redundant tokens to the management device for storage.

In a second aspect, an embodiment of the present invention provides a flow control device, which is located in a first service device in a service cluster, where the first service device is any service device in the service cluster, and the device includes:

The request receiving module is used to receive the access request sent by the user;

A request cache module, configured to store the access request in the buffer queue of the first service device if it is determined that there is no token for processing the access request in the token bucket of the first service device ;

A mode setting module for switching the first service device and the second service device in the service cluster to a centralized mode, where the centralized mode refers to a mode in which the management device in the service cluster participates in flow control ;

a request processing module, configured to use the token obtained from the management device to process the access request stored in the buffer queue of the first service device, wherein when the token bucket of the second service device is full, The second service device sends its own redundant tokens to the management device for storage.

In a third aspect, an embodiment of the present invention provides a service device, where the service device is any service device in a service cluster, and the service device includes: a memory, a processor, and a communication interface; wherein, the memory stores executable code that, when executed by the processor, causes the processor to execute:

Receive access requests sent by users;

If it is determined that there is no token for processing the access request in the token bucket of the service device, the access request is stored in the buffer queue of the service device;

Switching the service device and other service devices in the service cluster to a centralized mode, where the centralized mode refers to a mode in which management devices in the service cluster participate in flow control;

Use the token obtained from the management device to process the access request stored in the buffer queue of the service device, wherein when the token bucket of the other service device is replenished, the other service device will use its own redundant The token is sent to the management device for storage.

In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, where executable code is stored on the non-transitory machine-readable storage medium, and when the executable code is executed by a processor of an electronic device , so that the processor can at least implement the flow control method described in the first aspect.

In a fifth aspect, an embodiment of the present invention provides a service cluster, including:

a first service device, a second service device and a management device;

The first service device is configured to receive the access request sent by the user, and if it is determined that there is no token for processing the access request in the token bucket of the first service device, the access request is stored in the in the buffer queue of the first service device; and, switching the first service device and the second service device to a centralized mode, where the centralized mode refers to a mode in which the management device participates in flow control; and, using the token obtained from the management device, process the access request stored in the buffer queue of the first service device;

The second service device is configured to, in the centralized mode, send its own redundant tokens to the management device for storage when the token bucket of the second service device is full;

The management device is configured to store the token sent by the second service device, and, in response to an acquisition request of the first service device, provide the token to the first service device.

In the embodiment of the present invention, it is assumed that each service device in the service cluster provides services to the user, and a token bucket (Token Bucket) mechanism is used to limit the access traffic of the user. Under the token bucket mechanism, each service device includes three functional modules: replenisher, token bucket and buffer queue. Taking the first service device in the service cluster as an example, the replenisher of the first service device continuously replenishes a set number of tokens to the token bucket of the first service device at set time intervals. When the first service device receives the access request sent by the user, it goes to the token bucket of the first service device to obtain a token for processing the access request. If the token is not obtained, the token of the first service device is indicated. The bucket is empty. At this time, the first service device stores the received access request into the buffer queue of the first service device, and sends the first service device and the second service device in the service cluster (which is the second service device in the service cluster except the first service device in the service cluster). A general term for other service devices other than a service device) is switched to a centralized mode, where the centralized mode refers to a mode in which the management devices in the service cluster participate in flow control. That is to say, in the decentralized mode, the management device does not participate in the flow control, but each service device independently performs its own flow control, while in the centralized mode, the management device uniformly coordinates the service devices to achieve flow control.

Specifically, in the case where both the first service device and the second service device are in the centralized mode, if the token bucket of the second service device is full of tokens, the subsequent redundant tokens generated by the second service device Cards can be sent to the management device for storage. Based on this, since the first service device does not have a locally usable token at this time, in the centralized mode, the first service device can obtain the token shared by the second service device from the management device, and use the token from the management device. The token obtained to process the access request stored in the local buffer queue.

It can be seen that, in the embodiment of the present invention, in the centralized mode, tokens can be borrowed and lent between different service devices, so as to realize accurate distributed flow control. In addition, the management device does not work all the time. It only starts to work when a service device does not have a locally available token (that is, the access traffic exceeds the upper limit of the traffic limit of the service device), which helps to reduce the resource consumption of the service cluster. to improve overall service efficiency.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic diagram of a composition architecture of a service cluster according to an embodiment of the present invention;

2 is a schematic diagram of a flow control process provided by an embodiment of the present invention;

3 is a flowchart of a flow control method according to an embodiment of the present invention;

4 is a schematic diagram of another flow control process provided by an embodiment of the present invention;

5 is a schematic structural diagram of a flow control device according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a service device corresponding to the flow control apparatus provided in the embodiment shown in FIG. 5 .

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a," "the," and "the" as used in the embodiments of the present invention and the appended claims are intended to include the plural forms as well, unless the context clearly dictates otherwise, "a plurality" Generally at least two are included.

Depending on the context, the words "if", "if" as used herein may be interpreted as "at" or "when" or "in response to determining" or "in response to detecting". Similarly, the phrases "if determined" or "if detected (the stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detected (the stated condition or event)," depending on the context )" or "in response to detection (a stated condition or event)".

In addition, the sequence of steps in the following method embodiments is only an example, and is not strictly limited.

In the scenario of providing services to users through N service devices in the service cluster, N>1, it is assumed that the overall traffic limit for a user is limited access requests per second. Then, if the traffic is evenly distributed to each service device, the upper limit of the traffic of each service device is limit/N per second.

Wherein, in this embodiment of the present invention, the service device may be a server that provides a certain service, or may be a network device such as a gateway, a router, a switch, a forwarding device of an edge network, a Content Delivery Network (CDN) node, etc. .

In this embodiment of the present invention, the user refers to any subject who uses the service device. For example, the user may be a specific person or persons, and the corresponding IP address may be used as the user identifier. For another example, the user may also be some kind of client. For example, in the scenario of data interaction with the service device through a certain client, the service device restricts the traffic of the client.

Assuming that there are N service devices in the service cluster, the method of performing limit/N speed limit control per second on each service device is simple and convenient, but when the user traffic is unevenly distributed on different service devices, the actual The speed limit effect will be worse. For example, if N=2, for a user, assuming that the overall speed limit requirement is: no more than 100 access requests per second, at this time, both service devices have a capacity limit of 50 access requests per second . Assuming that 100 access requests are received successively within 1 second, 80 access requests are allocated to service device 1, and 20 access requests are allocated to service device 2. At this time, it is obvious that the number of access requests is 80 larger than that of service device 1. However, the number of access requests received by service device 2 is less than the upper limit of its capacity. It can be seen that from the overall point of view of service device 1 and service device 2, in fact, the user The traffic does not exceed the overall traffic limit (100 access requests per second), but under the above flow control mechanism, some access requests will be rejected even if the user does not trigger an access request exceeding the above traffic limit. The accuracy of flow control is not high, and the user experience is not good.

In order to realize accurate flow control at a lower cost, the flow control solution proposed by the embodiments of the present invention is provided. Based on this scheme, accurate flow control can be performed even when user traffic is unevenly distributed across multiple service devices.

Before introducing the execution process of the flow control solution provided by the embodiments of the present invention, some concepts involved in the embodiments of the present invention are first introduced. In the embodiment of the present invention, the token bucket (Token Bucket) method is used for traffic control, so first, the composition structure of the service cluster using the token bucket method is briefly introduced.

As shown in FIG. 1 , it is assumed that the service cluster includes a first service device and a second service device, as well as a management device.

Each service device may be provided with a current limiter, and each current limiter may include a replenisher, a token bucket and a buffer queue. It can be understood that the above-mentioned current limiter and the replenisher, token bucket, and buffer queue included in the current limiter are actually functional modules set in the service device.

As shown in FIG. 1 , the current limiter in the first service device is denoted as the first current limiter, and the current limiter in the second service device is denoted as the second current limiter. The first current limiter includes: a first feeder, a first token bucket and a first buffer queue. The second current limiter includes: a second feeder, a second token bucket and a second buffer queue.

In addition, each current limiter may be configured with a corresponding device identifier, for example, the device identifier of the first current limiter shown in FIG. 1 is name1@nodeA, and the device identifier of the second current limiter is name1@nodeB. Among them, name1 represents the user ID, which is used to indicate which user the current limiter is used for flow control, and nodeA and nodeB are used to distinguish different current limiters corresponding to the same user ID.

It can be understood that when a service device includes only one current limiter, the device identifier of the current limiter may be the device identifier of the service device. However, in practical applications, there may also be a situation where a service device includes multiple current limiters. At this time, when performing flow control for different users, the corresponding current limiter in the service device is selected.

In the case where a service device includes multiple current limiters, the work of each current limiter is independent, and the working process is similar. Therefore, for the convenience of description, it is assumed in this embodiment of the present invention that only one service device has Contains a current limiter.

The replenisher is used to replenish tokens into the token bucket. The replenisher runs periodically, that is, replenishes a set number of tokens into the token bucket at each set time interval. Based on the premise of the preset traffic threshold limit and the number N of service devices included in the service cluster, and assuming that the above set time interval is deltaTime, the number of tokens replenished by the replenisher to the token bucket every deltaTime can be determined by The above-mentioned traffic threshold limit, the number N of service devices, and deltaTime are determined, for example, limit/N*deltaTime.

The token bucket is used to store the tokens supplied by the dispenser. The token bucket has an upper limit of capacity. Optionally, the upper limit of the capacity of the token bucket is determined by the preset traffic threshold limit, the number N of service devices in the service cluster, and the preset burst traffic parameter Brust%. Brust% is used to set the burst traffic limit value of the service device. The larger the Brust% value is, the larger the burst network traffic is allowed. Generally, Brust% can be set to a value such as 50%. Optionally, the upper limit of the capacity of the token bucket can be expressed as: limit/N*Brust%.

Generally speaking, when the service device receives the access request from the user, it will first get the token from its own token bucket (the number of tokens to be taken out is related to the number of received access requests, such as equal), If the token used to process the access request can be taken out, that is, the number of tokens stored in the token bucket is sufficient to process the received access request, then the access request can be processed directly, and the amount of tokens stored in the token bucket is reduced. to the corresponding token. However, if there is no token available for processing the access request in the token bucket, the access request will be temporarily stored in the buffer queue. When the replenisher subsequently generates new tokens, the newly generated tokens will be used first. To process the access requests stored in the buffer queue, the newly generated tokens will be replenished into the token bucket only after the buffer queue is empty.

The upper limit of the capacity of the buffer queue is determined by the above-mentioned traffic threshold limit, the number N of service devices, and the preset buffer traffic parameter Queue%. Queue% determines the length of the buffer queue, the greater the Queue%, the greater the storage capacity of the queue. Optionally, the upper limit of the capacity of the buffer queue can be expressed as: limit/N*Queue%.

It can be understood that, the processing of the access request in the embodiment of the present invention refers to the processing of the response to the access request. For example, if the access request indicates to return to a certain page, the response result is to jump to this page.

The above describes the composition structure of the service cluster. Based on the above composition structure, it is assumed that the current first service device receives the access request sent by the user. At this time, each component in the service cluster can perform the following operations to complete the flow control:

The first service device: receives the access request sent by the user, and if it is determined that there is no token for processing the access request in the token bucket of the first service device, the access request is stored in the buffer queue of the first service device , and, switching the first service device and the second service device to a centralized mode, where the centralized mode refers to a mode in which the management device participates in flow control; and, using the token obtained from the management device to process the buffer queue of the first service device Access requests stored in .

Second service device: in the centralized mode, when the token bucket of the second service device is full, it will send its redundant tokens to the management device for storage.

Management device: storing the token sent by the second service device, and providing the token sent by the second service device to the first service device in response to an acquisition request from the first service device.

For ease of understanding, the execution process of the above flow control scheme is exemplarily described below with reference to FIG. 2 .

As shown in FIG. 2 , it is assumed that the token bucket of the first service device is currently empty, the buffer queue of the first service device is empty, and it is assumed that the token bucket of the second service device is not empty, and the token bucket of the second service device is empty. The buffer queue is empty. After that, the first service device receives the access request X sent by the user, and the first service device first goes to its token bucket to get the token, and finds that the token bucket is empty at this time, that is, there is no available token, and will access the Request X is stored in the buffer queue of the first service device. At this time, since the first service device does not have an available token, the access request X cannot be processed immediately, so the first service device can use the work of itself and other service devices in the service cluster (assumed to be the second service device in this paper) The mode is switched to centralized mode.

Optionally, the first service device may switch the working mode of itself and the second service device to the centralized mode when the first service device is originally in the decentralized mode and its own token bucket is empty. Optionally, when the first service device is originally in a decentralized mode, its own token bucket is empty, and the number of access requests cached in its own buffer queue reaches a set threshold, it can send itself and the second service device The working mode of the service device is switched to the centralized mode.

In the embodiment of the present invention, the working mode of the service device is divided into two modes: a centralized mode and a non-centralized mode. Among them, the centralized mode refers to the mode in which the management device participates in flow control, on the contrary, the decentralized mode refers to the mode in which the management device does not participate in the flow control.

In general, in the centralized mode, if a service device generates excess tokens, the excess tokens will be sent to the management device for storage. The device gets its stored token.

In the decentralized mode, each service device uses only locally generated tokens to process locally received access requests.

The redundant tokens generated by a service device refer to the redundant tokens generated after the token bucket of the service device is fully replenished. For example, taking the first service device as an example, the replenisher in the first service device continuously replenishes tokens to the token bucket of the first service device at set time intervals, assuming that the upper limit of the token bucket capacity is 100 Tokens, when the token bucket is full of 100 tokens, suppose the supply device of the first service device subsequently generates 5 tokens, these 5 tokens can no longer be stored in the token bucket, so These 5 tokens are redundant tokens, which are sent to the management device to be shared with other service devices.

In the hypothetical situation shown in FIG. 2 , since the current token bucket of the first service device is empty, the first service device triggers to switch the working modes of itself and the second service device to the centralized mode. At this time, the management will be triggered. The device performs operations related to flow control.

In practical applications, each service device can store a flag bit indicating the current working mode, and the flag position is a different value, indicating that it is in a different working mode, for example, the centralized mode is 01, and the decentralized mode is 00. The first service device can switch its own working mode to the centralized mode when the working mode switching conditions are met, and send a notification message to the second service device, so that the second service device also switches to the centralized mode according to the notification message. Wherein, in combination with the above, the working mode switching condition here means that the token bucket of the first service device is empty, and at this time, the buffer queue of the first service device needs to be used.

Optionally, the first service device may send a notification message to the management device, so as to send the notification message to the second service device through the management device.

In FIG. 2, since the token bucket of the second service device is not empty, the replenisher of the second service device replenishes a set number of tokens into the token bucket of the second service device every set time interval, assuming that The number of access requests received by the second service device is relatively small, and the above-mentioned set time interval is relatively short, so the token bucket of the second service device will be replenished soon, and the token bucket of the second service device will be filled. After the replenishment is full, the excess tokens subsequently generated by the replenisher of the second service device will be sent to the management device for storage. In FIG. 2, it is assumed that after the token bucket of the second service device is full, the second service device sends the excess 10 tokens to the management device for storage.

The second service device sends the extra 10 tokens to the management device for storage, which may be by starting a counter in the management device, and the management device sets the value of the counter=10 according to the number of tokens received.

After that, after placing itself in the centralized mode, the first service device can continuously send an acquisition request to the management device to acquire the token stored in the management device. In the above example, the first service device can obtain 10 tokens stored in the management device. Assuming that 1 token needs to be used to process the access request X, the first service device will perform a subtraction operation on these 10 tokens at a time , to indicate that one token is taken out for processing the access request X, and the remaining 9 tokens are stored in the token bucket of the first service device.

Assuming that before the replenisher of the first service device replenishes tokens into the token bucket of the first service device again, the first service device receives 9 access requests in turn, then the first service device can directly use its own token at this time. The above 9 tokens already stored in the bucket are used to process these 9 access requests.

It can be understood that the service device initially works in the decentralized mode. The conditions for switching the service device from the decentralized mode to the centralized mode are described above. Optionally, after the service device switches to the centralized mode , if the duration in the centralized mode has reached the set time, it can automatically switch back to the decentralized mode.

Based on this, when the duration after switching the first service device and the second service device to the centralized mode reaches the set duration, the first service device and the second service device are switched to the decentralized mode.

To sum up, in this embodiment of the present invention, by introducing the above-mentioned centralized mode, the management device can coordinate the sharing of tokens generated by different service devices among different service devices, so that when the token bucket of the service device is full, it is returned. The excess tokens are given to the management device, and the remaining tokens contributed by other service devices are consumed when the buffer queue of the service device is not empty, so that from the perspective of the service cluster as a whole, precise distributed control of user traffic is realized. As for the situation where the user's overall access traffic is actually far from reaching the upper limit of the capacity limit, the access will be restricted. In addition, the execution of the centralized mode is only triggered under certain conditions (the local token bucket of the service device is empty, that is, when the speed limit condition of a single machine is reached), and at other times, the service device only works in the decentralized mode, so Reduced involvement in managing devices. The higher the participation of the management device, the greater the overall resource consumption. Therefore, the management device is only triggered to participate in the flow control process under the above-mentioned specific conditions, which can effectively reduce the resource consumption.

FIG. 3 is a flowchart of a flow control method provided by an embodiment of the present invention. As shown in FIG. 3 , the method may include the following steps:

301. The first service device receives an access request sent by a user.

302. If the first service device determines that there is no token for processing the access request in the token bucket of the first service device, the first service device stores the access request in the buffer queue of the first service device.

303. The first service device switches the first service device and the second service device in the service cluster to a centralized mode, where the centralized mode refers to a mode in which the management device in the service cluster participates in flow control.

304. The first service device uses the token obtained from the management device to process the access request stored in the buffer queue of the first service device, wherein, when the token bucket of the second service device is full, the second service device will The excess tokens are sent to the management device for storage.

In this embodiment, it is assumed that the service cluster includes a first service device, a second service device, and a management device. The flow control solution provided in this embodiment is described from the perspective of the first service device, that is, in the process of executing flow control, the specific execution steps of the first service device may be implemented with reference to this embodiment.

For the relevant details of the execution process of this embodiment, reference may be made to the relevant descriptions above, which will not be repeated here.

Assuming that the first service device receives access request 1 at time T1 and finds that its token bucket is empty, it puts access request 1 in the buffer queue for temporary storage, and switches itself and the second service device to the centralized mode.

Assuming that the first service device receives the access request 2 at time T2 after a short time, if it finds that its own token bucket is still empty at this time, the access request 2 is also put into the buffer queue for temporary storage.

Assuming that the first service device obtains 10 tokens from the management device at the following time T3, it takes out two of them for processing access request 1 and access request 2 stored in the buffer queue, and uses the remaining 8 tokens The token is stored in the token bucket of the first service device. That is, if the first number of tokens (10 in the above example) acquired by the first service device from the management device is greater than the second number of access requests stored in the buffer queue of the first service device (in the above example, 2), then use the second number of tokens to process the second number of access requests, and store the remaining tokens (8 in the above example) into the token bucket of the first service device.

It can be understood that, in the above-mentioned hypothetical situation, the following conditions are actually also assumed: first, the first service device does not obtain a token from the management device during the period from T1 to T2, which may be due to the distance between time T2 and time T1. It is too close, so that the first service device has not yet sent an acquisition request to the management device for obtaining the token, it may be that the acquisition request was triggered but the management device does not have a token at this time; second, the replenisher of the first service device During the period from T1 to T2, no token is generated to be replenished into the token bucket.

It is worth noting that in step 303, the first service device can switch its working mode to the centralized mode, and send a notification message to the second service device, so that the second service device can switch to the centralized mode according to the notification message model. Specifically, the notification message may include identification information corresponding to the user who sent the access request, so that the second service device switches to the centralized mode when determining that its own device identification matches the identification information.

As mentioned above, if a service device contains only one current limiter, the device identification of the service device and the device identification of the current limiter can be the same identification, and the device identification can be composed of two parts, one part is corresponding to the user The identification information may be referred to as a user identification; the other part is identification information used to distinguish different service devices corresponding to the same user, and may be referred to as a node identification. For example, name1@nodeA in the example above, where name1 is the user ID and nodeA is the node ID.

The first service device may send a notification message carrying the user ID of name1 to each other service device in the service cluster (assuming that there are other service devices in the service cluster besides the second service device), so that the service receiving the notification message The device compares whether the user ID name1 is included in its own device ID. If it does not, it ignores the notification message. If it does, it switches its own working mode to the centralized mode according to the notification message.

In addition, it is worth noting that when both the first service device and the second service device work in the centralized mode, for each service device, there are two sources of tokens: one is the continuous supply of its own replenisher , and the other is the excess tokens contributed by other service devices obtained from the management device. For the first service device in the above example, in the centralized mode, at the beginning, since the local token bucket is empty, it is necessary to obtain the redundant tokens contributed by the second service device from the management device to process the access ask. However, as the replenisher of the first service device continuously replenishes tokens into its token bucket, and assuming that the number of access requests subsequently received by the first service device is relatively small, gradually, in the command of the first service device Tokens will continue to increase in the bucket, and the locally generated tokens may be used in the future to meet the processing requirements of locally received access requests. After the token bucket of the first service device is replenished through the replenisher of the first service device, the first service device can send its own excess tokens to the management device for storage, so that if the second service device subsequently fails due to a sudden When the tokens in the token bucket of the second service device are insufficient due to user traffic, the management device can obtain and use the excess tokens contributed by the first service device.

As described above, if the duration after the first service device and the second service device are switched to the centralized mode reaches the set duration, the first service device and the second service device are switched to the decentralized mode.

The foregoing embodiment describes the processing situation when the first service device finds that the local token bucket is empty after receiving the access request, that is, there is no token in the token bucket for processing the access request. However, as mentioned above, initially, each service device works in the decentralized mode. Taking the first service device as an example, when the first service device is in the decentralized mode, when the first service device receives When the user sends an access request, if it is determined that a token for processing the access request exists in the token bucket of the current first service device, the access request is processed using the token obtained from the token bucket of the first service device That's it. It will not switch to the centralized mode until an access request triggered by the user is received at a later time and it is found that the token bucket of the first service device is empty at this time.

In order to more intuitively compare the differences in the working process of the service device in the centralized mode and the non-centralized mode, an execution process of a flow control scheme is exemplarily described below with reference to FIG. 4 .

In Fig. 4, it is assumed that there are 10 tokens stored in the token bucket of the first service device, the buffer queue of the first service device is empty, the token bucket of the second service device has 80 tokens, and the second service device has 80 tokens in the token bucket. The buffer queue of the service device is empty. And it is assumed that both the first service device and the second service device are in a decentralized mode at this time.

At a certain moment, it is assumed that the first service device receives the access request A1 triggered by the user. Since the token bucket of the first service device is not empty at this time, the first service device performs an operation of decrementing the number of tokens in its token bucket by one. After that, the access request A1 is processed. At this time, the number of tokens in the token bucket of the first service device becomes 9.

In the same way, assuming that the first service device receives the access requests A2 to A10 in sequence, and performs the above operations in sequence to process the nine access requests one by one, at this time, the token bucket of the first service device will become empty. .

In addition, it is assumed that since the replenisher of the second service device continuously replenishes the token bucket of the second service device and the number of access requests received by the second service device is relatively small, the token bucket of the second service device has been Full, reaches 100 tokens. At this time, if the feeder of the second service device generates a new token again, the newly generated token may be discarded.

After that, it is assumed that the first service device receives the access request A11 triggered by the user. Since the token bucket of the first service device is empty at this time, the access request A11 is stored in the buffer queue of the first service device, and the first The service device switches to the centralized mode, and notifies the second service device to also switch to the centralized mode.

After switching to the centralized mode, it is assumed that the feeder of the second service device has generated 5 more tokens. Since the token bucket of the second service device is full at this time, the newly generated 5 tokens will be The second service device is sent to the management device. The management device starts a counter and sets the count value of the counter to 5, indicating that 5 tokens have been obtained.

On the other hand, the first service device may periodically poll the management device to obtain the token stored in the management device. After the first service device obtains the five tokens stored in the management device, first, the obtained token is used to process the access request A11 stored in the buffer queue of the first service device, since only one token is required to process the access request A11 That is, so the first service device stores the remaining 4 tokens into the token bucket of the first service device. It can be understood that, at this time, the value of the counter of the management device is decremented by 5.

It can be seen from this that, in the traffic control scheme provided by the embodiment of the present invention, it is equivalent to setting a centralized switching mechanism for switching from a decentralized mode to a centralized mode. The upper limit of the speed limit. At this time, the centralized switch is off, and the behavior of each service device is the stand-alone mode, that is, the locally generated token is used to process the access request received locally, without consuming the resources of the management device. However, when the access traffic triggered by the user reaches the upper limit of the speed limit of a certain service device, the centralized switch is turned on, and precise distributed traffic control is performed through the synchronization of the management device, which not only meets the needs of precise speed limit, but also avoids the need for Additional consumption of resources.

The flow control device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art can understand that these flow control devices can be configured by using commercially available hardware components through the steps taught in this solution.

5 is a schematic structural diagram of a flow control apparatus according to an embodiment of the present invention. The flow control apparatus is located in a first service device in a service cluster, and the first service device is any service device in the service cluster. As shown in FIG. 5 , the apparatus includes: a request receiving module 11 , a request caching module 12 , a mode setting module 13 , and a request processing module 14 .

The request receiving module 11 is configured to receive an access request sent by a user.

The request cache module 12 is configured to store the access request in the buffer queue of the first service device if it is determined that there is no token for processing the access request in the token bucket of the first service device middle.

The mode setting module 13 is used to switch the first service device and the second service device in the service cluster to a centralized mode, where the centralized mode means that the management devices in the service cluster participate in flow control model.

The request processing module 14 is configured to use the token obtained from the management device to process the access request stored in the buffer queue of the first service device, wherein when the token bucket of the second service device is full , the second service device sends its own redundant tokens to the management device for storage.

Optionally, the request processing module 14 is further configured to: if it is determined that a token for processing the access request exists in the token bucket of the first service device, use the token bucket from the first service device The access request is processed by the token obtained in the first service device, and the first service device is in a decentralized mode, and the decentralized mode refers to a mode in which the management device does not participate in flow control.

Optionally, the mode setting module 13 is further configured to: if the duration after the first service device and the second service device are switched to the centralized mode reaches the set duration, then the first service device and the second service device are switched to the centralized mode. The second service device is switched to a decentralized mode.

Optionally, the request processing module 14 is further configured to: if the first number of tokens obtained from the management device is greater than the second number of access requests stored in the buffer queue of the first service device, use the The second quantity of tokens is processed for the second quantity of access requests, and the remaining tokens are stored in the token bucket of the first service device.

Optionally, the mode setting module 13 is further configured to: in the centralized mode, after the token bucket of the first service device is fully replenished by the replenisher of the first service device, use its own redundant token bucket. The token is sent to the management device for storage.

Optionally, the request caching module 12 can be specifically configured to: switch the first service device to the centralized mode; send a notification message to the second service device, so that the second service device can The notification message switches to the centralized mode.

Wherein, optionally, the request cache module 12 may be specifically configured to: send a notification message including identification information corresponding to the user to the second service device, so that the second service device determines that its own device identification is the same as When the identification information matches, switch to the centralized mode.

Optionally, the upper limit of the capacity of the token bucket of the first service device is determined by a preset traffic threshold, the number of service devices included in the service cluster, and a preset burst traffic parameter; the first service The upper limit of the capacity of the buffer queue of the device is determined by the traffic threshold, the number of the service devices and the preset buffer traffic parameter; the number of tokens replenished by the replenisher of the first service device every the set time interval Determined by the traffic threshold, the number of service devices, and the set time interval.

The apparatus shown in FIG. 5 can implement the flow control scheme provided in the embodiments shown in the foregoing FIG. 1 to FIG. 4 . For the detailed execution process and technical effects, refer to the descriptions in the foregoing embodiments, which will not be repeated here.

In a possible design, the structure of the flow control apparatus shown in FIG. 5 can be implemented as a service device, the service device is any service device in a service cluster, and the service cluster also includes management devices and other service devices. As shown in FIG. 6 , the service device may include: a processor 21 , a memory 22 , and a communication interface 23 . The memory 22 stores executable codes, and when the executable codes are executed by the processor 21, the processor 21 can at least realize:

Receive access requests sent by users;

If it is determined that there is no token for processing the access request in the token bucket of the service device, the access request is stored in the buffer queue of the service device;

Switching the service device and other service devices in the service cluster to a centralized mode, where the centralized mode refers to a mode in which management devices in the service cluster participate in flow control;

Use the token obtained from the management device to process the access request stored in the buffer queue of the service device, wherein when the token bucket of the other service device is replenished, the other service device will use its own redundant The token is sent to the management device for storage.

In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, where executable codes are stored on the non-transitory machine-readable storage medium, and when the executable codes are executed by a processor of an electronic device , so that the processor can at least implement the flow control scheme provided in the above-mentioned embodiments shown in FIG. 1 to FIG. 4 .

The apparatus embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and certainly can also be implemented by combining hardware and software. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of computer products in essence or that make contributions to the prior art. In the form of a computer program product embodied on a medium (including but not limited to disk storage, CD-ROM, optical storage, etc.).

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (12)

1. A flow control method is applied to a first service device in a service cluster, wherein the first service device is any service device in the service cluster, and the method comprises the following steps:

receiving an access request sent by a user;

if the token bucket of the first service equipment is determined not to have the token for processing the access request, storing the access request into a buffer queue of the first service equipment;

switching the first service device and a second service device in the service cluster into a centralized mode, wherein the centralized mode refers to a mode in which a management device in the service cluster participates in flow control;

and processing the access request stored in the buffer queue of the first service equipment by using the token acquired from the management equipment, wherein when the token bucket of the second service equipment is full, the second service equipment sends the redundant token to the management equipment for storage.

2. The method of claim 1, further comprising:

if it is determined that the token used for processing the access request exists in the token bucket of the first service device, the access request is processed by using the token obtained from the token bucket of the first service device, the first service device is in an decentralized mode, the decentralized mode refers to a mode in which the management device does not participate in flow control, and a supply of the first service device continuously supplies a set number of tokens to the token bucket of the first service device at a set time interval.

3. The method of claim 1, further comprising:

and if the duration of the first service device and the second service device after being switched into the centralization mode reaches a set duration, switching the first service device and the second service device into a non-centralization mode.

4. The method of claim 1, wherein processing the access request stored in the buffer queue of the first service device using the token obtained from the management device comprises:

if the first number of the tokens acquired from the management device is larger than the second number of the access requests stored in the buffer queue of the first service device, processing the second number of the access requests by using the second number of the tokens, and storing the rest of the tokens into a token bucket of the first service device.

5. The method of claim 1, further comprising:

in the centralization mode, after the token bucket of the first service device is fully replenished through the replenishing machine of the first service device, redundant tokens are sent to the management device for storage.

6. The method of claim 1, wherein switching the first service device and the second service device in the service cluster to a centralized mode comprises:

switching the first service device to the centralized mode;

and sending a notification message to the second service equipment so that the second service equipment is switched to the centralization mode according to the notification message.

7. The method of claim 6, wherein the sending a notification message to the second service device to cause the second service device to switch to the centralized mode according to the notification message comprises:

and sending a notification message including identification information corresponding to the user to the second service device, so that the second service device is switched to the centralized mode when determining that the device identification of the second service device is matched with the identification information.

8. The method according to any one of claims 1 to 7, wherein the upper capacity limit of the token bucket of the first service device is determined by a preset traffic threshold, the number of service devices included in the service cluster, and a preset burst traffic parameter;

the upper limit of the capacity of the buffer queue of the first service device is determined by the flow threshold, the number of the service devices and a preset buffer flow parameter;

the number of tokens to be supplied by the supply of the first service device at each of the set time intervals is determined by the traffic threshold, the number of service devices, and the set time interval.

9. A flow control apparatus, located in a first service device in a service cluster, where the first service device is any service device in the service cluster, the apparatus comprising:

the request receiving module is used for receiving an access request sent by a user;

the request caching module is used for storing the access request into a cache queue of the first service equipment if the token bucket of the first service equipment is determined not to have the token for processing the access request;

a mode setting module, configured to switch the first service device and the second service device in the service cluster to a centralized mode, where the centralized mode is a mode in which a management device in the service cluster participates in flow control;

and the request processing module is used for processing the access request stored in the buffer queue of the first service device by using the token acquired from the management device, wherein when the token bucket of the second service device is full, the second service device sends redundant tokens to the management device for storage.

10. A service device, wherein the service device is any service device in a service cluster, and wherein the service device comprises: a memory, a processor, a communication interface; wherein the memory has stored thereon executable code that, when executed by the processor, causes the processor to perform:

receiving an access request sent by a user;

if it is determined that the token bucket of the service device does not contain the token for processing the access request, storing the access request into a buffer queue of the service device;

switching the service device and other service devices in the service cluster into a centralized mode, wherein the centralized mode refers to a mode in which a management device in the service cluster participates in flow control;

and processing the access request stored in the buffer queue of the service equipment by using the token acquired from the management equipment, wherein when the token bucket of the other service equipment is full, the other service equipment sends redundant tokens to the management equipment for storage.

11. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of a service device, causes the processor to perform the flow control method of any one of claims 1 to 8.

12. A service cluster, comprising:

the system comprises a first service device, a second service device and a management device;

the first service device is configured to receive an access request sent by a user, and store the access request into a buffer queue of the first service device if it is determined that no token for processing the access request is located in a token bucket of the first service device; switching the first service device and the second service device into a centralized mode, wherein the centralized mode refers to a mode in which the management device participates in flow control; and processing the access request stored in the buffer queue of the first service device using the token acquired from the management device;

the second service equipment is used for sending redundant tokens to the management equipment for storage when a token bucket of the second service equipment is full in the centralized mode;

the management device is configured to store the token sent by the second service device, and provide the token to the first service device in response to the acquisition request of the first service device.

Images