CN107741917A - An improved IO current limiting processing method and system - Google Patents

Abstract

The present invention discloses an IO current limiting processing method and system, wherein the method includes: step 1, setting an IO load limiting module at the storage device end; step 2, the IO load limiting module sending the host to the storage device end Data is transferred according to preset IOPS limits and bandwidth limits. By setting the IO load limit module on the storage device side, the data delivered by the host to the storage device side is transmitted according to the preset IOPS limit and bandwidth limit, which balances the load of the delivered IO to the LUN, and stabilizes the delivery of the storage system. The stability of IO greatly guarantees the stability of the system.

Description

An improved IO current limiting processing method and system

technical field

The invention relates to the technical field of data storage, in particular to an improved IO current limiting processing method and system.

Background technique

With the rapid development of computer technology, many technologies have been greatly improved, especially the successful use of back-end storage system partitions, which meets the security and stability requirements of the data sent by the server host. Moreover, the successful use of cache and flash memory meets the needs of various computer fields for computing resources and computing speed. This processing method greatly accelerates the speed of data processing.

However, although the common IO data transmission method between the host and the storage system enhances the efficiency of data transmission between the host and the storage system, it does not consider the bearing capacity of the LUN. If the IOPS delivered by the host layer is too large, or the bandwidth is too high, It is likely to cause instability of IO transmission, or even IO interruption.

Therefore, how to stabilize the data transmission between the host and the storage device and strengthen the security management of the data is an urgent problem to be solved.

Contents of the invention

The object of the present invention is to provide an IO current limiting processing method and system, which stabilizes the data transmission between the host and the storage device, and strengthens the security management of the data.

In order to solve the above technical problems, an embodiment of the present invention provides an IO current limiting processing method, including:

Step 1, setting the IO load limiting module on the storage device side;

Step 2, the IO load limiting module transmits the data delivered by the host to the storage device according to the preset IOPS limit and bandwidth limit.

Wherein, the step 1 includes:

Create a fast_path_fbr structure and a slow_path_fbr structure in the IO load limiting module, and register them in the system;

The system classifies and stores the IO issued by the host into the fast_path_fbr structure and the slow_path_fbr structure, and creates corresponding fast path threads and slow path threads, wherein the fast path threads are used to process the IO data in the fast_path_fbr, The slow path thread is used to process common non-IO commands, as well as fail IO in the fast_path_fbr structure and IO exceeding the waiting time threshold.

Wherein, before the step 1, it also includes:

Analyze the HCCB structure of the system, obtain the flag of the command issued by the host, and filter the non-IO commands.

Wherein, after the step 3, it also includes:

Estimate and output the IO delay time according to the processing speed of the fast_path_fbr structure and the slow_path_fbr structure and the stored IO quantity in the slow_path_fbr structure.

Wherein, after said step 3, also include:

Timing setting is performed on the IO load limiting module, and the opening time and closing time of the IO load limiting module are set.

In addition, the embodiment of the present invention also provides an IO current limiting processing system, including an IO load limiting module set on the storage device side, used to set IOPS and bandwidth are limited when the LUN specified on the storage device side is written or read.

Wherein, it also includes a timing module connected with the IO load limiting module, which is used to set the opening time and closing time of the IO load limiting module according to the needs of users.

Wherein, the IO load limiting module includes an IO retention unit for storing pending IO units that need to process IOs and trapped IOs.

Wherein, it also includes a delay feedback module connected with the IO unit to be processed and the IO retention unit storing the trapped IO of the IO load limiting module, for storing the IO quantity according to the IO unit to be processed and storing the trapped IO And the IO delay time is estimated and output by the IO retention unit, which is used to predict the IO processing delay time.

Wherein, it also includes a limit input module connected with the IO load limit module, for inputting the IOPS value and the bandwidth value of the IO load limit module according to user's needs.

Compared with the prior art, the IO current limiting processing method and system provided by the embodiments of the present invention have the following advantages:

In the IO current limiting processing method and system, by setting an IO load limiting module on the storage device side, the data sent by the host to the storage device side is transmitted according to the preset IOPS limit and bandwidth limit, which balances the IO load limit delivered to The load of the LUN prevents the possibility of LUN overload, stabilizes the stability of the IO delivered by the storage system, greatly ensures the stability of the system, and improves the reliability of the system.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are For some embodiments of the present invention, those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flow chart of the steps of a specific implementation of the IO current limiting processing method provided by the embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a specific implementation manner of an IO current limiting processing system provided by an embodiment of the present invention.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Please refer to Figures 1-2, Figure 1 is a schematic flow chart of the steps of a specific implementation of the IO current limiting processing method provided by the embodiment of the present invention; Figure 2 is a specific implementation of the IO current limiting processing system provided by the embodiment of the present invention Schematic diagram of the embodiment.

In a specific implementation manner, the IO current limiting processing method includes:

Step 1, setting the IO load limiting module on the storage device side;

Step 2, the IO load limiting module transmits the data delivered by the host to the storage device according to the preset IOPS limit and bandwidth limit.

It should be noted that in the present invention, IOPS and bandwidth can be limited at the same time, or one of them can be limited separately, as long as the security of data transmission can be ensured, and the carrying capacity of the LUN in the storage device is not exceeded, the IOPS and bandwidth can be guaranteed. Safe and reliable operation is sufficient, and the present invention does not limit the limits of IOPS and bandwidth.

In the IO current limiting processing method, by improving the management method of data transmission between the host and the storage device, the mode of direct transmission of commands and data IO between the original host and the volume or virtual volume of the back-end storage system is changed to a storage The IO load limiting module is added to the device side, so that when the host issues commands to the storage device and performs data transmission, it does not affect the IO delivery of the host, and can also limit the load of writing or reading a specific LUN, so that Keep working within the loadable range.

Furthermore, in order to improve management efficiency, on the basis of the above embodiments, the step 1 includes:

Create a fast_path_fbr structure and a slow_path_fbr structure in the IO load limiting module, and register them in the system;

The system classifies and stores the IO issued by the host into the fast_path_fbr structure and the slow_path_fbr structure, and creates corresponding fast path threads and slow path threads, wherein the fast path threads are used to process the IO data in the fast_path_fbr, The slow path thread is used to process common non-IO commands, as well as fail IO in the fast_path_fbr structure and IO exceeding the waiting time threshold.

Because the IO issued by the host is constant, and in the present invention, an IO load limiting module is added, so that some normal issued IOs cannot be processed normally and need to wait, while the processing of different types of shampoo IOs The priorities are different. Therefore, by classifying and storing these issued IOs first, creating different threads, and processing them through different threads, the management efficiency and processing efficiency are further improved. It does not affect the IO delivery on the host side, but also protects the LUN from being overloaded at a certain moment and causing IO interruption.

In order to further improve management efficiency, before step 1, it also includes:

Analyze the HCCB structure of the system, obtain the flag of the command issued by the host, and filter the non-IO commands.

HCCB structure is the abbreviation of host command control block, host command control block.

By analyzing the HCCB structure of the system, obtain the flag of the command issued by the host, that is, the IO and non-IO flags, filter the non-IO commands, and limit the value to the IO command level, so that the fast_path_fbr structure and the slow_path_fbr structure are stored in the host Issued IO, but not others, improves management efficiency.

However, due to the addition of the IO load limiting module, during the IO processing process, the task can only be processed within the loadable range, but the host will send it as long as there is a task, which is likely to cause a crash and exceed the processing capacity. A large number of IOs issued by the host will be stored in the fast_path_fbr structure and slow_path_fbr structure, making the actual corresponding IO delay very large, the system cannot predict, and management disorder occurs. Therefore, in order to solve this problem, in the above step 3 After that, also include:

Estimate and output the IO delay time according to the processing speed of the fast_path_fbr structure and the slow_path_fbr structure and the stored IO quantity in the slow_path_fbr structure.

According to the number of two types of fbr, the processing speed is fed back to the upper layer, and the IO delay time is estimated through the number of slow_path_fbr, and presented to the upper layer, which is used to predict the IO processing delay time, prevent the host from aborting IO, and improve the operation security of the system performance and reliability, improve management efficiency, avoid LUN IO overload, and ensure system stability.

In the process of data transmission between the host and the storage device in the present invention, different users have different restrictions on the IO load in different time periods, and even in some time periods, some LUNs need to limit IOPS, and some LUNs need to limit bandwidth. Make a different setup, so after said step 3 also include:

Timing setting is performed on the IO load limiting module, and the opening time and closing time of the IO load limiting module are set.

It should be pointed out that the present invention may have different or the same restrictions on the load of sending IOs to different LUNs, but in general, due to differences in the functions of different LUNs, the general restrictions will be different, and for the same LUN, The load in different time periods is different, the user can add timing settings according to the needs, and select the current limit time according to the choice.

In addition, the embodiment of the present invention also provides an IO current limiting processing system, including an IO load limiting module 20 set on the storage device end 30, which is used to issue a command to the storage device end 30 at the host 10 to process data During transmission, IOPS and bandwidth are limited when writing or reading to the LUN specified at the storage device end 30 .

In order to ensure stable and reliable system operation and improve operating efficiency, in one embodiment of the present invention, the IO current limiting processing system further includes a timing module connected to the IO load limiting module 20 for According to the need to set the on time and off time of the IO load limiting module 20.

By setting the timing module, the opening time and closing time of the IO load limiting module 20 can be set according to the user's needs, and different limitations can be performed, such as IOPS limitation in time period A, bandwidth limitation in time period B, and simultaneous IOPS and timeout in time period C. Limit to improve the system's high reliability and high efficiency.

In order to further improve the management efficiency of the system for issuing IOs, the IO load limiting module 20 includes an IO retention unit for storing pending IO units that need to be processed for IO and retained IOs.

By creating the fast_path_fbr structure of the IO unit to be processed and the slow_path_fbr structure of the IO retention unit in the IO load limiting module 20, and registering in the system;

The system classifies and stores the IOs issued by the host 10 into the fast_path_fbr structure and the slow_path_fbr structure, and creates corresponding fast path threads and slow path threads, wherein the fast path threads are used to process the IO data in the fast_path_fbr , the slow path thread is used to process common non-IO commands, as well as the fail IO in the fast_path_fbr structure and the IO exceeding the waiting time threshold, so as to ensure resource optimization.

The IO load limiting module 20 in the present invention can set different thresholds for different LUNs, use different levels of fast_path_fbr to correspond to the thresholds of different LUNs, and set a higher threshold with higher priority, and use fastpath more frequently, In order to improve the operating efficiency of the system.

In order to further improve the management efficiency of the system and avoid host 10 abort IO, in one embodiment, the IO current limiting processing system also includes an IO retention unit that is connected to the pending IO unit of the IO load limitation module 20 and stores the trapped IO The connected delay feedback module is used for estimating and outputting the IO delay time according to the IO unit to be processed and the number of IOs stored by the trapped IO through the IO retention unit, so as to predict the IO processing delay time.

By predicting the IO processing delay time, the host 10 is prevented from aborting IO, and the host 10 is prevented from issuing excessive IOs, thereby improving the operational reliability of the system.

In the present invention, since different thresholds may need to be set for different LUNs to implement different functions to ensure resource optimization, in one embodiment of the present invention, the IO current limiting processing system further includes The limit input module connected with the IO load limit module 20 is used to input the IOPS value and the bandwidth value of the IO load limit module 20 according to the needs of users.

It should be pointed out that, in the present invention, there is no limit to the input mode of the limited input module. The input interface can be set separately and set by the user according to the needs, or it can be called and stored in the pre-stored space according to the operation needs of the system. A restriction instruction executes the restriction scheme in it.

To sum up, in the IO current limiting processing method and system provided by the embodiments of the present invention, by setting an IO load limiting module on the storage device side, the data sent by the host to the storage device side is based on the preset IOPS limit and bandwidth limit The transmission balances the load of the issued IO to the LUN, so that the possibility of LUN overload does not occur, stabilizes the stability of the storage system issued IO, greatly ensures the stability of the system, and improves the operating reliability of the system .

The IO current limiting processing method and system provided by the present invention have been introduced in detail above. In this paper, specific examples are used to illustrate the principle and implementation of the present invention, and the descriptions of the above embodiments are only used to help understand the method and core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, some improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims (10)

1. An IO current limiting processing method is characterized in that, comprising: Step 1, setting the IO load limiting module on the storage device side; Step 2, the IO load limiting module transmits the data delivered by the host to the storage device according to the preset IOPS limit and bandwidth limit. 2. The IO current limiting processing method according to claim 1, wherein said step 1 comprises: Create a fast_path_fbr structure and a slow_path_fbr structure in the IO load limiting module, and register them in the system; The system classifies and stores the IO issued by the host into the fast_path_fbr structure and the slow_path_fbr structure, and creates corresponding fast path threads and slow path threads, wherein the fast path threads are used to process the IO data in the fast_path_fbr, The slow path thread is used to process common non-IO commands, as well as fail IO in the fast_path_fbr structure and IO exceeding the waiting time threshold. 3. The IO current limiting processing method according to claim 2, characterized in that, before said step 1, further comprising: Analyze the HCCB structure of the system, obtain the flag of the command issued by the host, and filter the non-IO commands. 4. The IO current limiting processing method according to claim 1, characterized in that, after the step 3, further comprising: Estimate and output the IO delay time according to the processing speed of the fast_path_fbr structure and the slow_path_fbr structure and the stored IO quantity in the slow_path_fbr structure. 5. The IO current limiting processing method according to claim 1, further comprising: after said step 3: Timing setting is performed on the IO load limiting module, and the opening time and closing time of the IO load limiting module are set. 6. A kind of IO current limiting processing system, it is characterized in that, comprise the IO load limiting module that is arranged on storage device end, when host computer issues order to described storage device end and carry out data transmission, set on described storage device end IOPS and bandwidth are limited when the specified LUN is written or read. 7. The IO current limiting processing system as claimed in claim 6, further comprising a timing module connected to the IO load limiting module, for setting the opening time and closing time of the IO load limiting module according to user needs . 8 . The IO current limiting processing system according to claim 7 , wherein the IO load limiting module includes an IO retention unit for storing pending IO units that need to process IOs and trapped IOs. 9. The IO current limiting processing system according to claim 8, further comprising a delay feedback module connected with the IO unit to be processed of the IO load limiting module and the IO retention unit storing the trapped IO, for According to the IO unit to be processed and the number of IOs stored by the trapped IO, the IO delay time is estimated and output by the IO retention unit, so as to predict the IO processing delay time. 10. IO current limiting processing system as claimed in claim 9, is characterized in that, also comprises the limit input module that is connected with described IO load limit module, is used for inputting the IOPS value of described IO load limit module and bandwidth value.