CN102541472A - A method and device for rebuilding a RAID array - Google Patents

Abstract

The invention relates to a method and a device for reconstructing a RAID array, which are used for executing reconstruction operation of the RAID array, wherein the RAID array is divided into physical blocks with the same size in advance, and the device comprises: the resource allocation unit is used for allocating one or more physical blocks for the logic resources when the logic resources are created and recording the corresponding relation between the logic resources and the physical blocks; the access recording unit is used for maintaining a resource access recording table, and the resource access recording table is used for recording whether each physical block is written with data or not; and the reconstruction processing unit is used for acquiring the physical block with the written state according to the resource access record table when reconstructing the RAID array, and reconstructing the physical block with the written state by taking the physical block as a unit. The invention can greatly improve the reconstruction efficiency and reduce the influence on the service in the reconstruction process to a lower level.

Description

A method and device for rebuilding a RAID array

technical field

The invention relates to network storage technology, in particular to a RAID array rebuilding technology.

Background technique

In the network environment involving data storage of many hosts, in order to improve the reliability and security of data storage, and at the same time for the expansion and flexibility of storage capacity, network storage technology emerges as the times require. Generally speaking, the role of the network storage system is to provide available storage space for client PCs or servers (generally collectively referred to as hosts or Hosts).

The front end of a general network storage system can be connected to the host through an IP network or FC network to provide data storage services for the host. In terms of data transmission, taking an IP-based network storage system as an example, the host can perform data read and write operations based on the standard iSCSI (Internet Small Computer System Interface) protocol network storage system. The core of the network storage system is the storage controller (Storage Controller), which performs data processing and writes the data to the back-end physical disk.

In order to improve the performance of writing physical disks and provide data redundancy, storage controllers usually support Redundant Array of Independent Disks (RAID, also known as RAID array, or array for short) technology. Physical disks are combined in different ways to form a disk group, which provides higher storage performance and reliability than a single disk.

Commonly used RAIDs include RAID0, RAID1, RAID5, RAID6, and RAID10 according to different data organization methods. Various levels of performance and reliability can be provided according to different RAID levels, and it can be guaranteed that in most cases, when one or more disks fail, the data in the remaining member disks can be recovered using the algorithm corresponding to the RAID level, that is, Ensure data is not lost. Through this algorithm, the data in the faulty disk can be reconstructed and written to the hot spare disk. After the reconstruction is completed, the hot spare disk can be used as a member disk of the array to restore the redundancy and reliability of the array. RAID array rebuild.

In a traditional network storage system, when an application needs a part of storage space, it is often pre-allocated a large enough space from the back-end storage system to the application in advance, and business expansion must be considered when allocating space As a result of comprehensive consideration of various factors, the size of the logical resource (LUN) is much larger than the actual storage space currently required, resulting in only a small amount of space in the LUN for storing user data, and a large amount of space for storing user data. space is free. In this case, on the one hand, the user's return on investment decreases; on the other hand, the storage space becomes larger, and the probability of rebuilding will also increase. During the rebuild process, if any other data disks are damaged, data loss will result. In addition, during the reconstruction process, the reconstruction IO will occupy system resources and affect the performance of read and write services. The reconstruction efficiency and reconstruction performance become the key factors affecting the reliability of the storage system.

Thin provisioning is a common feature in network storage systems. Its purpose is to solve the problem of storage oversupply mentioned above and allocate storage space according to actual needs. Its core principle is to "deceive" the client operating system, making the client operating system think that a large LUN has been allocated. For example, the client operating system sees a 2TB LUN, but in fact only a few LUNs are allocated for this resource on the storage device. Ten or hundreds of GB of physical space, the rest of the space is virtual. As the application writes more and more data, the physical storage utilization rate will also become higher and higher. When the actual allocated physical space is insufficient, additional physical space will be allocated to achieve the purpose of expansion on demand.

When a host (usually a variety of servers) recognizes a LUN, what it sees is not the real space, but the virtual space created by thin provisioning. The actual physical space allocated depends on the resource allocation policy, and may only be four times the total space. one-third, or even less.

When creating a thin-provisioned LUN, you need to specify the total capacity of the LUN, the size of the pre-allocated physical space of the LUN, the occupied RAID, and the physical space expansion policy corresponding to the LUN. The total LUN capacity refers to the size of the LUN seen by the client. The size of the pre-allocated physical space of the LUN refers to the size of the physical space actually occupied when the LUN is created. The LUN physical space expansion policy refers to the triggering conditions and expansion policies for the expansion of the LUN physical space. For example, The capacity expansion is triggered when the utilization rate of the pre-allocated physical space of the LUN reaches 80%, and the step size of each expansion is 5% of the total capacity of the LUN. The system allocates resources on the specified RAID based on the size of the pre-allocated physical space of the LUN, and creates a segment table of the LUN to identify the corresponding relationship between the LUN and the RAID, and modifies the segment table of the RAID to indicate that these segments have been used.

Because the actual physical space allocated by the thin-provisioned LUN is not equal to the total space seen by the client, a special LUN linear table needs to be maintained to record the correspondence between the LUN linear space and the actual physical space of the RAID. When a LUN receives an IO write request, it first allocates free space to be accessed by the application from the pre-allocated physical space, modifies the LUN linear table, and writes data. When an IO read request is received on the LUN, if there is a corresponding physical space in the LUN linear table, it will be accessed directly; if not, all 0s will be returned directly.

The most notable feature of thin provisioning is that storage space can be allocated according to the actual needs of the current business. The total storage space is reduced, and the space to be rebuilt is also reduced. That is, the risk of reconstruction is reduced from the perspective of minimizing storage space. However, the implementation of thin provisioning is complicated, and an obvious problem introduced is that it reduces performance. LUN maintains segment tables and linear distribution tables. Each IO needs to look up segment tables and linear distribution tables to find the corresponding physical space. Data channel processing The process is longer and the performance is worse. Therefore, thin provisioning is not suitable for users with high performance and reliability requirements but loose cost control.

In the prior art, another idea for reconstruction optimization is to only rebuild the allocated space in the RAID, and rebuild the allocated area according to the allocation information recorded in the RAID, so as to reduce the task load of the RAID reconstruction and avoid the Useless work, thereby shortening the time required for the rebuild process. However, as mentioned above, the size of logical resources in the storage system is usually much larger than the actual storage space currently required, which will result in only a small amount of space in the logical resources for storing user data, and a large amount of space is idle. Obviously, rebuilding only the allocated areas of the RAID is not an optimal solution. That is, the risk of data loss during the reconstruction process faced by users who have high performance requirements and high reliability requirements but loose cost control cannot be avoided to the greatest extent.

Contents of the invention

In view of this, the present invention provides a RAID array rebuilding device for performing a RAID array rebuilding operation in a network storage system, wherein the RAID array is pre-divided into physical blocks of the same size, and the device includes:

A resource allocation unit, configured to allocate one or more physical blocks to the logical resource when creating the logical resource, and record the correspondence between the logical resource and the physical block;

The access record unit is used to maintain a resource access record table, and the resource access record table is used to record whether each physical block has been written with data; wherein the access record unit records the resource access record table when data is written into the physical block Mark the status of the physical block in the resource access record as written, and when the logical resource is deleted, mark the status of the physical block corresponding to the logical resource in the resource access record as unwritten; and

The reconstruction processing unit is configured to obtain the written physical blocks according to the resource access record table when rebuilding the RAID array, and rebuild the written physical blocks in units of physical blocks.

The present invention also provides a RAID array reconstruction method for performing a RAID array reconstruction operation in a network storage system, wherein the RAID array is pre-divided into physical blocks of the same size, and the method includes:

A. Allocate one or more physical blocks for logical resources when creating logical resources, and record the correspondence between logical resources and physical blocks;

B. Maintain a resource access record table, which is used to record whether each physical block has written data; and when data is written into the physical block, the state of the physical block in the resource access record table is marked as has been written, and when the logical resource is deleted, mark the status of the physical block corresponding to the logical resource in the resource access record as not written; and

C. When rebuilding the RAID array, the physical blocks whose status is written are obtained according to the resource access record table, and the physical blocks whose status is written are rebuilt in units of physical blocks.

Since the present invention only rebuilds the actually used physical space during the reconstruction process, compared with the prior art, the efficiency and speed of reconstruction are greatly improved, and the risk of data loss caused by reconstruction is effectively avoided, and normal data The business impact of reading and writing is very low.

Description of drawings

FIG. 1 is a logical schematic diagram of a network storage device of the present invention.

Fig. 2 is a flow chart of data writing process in an embodiment of the present invention.

Fig. 3 is a processing flow diagram of logical resource deletion in an embodiment of the present invention.

Fig. 4 is a processing flow diagram of array reconstruction in an embodiment of the present invention.

Detailed ways

Generally speaking, on the basis of the existing data flow processing and RAID reconstruction management, the present invention introduces a resource access record table, tracks data writing, and only rebuilds the area where data has been written, thereby minimizing The amount of tasks that need to be reconstructed improves the efficiency of reconstruction, reduces the time of reconstruction, and reduces the impact of reconstruction on the performance of read and write services.

In the present invention, it is necessary to maintain the resource access record table, which can be recorded based on RAID strips, or based on fixed-length RAID resource blocks (please refer to the relevant patents previously applied by the applicant), depending on the specific implementation. Called a physical block (Block), a Block represents a specific length of storage space. The resource access record table can be structured in any format, and can also be located in any achievable position of the system, mainly depending on the requirements for system performance and space. For example, putting it in a lower-level location will improve performance, but the implementation complexity may be slightly higher; otherwise, the performance will be average, but the implementation will be easy. In an implementation manner, the resource access record table may be maintained at the level of the RAID module of the storage system. In addition, in order to improve the retrieval efficiency and reduce the system resources occupied by the resource access record table, the bitmap method can be used for recording. One bit corresponds to a Block. For example, a bit of 1 indicates that the corresponding Block has written data; a bit of 0 indicates that the corresponding No data has been written to the block.

Please refer to Figure 1, Figure 2 and Figure 3. The RAID array rebuilding device 20 of the present invention is applied in the network storage system 10, and the network storage system 10 further includes a read-write business processing device 30, and the RAID array rebuilding device 20 includes a resource allocation unit 22, an access recording unit 24, and a reconstruction processing Unit 26. The above-mentioned devices are abstracted from the logic level, and the typical way is to realize it through the processor plus program code, but it can also be realized through hardware, firmware or a combination of software and hardware. The general processing flow of the above-mentioned RAID array rebuilding device 20 is described below.

Step 101 , allocate one or more physical blocks for the logical resource when creating the logical resource, and record the corresponding relationship between the logical resource and the physical block; this step is executed by the resource allocation unit 22 .

In the present invention, the RAID array is pre-divided into physical blocks of the same size, and a typical physical block can be a stripe (Stripe), or a fixed-length resource block proposed by the applicant in a previous patent application; the present invention The division method and actual size of stripes or resource blocks are not concerned; the essence is to divide the physical resources of the RAID array into blocks, which are collectively referred to as physical blocks in the present invention.

When the network storage system creates a logical resource, it selects one or more physical blocks from the RAID array and assigns it to the logical resource. In this way, the logical resource has one or more corresponding physical blocks, and the resource allocation unit 22 allocates these The corresponding relationship is recorded for subsequent processing.

It should be noted that the network storage system 10 may support the thin provisioning technology. Once the user enables the thin provisioning technology, the size of the logical resource may be inconsistent with the sum of the physical blocks. Therefore, the corresponding relationship mentioned here means that the physical blocks are Which logical resource is assigned to does not care about whether the size of the logical resource corresponds to the physical resource.

Step 102, maintaining a resource access record table, the resource access record table is used to record whether data is written into each physical block; when data is written into the physical block, the status of the physical block in the resource access record table is marked as has been written, and when the logical resource is deleted, the status of the physical block corresponding to the logical resource in the resource access record is marked as unwritten; this step is performed by the access record unit 24.

As mentioned above, a typical and effective implementation of the resource access record table is Bitmap, which is a very popular technology in the field of network storage, and will not be described in detail here. The update processing of each physical block in the resource access record table can be set in series with the processing of the read-write service device or in parallel with the read-write service.

For serial processing, when a read-write service processing device (usually a RAID service processing module) receives a write command from a logic resource or an application program in a network storage system, it can proceed according to the general write process flow. Processing, if the data writing is successful, it can go to step 102 for processing, otherwise return, such as prompting that the writing operation of the writing end failed.

For parallel processing, step 102 is parallel to the processing of the write command by the read-write service device. Even if the final processing result of the write command is failure, step 102 may still mark the status of the physical block as written. The result of this is that the state record of the physical block in the resource access record table may not match the actual state of the physical block. However, this is more efficient and can improve the processing speed of business processes. And even if there is a non-conforming situation, the problem caused is only a slight increase in subsequent reconstruction work, that is to say, some physical blocks that should not be reconstructed are reconstructed.

The user may delete the logical resource that has been created. Before deleting the logical resource, the access record unit 24 can first obtain the corresponding physical blocks occupied by the logical resource, and then change the status of these physical blocks to unavailable in the resource access record table. Write, execute the deletion of logical resources and then release the physical space to complete the operation of deleting the entire logical resource. After the logical resource is deleted, the resource access record table needs to be updated accordingly, because the subsequent reconstruction is carried out according to the records of the status of each physical block in the logical resource record table, and timely updating can ensure that the scope of reconstruction is limited to the business or application Each physical block actually occupied by the layer.

Furthermore, at the initial stage, it is obvious that the state of all physical blocks is unwritten data. However, data may be written into the physical block repeatedly, so when data is written into the physical block, you can first check whether the status of the physical block has been written, if it is returned, otherwise continue. It is equivalent to making a state change on the first write, and skipping it later. The optimization scheme for the above operations is to directly update the physical block as written every time data is written to the physical block, which can subtract the process of reading and judging, thereby improving efficiency.

Furthermore, the resource access record table may show an invalid state, such as the Bitmap cannot be set, or the update cannot be unsuccessful. At this time, for more rigorous consideration, before marking the state of the physical block, check whether the resource access record table is valid, if so, update the resource access record table, otherwise do not update and transfer to other different reconstruction processing units, such as It is said to transfer to an existing software/hardware implemented reconstruction processing unit to perform a common reconstruction processing flow. In this way, it can be ensured that the existing reconstruction processing unit is fully utilized as a backup, and the reliability of reconstruction processing is improved. Correspondingly, when updating the state of the physical block, it may be checked whether the update is successful, and if the update is successful, continue, otherwise, mark the resource access record table as invalid.

Step 103 , when rebuilding the RAID array, obtain the written physical blocks according to the resource access record table, and rebuild the written physical blocks in units of physical blocks. This step is executed by the reconstruction processing unit 26 .

Please refer to Figure 4. When the RAID array is in a degraded state, the system can prompt the administrator to rebuild, or trigger the rebuild immediately. Once the system finds that there is an available hot spare disk, the system can start the rebuild operation. Usually rebuilding is based on the level of the RAID array, using the corresponding verification algorithm to calculate the data on the damaged disk and write it to the corresponding hot spare disk. The physical blocks obtained by the present invention are all physical blocks with data, so the reconstruction work is very meaningful, and a large amount of unnecessary reconstruction work is saved. Compared with the implementation manner of rebuilding allocated physical resources in the prior art, the present invention rebuilds physical blocks that have been written into data, and the efficiency is greatly improved. Because often many physical resources that have been allocated have not actually written data.

The reconstruction processing unit 26 can further perform various optimization processes. If new data is written into the physical block during the reconstruction process, the reconstruction processing unit 26 may simultaneously write the new data into the data disk and the hot spare disk. The advantage of this is that the writing of business data and the rebuilding process do not affect each other during the rebuilding process, which means that the rebuilding is completed when the business is written, so the rebuilding processing unit does not need to rebuild the newly written data separately.

In terms of implementation, there are two ways. For example, when reconstruction starts, the reconstruction processing unit 26 can generate a reconstruction list (the current state is a list composed of written physical blocks) according to the current resource access record table, and process each physical block according to the order of the reconstruction list to rebuild.

Of course, rebuilding can also be performed directly according to the resource access record table. The rebuilding processing unit 26 takes N (N is a natural number) physical blocks whose state is written and rebuilds each time until all the physical blocks are reconstructed.

Furthermore, when new data is written in the reconstruction process, according to the method of step 102, the access record unit 24 will update the state of the physical block to have been written, which can ensure the integrity and real-time performance of the resource access record table . Assuming that the hot spare disk fails during the rebuilding process, and replace it with another hot spare disk, since the resource access record table is complete (or updated in time), the rebuilding process will not be affected in any way.

The above descriptions are only preferred implementation modes of the present invention, and any equivalent modifications made based on the spirit of the present invention shall fall within the scope of the claims of the present invention.

Claims (10)

1. a RAID array rebuild device is used to carry out the RAID array rebuild operation in the network store system, and wherein said RAID array is divided into big or small same physical piece in advance, and this device comprises:

Resource allocation unit, being used for when creating logical resource is that logical resource distributes one or more physical blocks, and the corresponding relation between record logical resource and the physical block;

The Visitor Logs unit is used to safeguard a resource access record sheet, and this resource access record sheet is used to write down each physical block and whether has been written into data; Wherein this Visitor Logs unit when having data to write physical block with the status indication of this physical block in the resource access record sheet for writing; And when said logical resource is deleted, with the status indication of this logical resource corresponding physical piece in the resource access record for not writing; And

The reconstruction process unit is used for when rebuilding the RAID array, obtaining the physical block of state for having write according to said resource access record sheet, is that the unit rebuilds for the physical block that has write state with the physical block.

2. reconstructing device according to claim 1; It is characterized in that; When said reconstruction process unit was further used in finding process of reconstruction, having the new data write state to be the physical block that does not write, the heat that the data that need the Write fault disk is write this failed disk and correspondence was equipped with disk.

3. reconstructing device according to claim 1; It is characterized in that said network store system also comprises the read-write business processing device, it is used for the deal with data read write command; If handle failure then return, if write order and handle and successfully then change the Visitor Logs cell processing over to.

4. reconstructing device according to claim 3; Wherein said Visitor Logs unit was further used for before mark physics bulk state; Check whether said resource access record sheet is effective; If then upgrade said resource access record sheet, otherwise do not upgrade and change other different reconstruction process unit over to; And whether inspection upgrades success when upgrading said physical block state, if upgrades successfully then continuation, otherwise said resource access record sheet is labeled as invalid.

5. reconstructing device according to claim 1, wherein said Visitor Logs unit are further used for obtaining this physical block current state before the mark physics bulk state, if current state is then returned otherwise continuation for writing.

6. a RAID array rebuild method is used to carry out the RAID array rebuild operation in the network store system, and wherein said RAID array is divided into big or small same physical piece in advance, and this method comprises:

A, when creating logical resource, be that logical resource distributes one or more physical blocks, and the corresponding relation between record logical resource and the physical block;

B, resource access record sheet of maintenance, this resource access record sheet is used to write down each physical block and whether has been written into data; And when having data to write physical block with the status indication of this physical block in the resource access record sheet for writing, and when said logical resource is deleted, with the status indication of this logical resource corresponding physical piece in the resource access record for not writing; And

C, when rebuilding the RAID array, obtaining the physical block of state for having write according to said resource access record sheet, is that the unit rebuilds for the physical block that has write state with the physical block.

7. method according to claim 6; It is characterized in that; Step C further is included in when finding to have in the process of reconstruction new data write state to be the physical block that does not write, and the heat that the data that need the Write fault disk is write this failed disk and correspondence is equipped with disk.

8. method according to claim 6 is characterized in that, also comprises:

D, deal with data read write command are if handle failure then return, if write order and handle and successfully then change step B over to.

9. method according to claim 6; Wherein step B further comprises: before mark physics bulk state; Check whether said resource access record sheet is effective,, otherwise do not upgrade and change other different reconstruction process flow processs over to if then upgrade said resource access record sheet; And whether inspection upgrades success when upgrading said physical block state, if upgrades successfully then continuation, otherwise said resource access record sheet is labeled as invalid.

10. method according to claim 6, wherein said step B further comprises: before new physical block state more, obtaining this physical block current state, if current state is then returned otherwise continuation for writing.

Images