CN115454343A - Data processing method, device and medium based on RAID chip - Google Patents

Abstract

The invention discloses a data processing method based on a RAID chip, which realizes RAID coding and/or RAID decoding based on the RAID chip, namely realizes data storage and/or data recovery based on the RAID chip, reduces consumption of hardware cache and saves CPU resources. Meanwhile, the general task is divided into a plurality of slice subtasks, namely, a large task is divided into a plurality of small tasks which are processed by different data processing units, so that the slice subtasks are beneficial to fine management and control in the aspect of task scheduling, and fault location and fault elimination are easy when any one slice subtask has an error. In the data processing aspect, the multiple data processing units simultaneously process each slice subtask, so that the data processing efficiency can be improved, the area of a hardware system is reduced, CPU resources are saved, and the data processing efficiency of the RAID is improved.

Description

Data processing method, device and medium based on RAID chip

technical field

The present application relates to the technical field of data processing, in particular to a data processing method, device and medium based on a RAID chip.

Background technique

Disk array (Redundant Arrays of Independent Disks, RAID) is a large-capacity and redundant disk group obtained by combining multiple independent disks. Using RAID storage technology can greatly increase storage capacity and improve system input and output request capabilities. , and through data distributed storage technology, parallel access means and information redundancy technology to improve data reliability.

The data in RAID is distributed to each disk in units of blocks. When the data of one disk in RAID is damaged, the damaged data can be recovered by encoding or decoding RAID by using the remaining undamaged data and corresponding checksum information. Currently, RAID software is usually run on the CPU for encoding and decoding to restore damaged data. Obviously, this method will occupy too much CPU resources. When the task of processing data is large, increasing the CPU frequency to meet business needs will lead to High power consumption, if the CPU frequency is not increased, the task processing efficiency will be low.

It can be seen that how to avoid occupying too many CPU resources while improving the encoding and decoding efficiency of RAID, and then improve the efficiency of data processing is an urgent problem to be solved by those skilled in the art.

Contents of the invention

The purpose of this application is to provide a data processing method, device and medium based on a RAID chip, which can improve the encoding and decoding efficiency of RAID without occupying CPU resources, thereby improving the data processing efficiency.

In order to solve the above technical problems, the application provides a data processing method based on RAID chips, which is applied to RAID chips, including:

After obtaining the task request, parsing the task request to obtain corresponding data-related information to be processed;

Retrieving data blocks to be processed according to the relevant information of the data to be processed;

Segmenting the total task corresponding to the task request into a plurality of sliced subtasks according to a preset segmentation rule;

Each of the slicing subtasks and corresponding data blocks to be processed are transmitted to different data processing units for RAID encoding and/or RAID decoding.

Preferably, the transmitting each of the slicing subtasks and corresponding data blocks to be processed to different data processing units for RAID encoding and/or RAID decoding includes:

According to the characteristics of the RAID storing the data block to be processed, a corresponding RAID algorithm is invoked to perform RAID encoding and/or RAID decoding.

Preferably, the data related information to be processed at least includes: the number of data blocks, the number of check blocks, the number of slice subtasks, and data block address information.

Preferably, said data processing unit performing RAID encoding and/or RAID decoding includes:

When the data processing unit performs RAID encoding, it includes:

calling the corresponding RAID algorithm so as to calculate the number of check blocks according to the number of data blocks and the number of slice subtasks;

and generate a check block corresponding to the data block to be processed;

storing the data block to be processed and the corresponding check block in the corresponding disk;

When the data processing unit performs RAID decoding, it includes:

calling the corresponding RAID algorithm so as to determine the target data block of the missing disk according to the data block to be processed and the check block corresponding to the data block to be processed;

Data in the target data block is restored.

Preferably, said retrieving data block to be processed according to said data-to-be-processed related information includes:

Determine the start address of the data block to be processed according to the data block address information;

The data blocks to be processed are retrieved according to the start address and the number of data blocks.

Preferably, after the data block to be processed is called according to the relevant information of the data to be processed, the method further includes:

Perform information check on each of the data blocks to be processed according to the address information of the data block, and store the check result.

Preferably, before the parsing of the task request to obtain the relevant information of the data to be processed corresponding to the task unit, the method further includes:

Check whether the task request satisfies the preset condition, if not, end the data processing, and transmit the abnormal signal to the response processing unit.

In order to solve the above-mentioned technical problems, the present application also provides a data processing device based on a RAID chip, which is applied to a RAID chip, including:

A parsing module, configured to parse the task request to obtain corresponding data related information to be processed after the task request is obtained;

A call module, configured to call the data block to be processed according to the relevant information of the data to be processed;

A segmentation module, configured to segment the total task corresponding to the task request into a plurality of sliced subtasks according to preset segmentation rules;

The transmission module is configured to transmit each of the slicing subtasks and corresponding data blocks to be processed to different data processing units for RAID encoding and/or RAID decoding.

In order to solve the above technical problems, the present application also provides a data processing device based on a RAID chip, including a memory for storing computer programs;

The processor is configured to implement the steps of the RAID chip-based data processing method when executing the computer program.

In order to solve the above-mentioned technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data based on the RAID chip is realized. The steps of the processing method.

A data processing method based on a RAID chip provided by the present invention is applied to a RAID chip, and includes: after obtaining a task request for data processing, parsing the task request to obtain corresponding data-related information to be processed, and according to the data-to-be-processed related information The information retrieves the data blocks to be processed, and then divides the total task corresponding to the task request into multiple slice subtasks according to the preset segmentation rules, so that different data processing units can process different slice subtasks and corresponding pending tasks. Process data blocks for RAID encoding and/or RAID decoding. It can be seen that the technical solution provided by this application implements RAID encoding and/or RAID decoding based on the RAID chip, that is, implements data storage and/or data recovery based on the RAID chip, reduces hardware cache consumption, and saves CPU resources. At the same time, the total task is divided into multiple sliced subtasks, that is, a large task is divided into multiple small tasks, which are processed by different data processing units. Therefore, in terms of task scheduling, sliced subtasks are beneficial With fine-grained management and control, it is easy to locate and troubleshoot faults when an error occurs in any slice subtask. At the data processing level, multiple data processing units simultaneously process each slice subtask to improve the efficiency of data processing, thereby reducing the area of the hardware system, saving CPU resources, and improving the data processing efficiency of RAID.

In addition, the present application also provides a data processing device and medium based on a RAID chip, corresponding to the above data processing method based on a RAID chip, and has the same effect as above.

Description of drawings

In order to illustrate the embodiments of the present application more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. As far as people are concerned, other drawings can also be obtained based on these drawings on the premise of not paying creative work.

Fig. 1 is the flow chart of a kind of data processing method based on RAID chip that the embodiment of the present application provides;

FIG. 2 is a schematic diagram of a hardware architecture based on RAID data processing provided by an embodiment of the present application;

Fig. 3 is the schematic diagram of a kind of RAID coding based on RAID chip provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of RAID decoding based on a RAID chip provided by an embodiment of the present application;

FIG. 5 is a structural diagram of a data processing device based on a RAID chip provided in an embodiment of the present application;

6 is a structural diagram of a data processing device based on a RAID chip provided by another embodiment of the present application;

The reference signs are as follows: 1 is an information processing module, and 2 is a task processing module.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of this application.

The core of the present application is to provide a data processing method, device and medium based on RAID chips, which are applied to RAID chips to avoid occupying CPU resources. When performing data processing, the total task is divided into multiple slice subtasks, and the Different data processing units perform RAID encoding and/or RAID decoding on slice subtasks, thereby improving data processing efficiency.

In order to enable those skilled in the art to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the drawings and specific implementation methods.

RAID is a large-capacity and redundant disk group obtained by combining multiple independent disks. Using RAID storage technology can greatly increase storage capacity and improve system input and output request capabilities. Through distributed data storage technology, Parallel access means and information redundancy technology improve data reliability.

The data in RAID is distributed to each disk in units of blocks. When the data of one disk in RAID is damaged, the damaged data can be recovered by encoding or decoding RAID by using the remaining undamaged data and corresponding checksum information. Currently, RAID software is usually run on the CPU for encoding and decoding to restore damaged data. Obviously, this method will occupy too much CPU resources. When the task of processing data is large, increasing the CPU frequency to meet business needs will lead to High power consumption, if the CPU frequency is not increased, the task processing efficiency will be low.

In order to solve the above-mentioned technical problems, the embodiment of the present application provides a data processing method based on a RAID chip, which is applied to a RAID chip, thereby avoiding occupying CPU resources, and dividing the total task when performing RAID encoding and RAID decoding There are multiple slice subtasks, and different data processing units process different slice subtasks, thereby improving data processing efficiency.

Fig. 1 is the flow chart of a kind of data processing method based on RAID chip that the embodiment of the present application provides, as shown in Fig. 1, this method is applied to RAID chip, comprises:

S10: After obtaining the task request, analyze the task request to obtain the corresponding information about the data to be processed;

In a specific embodiment, in order to avoid taking up too many CPU resources, the data processing scheme provided by the embodiment of the present application is applied to a RAID chip. FIG. 2 is a schematic diagram of a hardware architecture based on RAID data processing provided by the embodiment of the present application. As shown in Figure 2, the RAID chip includes an information processing module 1 and a task processing module 2, wherein the information processing module 1 includes a task scheduling unit, a data address scheduling unit, a data parameter scheduling unit, an address format checking unit, and a task slice processing unit and response processing unit.

Many kinds of RAID algorithms are derived according to the strategy and architecture of using or combining these three technologies, among which RAID5, RAID6 and RAID TP are the most widely used. In RAID, data is distributed to each disk in units of blocks, and RAID5 stores data and corresponding parity information on each disk that forms RAID5. When data on one disk of RAID5 is damaged, the damaged data can be restored by using the remaining undamaged data and corresponding checksum information. Therefore, RAID5 requires a hardware computing engine for encoding or decoding. RAID6 adds a check disk on the basis of RAID5, that is, RAID6 can restore the lost data of two disks, and RAID6 needs two hardware computing engines for encoding or decoding. RAID TP adds a check disk on the basis of RAID6, that is, RAID TP can restore the lost data of three disks, and RAID TP requires three hardware computing engines for encoding or decoding.

Therefore, when performing RAID encoding and RAID decoding, it is necessary to determine the number of data blocks corresponding to the data to be processed, the number of check blocks, data block address information, and valid data bits. In order to improve the data processing speed, the task processing module 2. The total task requested by the task needs to be divided into multiple slice subtasks. Therefore, information such as the number of slice subtasks needs to be obtained before data processing.

During implementation, after the task scheduling unit obtains the task request, it parses the task request to obtain information about the data to be processed. The information about the data to be processed includes at least the number of data blocks, the number of check blocks, the number of slice subtasks, Address information and valid data bits, etc., wherein, the data block address information includes address list pointers and parameter list pointer addresses, etc. Then, the related information of the data to be processed is transmitted to the data address scheduling unit, the data parameter scheduling unit and the address format checking unit according to the type.

S11: Retrieving the data block to be processed according to the relevant information of the data to be processed;

Further, the data address scheduling unit schedules the address information that requires data processing according to information such as the address list pointer and address quantity in the data block address information. It should be noted that when RAID encoding is performed, the address information that requires data processing For the disk address information of the data to be stored, when performing RAID decoding, the address information that needs to be processed includes the disk address information corresponding to the damaged data block and the new disk address information to be replaced. The data address scheduling unit schedules the required information and transmits it to the address format checking unit.

The data parameter scheduling unit schedules the required parameter information according to the pointer address of the parameter list, and transmits it to the address format checking unit. Wherein, the parameter information includes parameter information such as the size of each slice subtask, the size of each total task, and the like. After the address format checking unit acquires various types of information, it checks and verifies the information, and transmits the information to the task slice processing unit, and then the task slice processing unit transmits it to the data input unit.

The data input unit determines the start address of the data block according to the address information of the data block, and calls the data block to be processed according to the number of each data block of the start address.

S12: Divide the total task corresponding to the task request into multiple subtasks according to the preset segmentation rules;

S13: Transmit each slice subtask and corresponding data blocks to be processed to different data processing units for RAID encoding and/or RAID decoding.

Further, the task slicing processing unit divides the total task corresponding to the task request into multiple sliced subtasks according to the preset slicing rules, wherein the preset slicing rules can be user-defined fixed unit size The slicing subtask may also be divided evenly according to the number of slicing subtasks configured by the user, which is not limited in this application.

After the total task is divided into multiple slice subtasks, correspondingly, there are multiple data blocks for each slice subtask, as shown in Figure 2, in the task processing module 2, N data processing units are included, and each slice subtask The tasks and corresponding data blocks to be processed are transmitted to different data processing units for RAID encoding and/or RAID decoding, and each data processing unit performs processing to obtain a processing result and then outputs it to the data output unit.

The data processing method based on the RAID chip provided by the embodiment of the present application is applied to the RAID chip, including: after obtaining the task request for data processing, parsing the task request to obtain the corresponding data-related information to be processed, and according to the data-to-be-processed related information The information retrieves the data blocks to be processed, and then divides the total task corresponding to the task request into multiple slice subtasks according to the preset segmentation rules, so that different data processing units can process different slice subtasks and corresponding pending tasks. Process data blocks for RAID encoding and/or RAID decoding. It can be seen that the technical solution provided by this application implements RAID encoding and/or RAID decoding based on the RAID chip, that is, implements data storage and/or data recovery based on the RAID chip, reduces hardware cache consumption, and saves CPU resources. At the same time, the total task is divided into multiple sliced subtasks, that is, a large task is divided into multiple small tasks, which are processed by different data processing units. Therefore, in terms of task scheduling, sliced subtasks are beneficial With fine-grained management and control, it is easy to locate and troubleshoot faults when an error occurs in any slice subtask. At the data processing level, multiple data processing units simultaneously process each slice subtask to improve the efficiency of data processing, thereby reducing the area of the hardware system, saving CPU resources, and improving the data processing efficiency of RAID.

Different types of RAID require different numbers of hardware computing engines, and the calculation methods are also different. Therefore, when performing RAID encoding and/or RAID decoding, each data processing unit needs to first determine the RAID characteristics for storing the data blocks to be processed, and then adjust Use the corresponding RAID algorithm to perform RAID encoding and/or RAID decoding.

其中，*表示伽罗华域乘法，

表示异或操作，K1、K2、K3…Kn为伽罗华乘法运算参数。Table 1 is a schematic diagram of stripe distribution corresponding to a RAID5 algorithm provided by the embodiment of the present application. As shown in Table 1, D1, D2...Dn are data blocks from different disks, P is a parity block, and the data block and parity The verification block satisfies the following operational relationship:

Among them, * means Galois field multiplication,

Indicates an XOR operation, and K1, K2, K3...Kn are Galois multiplication parameters.

RAID5解码公式为：

其中，x取值范围为大于0的自然数，Dx为丢失数据块，Kx为丢失数据块对应的伽罗华乘法运算参数，在

中不包括Dx和Dx对应的Kx项，例如，若x取1时，不包含K1*D1。由此，RAID5任意一个数据块丢失，均可通过其他数据块进行恢复。Then the RAID5 encoding formula of the RAID5 algorithm is:

The RAID5 decoding formula is:

Among them, the value range of x is a natural number greater than 0, Dx is the missing data block, Kx is the Galois multiplication operation parameter corresponding to the missing data block, in

Dx and the Kx item corresponding to Dx are not included in , for example, if x is 1, K1*D1 is not included. Therefore, if any data block in RAID5 is lost, it can be recovered through other data blocks.

Table 1 Stripe distribution corresponding to RAID5 algorithm

根据解二元一次方程组原理，RAID6最多可以恢复两块磁盘缺失的数据。Table 2 is a schematic diagram of stripe distribution corresponding to a RAID6 algorithm provided by the embodiment of the present application. As shown in Table 2, D1, D2...Dn are data blocks, and P and Q are check blocks, and satisfy the following operational relationship:

According to the principle of solving binary linear equations, RAID6 can restore the missing data of two disks at most.

Table 2 Stripe distribution corresponding to the RAID6 algorithm

根据解三元一次方程组原理，RAID TP最多可以恢复三块磁盘缺失的数据。Table 3 is a schematic diagram of the stripe distribution corresponding to a RAID TP algorithm provided by the embodiment of the present application. As shown in Table 3, D1, D2...Dn are data blocks, P, Q, and R are check blocks, and satisfy the following Operation relationship:

According to the principle of solving linear equations in three variables, RAID TP can restore the missing data of up to three disks.

Table 3 Stripe distribution corresponding to RAID TP algorithm

Different RAID algorithms need to calculate different numbers of check blocks, so different RAID algorithms require different hardware circuits for encoding or decoding. In a specific implementation, writing data to disk is divided into different modes. Mode 1: Write with full stripes. When there is no data in the disk, when writing all the data in the stripe to the disk, the verification information of the data block is calculated using the corresponding RAID algorithm, and then the data is directly transferred based on the verification information. Just write to disk. Mode 2: read, modify and write, read out the old data block that needs to be modified and the corresponding check block, and combine it with the new data block to be written to calculate the new check block corresponding to the newly written data block, and according to the new Parity blocks write new data blocks to disk. Mode 3: Reconstructive writing, read out the data blocks that do not need to be modified on the stripe and combine them with the new data blocks to be written for settlement to obtain a new check block, and then write the new data block to the disk according to the new check block .

Table 4 is a schematic diagram of stripe distribution corresponding to another RAID5 algorithm provided by the embodiment of the present application. As shown in Table 4, D1, D2...D7 are data blocks, and P is a check block. If necessary, data blocks D2 and When D3 is updated to data blocks D2' and D3', it is necessary to consider whether the unmodified data blocks D1, D4, D5, D6, and D7 will be dropped during the process of writing data blocks D2' and D3'. Therefore , in order to avoid disk loss of unmodified data, it is necessary to introduce a partial parity log (PartialParity Log, referred to as PPL), PPL is the checksum of unmodified data blocks, and in Table 4

Table 4 Stripe distribution corresponding to the RAID5 algorithm

D1 D2 D3 D4 D5 D6 D7 P

It can be seen that the hardware engine circuits for calculating and verifying different write modes are different, and when writing data blocks to disk, modes 2 and 3 also need to calculate PPL. Among them, the data blocks of all disks in mode one will be modified, so all new data blocks are used for verification, then

根据异或运算原理可得：

原数据块D2和D3被新数据块D2′和D3′替换后新的校验P′为：

且

Mode 2 is suitable for scenarios where there are fewer disk data blocks to be modified (for example, when the disk data blocks to be modified are less than half of the total number of stripes), for example, as shown in Table 4, update data blocks D2 and D3 to data When block D2' and D3', the original strip satisfies:

According to the principle of XOR operation, it can be obtained:

After the original data blocks D2 and D3 are replaced by new data blocks D2' and D3', the new check P' is:

and

且

由于P′为新的校验算法，则模式三下

Mode 3 is used when there are many disk data blocks to be modified (for example, when the disk data blocks to be modified exceed half of the total number of stripes), for example, in n disks, data blocks D2 and D3 will be modified, then The new calibration P' calculation formula is

and

Since P' is a new verification algorithm, the mode is three

To sum up, when performing RAID encoding and RAID decoding, it is necessary to call the corresponding RAID algorithm according to the nature of the RAID. In addition, it is also necessary to obtain information related to the data to be processed according to the task request. The information related to the data to be processed includes at least: data The number of blocks, the number of check blocks, the number of slicing subtasks, and the address information of data blocks, and then call the corresponding RAID algorithm based on the relevant information of the data to be processed to perform RAID encoding and RAID decoding.

The data processing method based on the RAID chip provided by the embodiment of the present application is applied to the RAID chip to analyze the task request to determine the number of data blocks, the number of check blocks, the number of slice subtasks, and the data block address information waiting to be processed. Information, and call the corresponding RAID algorithm based on the data-related information to be processed to perform RAID encoding and RAID decoding, saving CPU resources and improving the reliability of RAID data processing.

On the basis of the above-mentioned embodiments, after the data processing unit obtains the relevant information of the data to be processed, when performing RAID encoding on the data, it first needs to call the corresponding RAID algorithm, and use the corresponding algorithm according to the number of data blocks and slice subtasks The number of check blocks is calculated, and the check block corresponding to the data block to be processed is generated, and then the data block to be processed and the corresponding check block are stored in the corresponding disk. For ease of understanding, an example will be given below.

FIG. 3 is a schematic diagram of a RAID chip-based RAID encoding provided by the embodiment of the present application, for example, 8+6 encoding, that is, each slice subtask contains 8 data blocks, and each slice subtask generates 6 data blocks. Assuming that the total task is 8MByte and each data block is 4Kbyte, as shown in Figure 3, after the task scheduling unit receives the task request, it parses the task request and transmits it to the task slice processing unit to divide the total task There are 256 slice subtasks, and each slice subtask includes 8 data blocks, and each data block is 4Kbyte. After segmentation, the data block to be processed is transmitted to the data input unit for calculation. Generate 6 check blocks and transmit them to the data output unit, and then transmit them to the response processing unit to realize RAID encoding.

When the data processing unit performs RAID decoding, it also needs to call the corresponding RAID algorithm first, and then use the RAID algorithm to determine the target data block of the missing disk according to the data block to be processed and the check block corresponding to the data block to be processed, and then perform The block is recovered, thereby completing the recovery of the damaged data. For ease of understanding, an example will be given below.

FIG. 4 is a schematic diagram of RAID decoding based on a RAID chip provided by the embodiment of the present application. As shown in FIG. 4, for example, 6+3 decoding, that is, each slice subtask includes 6 data blocks and 3 parity Check blocks, assuming that there are 3 data blocks to be processed, that is to say, 3 data blocks to be restored, if the total task is 192Kbyte, each data block is 4Kbyte. As shown in Figure 3, if the data blocks to be restored are data block 2, data block 4, and data block 6, after the task scheduling unit receives the task request, it parses the task request and transmits it to the task slice processing unit, and cuts the total task Divided into 8 slicing subtasks, and each slicing subtask includes 3 data blocks and 3 check blocks, each data block and check block are 4Kbyte, after splitting, the pending data block is transferred to the data input The unit performs calculations, thereby generating data block 2, data block 4, and data block 6, which are transmitted to the data output unit, and then transmitted to the response processing unit by the data output unit, thereby realizing RAID decoding.

The RAID chip-based data processing method provided in the embodiment of the present application performs RAID encoding and RAID decoding based on RAID, saving CPU resources and hardware area. Divide the total task into multiple slice subtasks, and perform calculations by different data processing units, while improving data processing efficiency, if a fault occurs during data processing, it can be quickly located and troubleshooted, thereby improving the reliability of RAID .

In a specific embodiment, as shown in FIG. 2 , the data processing unit determines the address of the data block to be processed according to the data block address information in the data related data to be processed after obtaining the data related to the data to be processed after the information processing module 1 has parsed it. The start address, and according to the start address and the number of data blocks to call the data blocks to be processed, and calculate based on the called data blocks to be processed.

It should be noted that, after obtaining the task request, the task scheduling unit in the information processing module 1 first checks the task request before parsing the task request to check whether the task request meets the preset conditions, such as whether the format of the task request is correct , the present application does not limit the preset conditions. If the task request does not meet the preset condition, the data processing is ended, and the abnormal signal is transmitted to the response processing unit, so as to remind the user to process.

Further, if the task request satisfies the preset condition, the task request is parsed to obtain the related information of the data to be processed, and after the data block to be processed is retrieved according to the related information of the data to be processed, the address format checking unit performs the processing according to the address information of the data block Information verification is performed on each data block to be processed, and the verification result is stored, thereby avoiding system confusion when the verification result is transmitted to the lower-level processing unit.

In the data processing method based on the RAID chip provided by the embodiment of the present application, the processing units such as the task processing unit and the address format checking unit in the information processing module 1 check and check the information, thereby ensuring the reliability of the entire data processing process . In addition, the data input unit in the task processing module 2 retrieves the data block to be processed according to the address information of the data block for data calculation, thereby ensuring accurate and fast retrieval of data and further improving data processing efficiency.

In the foregoing embodiments, the data processing method based on the RAID chip is described in detail, and the present application also provides an embodiment corresponding to a data processing device based on the RAID chip. It should be noted that this application describes the embodiments of the device part from two perspectives, one is based on the perspective of functional modules, and the other is based on the perspective of hardware structure.

Fig. 5 is a structural diagram of a data processing device based on a RAID chip provided by an embodiment of the present application. As shown in Fig. 5, the device includes:

The parsing module 10 is configured to parse the task request to obtain corresponding data-related information to be processed after the task request is obtained;

Calling module 11, for calling the data block to be processed according to the related information of the data to be processed;

The segmentation module 12 is used to segment the total task corresponding to the task request into a plurality of sliced subtasks according to preset segmentation rules;

The transmission module 13 is configured to transmit each slice subtask and corresponding data blocks to be processed to different data processing units for RAID encoding and/or RAID decoding.

Since the embodiment of the device part corresponds to the embodiment of the method part, please refer to the description of the embodiment of the method part for the embodiment of the device part, and details will not be repeated here.

The data processing device based on the RAID chip provided by the embodiment of the present application is applied to the RAID chip, including: after obtaining the task request for data processing, parsing the task request to obtain the corresponding data-related information to be processed, and according to the data-to-be-processed related information The information retrieves the data blocks to be processed, and then divides the total task corresponding to the task request into multiple slice subtasks according to the preset segmentation rules, so that different data processing units can process different slice subtasks and corresponding pending tasks. Process data blocks for RAID encoding and/or RAID decoding. It can be seen that the technical solution provided by this application implements RAID encoding and/or RAID decoding based on the RAID chip, that is, implements data storage and/or data recovery based on the RAID chip, reduces hardware cache consumption, and saves CPU resources. At the same time, the total task is divided into multiple sliced subtasks, that is, a large task is divided into multiple small tasks, which are processed by different data processing units. Therefore, in terms of task scheduling, sliced subtasks are beneficial With fine-grained management and control, it is easy to locate and troubleshoot faults when an error occurs in any slice subtask. At the data processing level, multiple data processing units simultaneously process each slice subtask to improve the efficiency of data processing, thereby reducing the area of the hardware system, saving CPU resources, and improving the data processing efficiency of RAID.

FIG. 6 is a structural diagram of a data processing device based on a RAID chip provided by another embodiment of the present application. As shown in FIG. 6, the data processing device based on a RAID chip includes: a memory 20 for storing computer programs;

The processor 21 is configured to implement the steps of the data processing method based on the RAID chip as mentioned in the above embodiment when executing the computer program.

The RAID chip-based data processing device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer or a desktop computer, and the like.

Wherein, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 may be at least one of a Digital Signal Processor (DSP for short), a Field-Programmable Gate Array (FPGA for short), and a Programmable Logic Array (PLA for short). implemented in the form of hardware. Processor 21 may also include a main processor and a coprocessor, the main processor is a processor for processing data in a wake-up state, and is also called a central processing unit (Central Processing Unit, referred to as CPU); It is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processor (Graphics Processing Unit, GPU for short), and the GPU is used for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI for short) processor, and the AI processor is used to process computing operations related to machine learning.

Memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory, and non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can realize the relevant steps of the data processing method based on the RAID chip disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Wherein, the operating system 202 may include Windows, Unix, Linux and so on. The data 203 may include but not limited to relevant data involved in the RAID chip-based data processing method and the like.

In some embodiments, the data processing device based on the RAID chip may further include a display screen 22 , an input/output interface 23 , a communication interface 24 , a power supply 25 and a communication bus 26 .

Those skilled in the art can understand that the structure shown in FIG. 6 does not limit the data processing device based on the RAID chip, and may include more or less components than those shown in the figure.

The data processing device based on the RAID chip provided by the embodiment of the present application includes a memory and a processor. When the processor executes a program stored in the memory, the following method can be implemented: a data processing method based on the RAID chip.

The data processing device based on the RAID chip provided by the embodiment of the present application implements data storage and/or data recovery based on the RAID chip, reduces hardware cache consumption, and saves CPU resources. At the same time, the total task is divided into multiple sliced subtasks, that is, a large task is divided into multiple small tasks, which are processed by different data processing units. Therefore, in terms of task scheduling, sliced subtasks are beneficial With fine-grained management and control, it is easy to locate and troubleshoot faults when an error occurs in any slice subtask. At the data processing level, multiple data processing units simultaneously process each slice subtask to improve the efficiency of data processing, thereby reducing the area of the hardware system, saving CPU resources, and improving the data processing efficiency of RAID.

Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. A computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps described in the foregoing method embodiments are implemented.

It can be understood that if the methods in the above embodiments are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , executing all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM for short), random access memory (Random Access Memory, RAM for short), magnetic disk or optical disk, etc., which can store program codes. medium.

A data processing method, device and medium based on a RAID chip provided by the present application has been introduced in detail above. Each embodiment in the description is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part. It should be pointed out that those skilled in the art can make some improvements and modifications to the application without departing from the principles of the application, and these improvements and modifications also fall within the protection scope of the claims of the application.

It should also be noted that in this specification, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or order between the operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Claims (10)

1. A data processing method based on RAID chips, applied to RAID chips, is characterized in that, comprising: After obtaining the task request, parsing the task request to obtain corresponding data-related information to be processed; Retrieving data blocks to be processed according to the relevant information of the data to be processed; Segmenting the total task corresponding to the task request into a plurality of sliced subtasks according to a preset segmentation rule; Each of the slicing subtasks and corresponding data blocks to be processed are transmitted to different data processing units for RAID encoding and/or RAID decoding. 2. the data processing method based on RAID chip according to claim 1, is characterized in that, described each described slicing subtask and corresponding data block to be processed are transmitted to different data processing units to carry out RAID coding and/or or RAID decoding includes: According to the characteristics of the RAID storing the data block to be processed, a corresponding RAID algorithm is invoked to perform RAID encoding and/or RAID decoding. 3. The data processing method based on the RAID chip according to claim 1, wherein the data related information to be processed at least includes: the number of data blocks, the number of check blocks, the number of slice subtasks and the number of data blocks Address information. 4. the data processing method based on RAID chip according to claim 3, is characterized in that, described data processing unit carries out RAID encoding and/or RAID decoding and comprises: When the data processing unit performs RAID encoding, it includes: calling the corresponding RAID algorithm so as to calculate the number of check blocks according to the number of data blocks and the number of slice subtasks; and generate a check block corresponding to the data block to be processed; storing the data block to be processed and the corresponding check block in the corresponding disk; When the data processing unit performs RAID decoding, it includes: calling the corresponding RAID algorithm so as to determine the target data block of the missing disk according to the data block to be processed and the check block corresponding to the data block to be processed; Data in the target data block is restored. 5. the data processing method based on RAID chip according to claim 3, is characterized in that, described according to described data-to-be-processed related information transfer pending data block comprises: Determine the start address of the data block to be processed according to the data block address information; The data blocks to be processed are retrieved according to the start address and the number of data blocks. 6. the data processing method based on the RAID chip according to claim 4, is characterized in that, after described according to the relevant information of the data to be processed to transfer the data block to be processed, further comprising: Perform information check on each of the data blocks to be processed according to the address information of the data block, and store the check result. 7. the data processing method based on RAID chip according to claim 1, is characterized in that, before described parsing described task request obtains the corresponding pending data related information of described task unit, also comprises: Check whether the task request satisfies the preset condition, if not, end the data processing, and transmit the abnormal signal to the response processing unit. 8. A data processing device based on RAID chips, applied to RAID chips, is characterized in that, comprising: A parsing module, configured to parse the task request to obtain corresponding data related information to be processed after the task request is obtained; A call module, configured to call the data block to be processed according to the relevant information of the data to be processed; A segmentation module, configured to segment the total task corresponding to the task request into a plurality of sliced subtasks according to preset segmentation rules; The transmission module is configured to transmit each of the slicing subtasks and corresponding data blocks to be processed to different data processing units for RAID encoding and/or RAID decoding. 9. A data processing device based on a RAID chip, characterized in that it includes a memory for storing computer programs; The processor is configured to implement the steps of the RAID chip-based data processing method according to any one of claims 1 to 7 when executing the computer program. 10. A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method based on any one of claims 1 to 7 is implemented. The steps of the data processing method of the RAID chip.

Images