CN115793984B - Data storage method, device, computer equipment and storage medium - Google Patents

Abstract

The invention discloses a data storage method, a device, computer equipment and a storage medium, wherein the method comprises the following steps: checking a storage system data tag on a periodic basis, wherein the storage system data tag includes cold data and hot data, the storage system configured to store the data tagged as cold data based on a first encoding and store the data tagged as hot data based on a second encoding; and responding to the detected data mark triggering mark switching condition, switching the detected data mark, and correspondingly switching the coding mode to store corresponding data in a mode based on the switched coding mode. By the scheme of the invention, the overall degradation reading delay and the overall storage overhead of the storage system are reduced.

Description

A data storage method, device, computer equipment and storage medium

technical field

The present invention relates to the technical field of storage, in particular to a data storage method, device, computer equipment and storage medium.

Background technique

With the rapid development of communication technology and network technology, digital information is growing exponentially, and data storage technology is therefore facing huge challenges. The reliability of data in the storage system and the energy consumption of the storage system are more and more concerned by people. Nowadays, in the face of such a huge data scale, the reliability of the data in the storage system is inversely proportional to the number of components contained in the storage system, that is, the more components in the storage system, the lower the reliability of the data in the storage system. According to relevant surveys, in an Internet data center composed of 600 disks, about 30 disks will be damaged every month. In a large-scale storage system, the decline in data reliability caused by disk failure is quite serious. For this problem, people have launched research on related fault-tolerant technologies.

Erasure code (Erasure Code) belongs to a kind of forward error correction technology in coding theory, which was first applied in the field of communication to solve problems such as loss and loss in data transmission. Since the erasure code technology has achieved good results in preventing data loss, it has been introduced into the storage field. Erasure coding can effectively reduce storage overhead while ensuring the same reliability, so erasure coding technology is widely used in major storage systems and data centers.

Erasure coding (EC) is a data protection method that divides data into fragments, expands and encodes redundant data, and stores it in different locations, such as disks, storage nodes, or other geographic locations. Divide the original data into k data blocks, generate m coding blocks according to the coding matrix, and distribute n (n=k+m) blocks to different servers. Only k blocks are needed to restore the original data.

Most storage systems using erasure codes store data using only one erasure code. However, it is difficult to reduce degraded read latency while maintaining low storage overhead by using only one erasure code.

Contents of the invention

In view of this, the present invention proposes a data storage method, device, computer equipment, and storage medium, which uses multiple encoding methods to store data, and selects an appropriate encoding according to the characteristics of the data to reduce degradation reading while maintaining low storage overhead. Delay.

Based on the above purpose, an aspect of the embodiments of the present invention provides a data storage method, which specifically includes the following steps:

The storage system data mark is checked periodically, wherein the storage system data mark includes cold data and hot data, and the storage system is configured to store the data marked as cold data based on the first code, and to store the data marked as cold data based on the second encoding and storing the data marked as hot data;

In response to the detected data mark triggering the mark switching condition, the detected data mark is switched, and the encoding mode is switched correspondingly to store corresponding data based on the switched encoding mode.

In some embodiments, the first encoding comprises RS encoding and the second encoding comprises LRC encoding.

In some embodiments, the RS code is (k _RS , g _RS ) RS code, and the LRC code is (k _LRC , l _LRC , g _LRC ) LRC code, where k _RS represents the data in the RS code The number of blocks and g _RS represent the number of global parity blocks in the RS code, k _LRC represents the number of data blocks in the LRC code, l _LRC represents the number of local parity blocks in the LRC code, and g _LRC represents all The number of global check blocks in the LRC encoding, where k _RS =k _LRC , g _RS =g _LRC +1.

In some embodiments, in response to the checked data flag triggering a flag switching condition, switching the checked data flag and correspondingly switching the encoding method includes:

In response to the detected data being marked as hot data, determine whether the number of accesses to the data reaches a first threshold;

In response to the number of access times of the data not reaching the first threshold, the data tag is switched from the hot data to the cold data, and the encoding method of the data is switched from the second encoding storage to the first encoding .

In some embodiments, in response to the checked data flag triggering a flag switching condition, switching the checked data flag and correspondingly switching the encoding method includes:

In response to the checked data being marked as cold data, it is determined whether the number of accesses to the data reaches a second threshold;

In response to the number of accesses to the data reaching the second threshold, the data tag is switched from the cold data to the hot data, and the encoding method of the data is switched from the first encoding to the second encoding.

In some embodiments, switching the encoding method of the data from the second encoding storage to the first encoding includes:

The encoding method of the data is switched from the second encoding storage to the first encoding based on the first encoding switching algorithm.

In some implementation manners, switching the encoding method of the data from the first encoding to the second encoding includes:

The encoding mode of the data is switched from the first encoding to the second encoding based on the second encoding switching algorithm.

In some implementations, switching the encoding method of the data from the second encoding storage to the first encoding based on the first encoding switching algorithm includes:

Obtaining the RS-encoded data block based on the LRC-encoded data block;

The RS-coded global check block is obtained based on the LRC-coded global check block and the local check block.

In some embodiments, obtaining the RS-coded data block based on the LRC-coded data block includes:

Acquire the LRC encoded data block, and use the acquired data block as the RS encoded data block.

In some implementation manners, obtaining the RS-coded global check block based on the LRC-coded global check block and the local check block includes:

Using the g _LRC global check blocks of the LRC code as the g _RS -1 global check blocks of the RS code;

Perform XOR calculation on the 1 _LRC local check blocks of the LRC code, and use the XOR calculation result as the last global check block of the RS code.

In some implementation manners, switching the encoding mode of the data from the first encoding to the second encoding based on the second encoding switching algorithm includes:

obtaining the LRC-coded data block based on the RS-coded data block;

Obtaining the LRC-coded global check block based on the RS-coded global check block;

The LRC-coded local check block is obtained based on the RS-coded data block and the last global check block.

In some embodiments, obtaining the LRC-coded data block based on the RS-coded data block includes:

Obtain the RS-encoded data block, and use the obtained data block as the LRC-encoded data block.

In some embodiments, obtaining the LRC-coded global check block based on the RS-coded global check block includes:

The g _RS -1 global check blocks encoded by the RS are used as the g _LRC global check blocks encoded by the LRC.

In some embodiments, obtaining the local check block of the LRC code based on the RS coded data block and the last global check block includes:

Obtain the last l _LRC -1 local check blocks of the LRC encoding based on the RS encoded data block, and combine the last global check block of the RS encoding with the last l _LRC -1 of the LRC encoding The data block corresponding to the partial check block is XORed to obtain the first partial check block.

In some implementation manners, obtaining the last 1 _LRC -1 local check blocks of the LRC code based on the RS coded data block includes:

The last l _LRC -1 local check blocks of the LRC encoding are obtained based on the last k _RS /l _LRC *(l _LRC -1) data blocks of the RS encoding.

In some implementation manners, the last global parity block of the RS code is obtained by XOR calculation of k _RS data blocks.

Another aspect of the embodiments of the present invention also provides a data storage device, including:

A check module, the check module is configured to periodically check the storage system data tags, wherein the storage system data tags include cold data and hot data, and the storage system is configured to store the tags based on the first code data that is cold data, and storing said data marked as hot data based on a second encoding;

A switching module configured to trigger a flag switching condition in response to the detected data flag, switch the detected data flag, and switch the coding mode accordingly to store corresponding data based on the switched coding mode.

In some embodiments, the first encoding comprises RS encoding and the second encoding comprises LRC encoding.

In still another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory, the memory stores a computer program that can run on the processor, and the computer program is executed by the The steps of the above method are realized when the processor executes.

In yet another aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, and the computer-readable storage medium stores a computer program for implementing the above method steps when executed by a processor.

The present invention has at least the following beneficial technical effects: by periodically checking the data marks of the storage system, wherein the data marks of the storage system include cold data and hot data, the storage system is configured to store the data marked as cold data based on the first code data, and store the data marked as hot data based on the second code; in response to the detected data mark triggering the mark switching condition, the detected data mark is switched, and the encoding mode is switched correspondingly based on the switched encoding mode The solution for storing corresponding data reduces the overall degraded read latency and storage overhead of the storage system.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and those skilled in the art can obtain other embodiments according to these drawings without any creative effort.

Fig. 1 is the block diagram of an embodiment of the data storage method provided by the present invention;

FIG. 2 is a schematic diagram of RS coding based on Vandermonde matrix provided by the present invention;

Fig. 3 is a schematic diagram of RS coding based on Cauchy matrix;

Fig. 4 is a schematic diagram of RS code recovery data;

FIG. 5 is a schematic diagram of an embodiment of (6,3) RS coding provided by the present invention;

FIG. 6 is a block diagram of an embodiment of (6, 3, 2) LRC encoding provided by the present invention;

Fig. 7 is a schematic diagram of an embodiment of switching from (6, 3, 2) LRC coding to (6, 3) RS coding provided by the present invention;

FIG. 8 is a schematic diagram of an embodiment of switching from (6,3) RS coding to (6,3,2) LRC coding provided by the present invention;

FIG. 9 is a schematic structural diagram of an embodiment of a data storage device provided by the present invention;

FIG. 10 is a schematic structural diagram of an embodiment of a computer device provided by the present invention;

FIG. 11 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are to distinguish two entities with the same name but different parameters or parameters that are not the same, see "first" and "second" It is only for the convenience of expression, and should not be construed as a limitation on the embodiments of the present invention, which will not be described one by one in the subsequent embodiments.

Based on the above objective, the first aspect of the embodiments of the present invention proposes an embodiment of a data storage method. As shown in Figure 1, it includes the following steps:

S10. Periodically check the storage system data mark, wherein the storage system data mark includes cold data and hot data, and the storage system is configured to store the data marked as cold data based on the first code, and based on the second encoding stores the data marked as hot data;

S20. In response to the checked data label triggering a label switching condition, switch the checked data label, and switch the coding mode accordingly to store corresponding data based on the switched coding mode.

Specifically, the data stored in the storage system is checked periodically, and the data includes cold data and hot data. When the number of visits of a hot data does not reach a set value within a certain period of time, the hot data will be marked as cold data; and when the number of visits of a cold data reaches the set value within a certain period of time, the cold data will be marked is marked as hot data. The switching of hot and cold data marks also involves the switching of encoding. In this embodiment, cold data is stored based on the first encoding, and hot data is stored based on the second encoding. The first encoding and the second encoding are different types of erasure codes. The first encoding and the second encoding are selected based on the characteristics of the data. The access frequency of hot data is high, so the second encoding will use the erasure code with low recovery delay, and the access frequency of cold data is low, so the correction code with low storage overhead will be used. Code erasure reduces the overall degraded read latency and storage overhead of the storage system.

In a specific embodiment, when data first enters the storage system, it may be marked as hot data by default. Data marked as hot data is stored based on the second encoding with low degradation read latency, and data marked as cold data is stored based on the first encoding with low storage overhead. Periodically check the data stored in the storage system. When the number of visits of a hot data does not reach a set value within a certain period of time, the hot data will be marked as cold data; and when the number of visits of a cold data reaches the set value within a certain period of time, the cold data will be marked is marked as hot data. The switching of the hot and cold data tags simultaneously switches the codes corresponding to the hot and cold data tags, thereby reducing the overall degraded read latency and storage overhead of the storage system.

In some embodiments, the first encoding comprises RS encoding and the second encoding comprises LRC encoding.

Specifically, for hot data with high frequency of access, (k, l, r-1) LRC coding with low recovery delay is used; for cold data with low frequency of access, (k, r) RS coding is used, where , k represents the number of data blocks, l represents the number of local check blocks, and r represents the number of global check blocks. Based on this, regardless of hot or cold data, the storage system can tolerate the damage of r data blocks.

RS coding (Reed-solomon codes, a forward error correction channel coding), is related to two parameters k and r. Given two positive integers k and r, the RS code encodes k data blocks into r additional check blocks, and the r check blocks can be formed based on Vandermonde matrix or Cauchy matrix. As shown in Figure 2 and Figure 3, they are RS codes based on Vandermonde matrix and Cauchy matrix respectively. The k*k matrix in the upper part corresponds to k original data blocks, and the r*k matrix in the lower part corresponds to the encoding matrix. By multiplying the original data D ₁ to D _k , the newly added P ₁ to P _r It is the r verification data obtained by encoding. When any r pieces of data are wrong or lost during transmission and need to be corrected, the inverse matrix of the matrix corresponding to the remaining data is multiplied by the data to obtain the original data blocks D ₁ to D _k . Taking data loss from D ₁ to D _r and decoding as an example, the process is shown in Figure 4. (k, r) RS code is MDS (Maximum Distance Separayle, maximum distance separable code) code, when any r blocks fail, it can be recovered according to the remaining data blocks and check blocks.

LRC coding (Locxl Reconstruction Codes, a local erasure code), (k, l, r-1) LRC divides k data blocks into l local groups. It encodes l local parts, one local parity block per local group, and r global parity blocks. Any single block failure can be decoded from the k/l fragments within its local group. LRC encoding can tolerate up to r+1 arbitrary segment failures, and can also tolerate more than r (up to l+r-1) failures, provided the information is theoretically decodable. Finally, LRC provides lower storage overhead. Of all codes that can decode single block failures from k/l fragments and tolerate r failures, LRC requires the fewest parity blocks. At the same time, LRC reduces the reconstruction cost compared with RS codes. Therefore, LRC has a lower recovery delay.

Therefore, this embodiment reduces the overall degraded read delay and storage overhead of the storage system by using two different erasure codes for hybrid storage.

In a specific embodiment, because code switching also requires a very high data transmission volume, an efficient code switching algorithm is configured to reduce the high data transmission volume and calculation volume caused by code switching. In the embodiment of the present invention, according to the properties of LRC coding and RS coding, an efficient switching algorithm for ( k , l , g-1 ) LRC coding and ( k , g ) RS coding is proposed to ensure that the switching of hot and cold data will not become The bottleneck in the system reduces the amount of data transmission and calculation during encoding switching.

In some embodiments, the RS code is (k _RS , g _RS ) RS code, and the LRC code is (k _LRC , l _LRC , g _LRC ) LRC code, where k _RS represents the data in the RS code The number of blocks and g _RS represent the number of global parity blocks in the RS code, k _LRC represents the number of data blocks in the LRC code, l _LRC represents the number of local parity blocks in the LRC code, and g _LRC represents all The number of global check blocks in the LRC encoding, where k _RS =k _LRC , g _RS =g _LRC +1.

Specifically, in order to ensure efficient switching between RS coding and LRC coding, RS coding and LRC coding must meet the following conditions:

1) k _RS = k _LRC and g _RS = g _LRC +1;

2) The last one of the g _RS global parity blocks of RS is obtained by XOR of all k _RS data blocks;

3) The g _LRC global parity blocks of the LRC are consistent with the g _RS -1 global parity blocks of the RS.

In some embodiments, in response to the checked data flag triggering a flag switching condition, switching the checked data flag and correspondingly switching the encoding method includes:

In response to the detected data being marked as hot data, determine whether the number of accesses to the data reaches a first threshold;

In response to the number of access times of the data not reaching the first threshold, the data tag is switched from the hot data to the cold data, and the encoding method of the data is switched from the second encoding storage to the first encoding .

Specifically, in order to reduce the overall degraded read latency and storage overhead of the storage system, when the data tag is switched from hot data to cold data, the encoding method for storing the data is switched accordingly.

In some embodiments, in response to the checked data flag triggering a flag switching condition, switching the checked data flag and correspondingly switching the encoding method includes:

In response to the checked data being marked as cold data, it is determined whether the number of accesses to the data reaches a second threshold;

In response to the number of accesses to the data reaching the second threshold, the data tag is switched from the cold data to the hot data, and the encoding method of the data is switched from the first encoding to the second encoding.

Specifically, in order to reduce the overall degraded read latency and storage overhead of the storage system, when the data tag is switched from cold data to hot data, the encoding method for storing the data is switched accordingly.

In some embodiments, switching the encoding method of the data from the second encoding storage to the first encoding includes:

The encoding method of the data is switched from the second encoding storage to the first encoding based on the first encoding switching algorithm.

In some implementation manners, switching the encoding method of the data from the first encoding to the second encoding includes:

The encoding mode of the data is switched from the first encoding to the second encoding based on the second encoding switching algorithm.

In some implementations, switching the encoding method of the data from the second encoding storage to the first encoding based on the first encoding switching algorithm includes:

Obtaining the RS-encoded data block based on the LRC-encoded data block;

The RS-coded global check block is obtained based on the LRC-coded global check block and the local check block.

In some embodiments, obtaining the RS-coded data block based on the LRC-coded data block includes:

Acquire the LRC encoded data block, and use the acquired data block as the RS encoded data block.

In some implementation manners, obtaining the RS-coded global check block based on the LRC-coded global check block and the local check block includes:

Using the g _LRC global check blocks of the LRC code as the g _RS -1 global check blocks of the RS code;

Perform XOR calculation on the 1 _LRC local check blocks of the LRC code, and use the XOR calculation result as the last global check block of the RS code.

In some implementation manners, switching the encoding mode of the data from the first encoding to the second encoding based on the second encoding switching algorithm includes:

obtaining the LRC-coded data block based on the RS-coded data block;

Obtaining the LRC-coded global check block based on the RS-coded global check block;

The LRC-coded local check block is obtained based on the RS-coded data block and the last global check block.

In some embodiments, obtaining the LRC-coded data block based on the RS-coded data block includes:

Obtain the RS-encoded data block, and use the obtained data block as the LRC-encoded data block.

In some embodiments, obtaining the LRC-coded global check block based on the RS-coded global check block includes:

The g _RS -1 global check blocks encoded by the RS are used as the g _LRC global check blocks encoded by the LRC.

In some embodiments, obtaining the local check block of the LRC code based on the RS coded data block and the last global check block includes:

Obtain the last l _LRC -1 local check blocks of the LRC encoding based on the RS encoded data block, and combine the last global check block of the RS encoding with the last l _LRC -1 of the LRC encoding The data block corresponding to the partial check block is XORed to obtain the first partial check block.

In some implementation manners, obtaining the last 1 _LRC -1 local check blocks of the LRC code based on the RS coded data block includes:

The last l _LRC -1 local check blocks of the LRC encoding are obtained based on the last k _RS /l _LRC *(l _LRC -1) data blocks of the RS encoding.

In some implementation manners, the last global parity block of the RS code is obtained by XOR calculation of k _RS data blocks.

In a specific embodiment, (k _LRC , l _LRC , g _LRC ) LRC encoding is used for hot data with high frequency of access; (k _RS , g _RS ) RS is used for cold data with low frequency of access Coding, where k _RS and k _LRC represent the number of data blocks, l _LRC represents the number of local check blocks, and g _RS and g _LRC represent the number of global check blocks.

It can be seen from Figure 5 and Figure 6 that when k _RS = k _LRC and g _RS = g _LRC +1 and the third global redundant block of RS coding is obtained by XOR of all k data blocks, the corresponding RS Encoding and LRC encoding can share the redundant block of the same RS code. Under this condition, switching between RS coding and LRC coding is quite efficient, and the actual switching algorithm will be introduced below. To avoid confusion, in this switching algorithm, record k = k _RS = k _LRC and g = g _RS = g _LRC +1, that is, the two codes used in the storage system are ( k , g ) RS code and ( k , l , g-1 ) LRC.

S1: For hot data with high frequency of access, use ( k , l , g-1 ) LRC coding; for cold data with low frequency of access, use ( k , g ) RS coding, and satisfy k _RS = k _LRC and g _RS = g _LRC +1 and RS encodes the last global redundant block f _g (x) , obtained by XORing all k data blocks.

S2: When the data label is converted from hot data to cold data, the encoding method is converted from LRC encoding to RS encoding, and skip to S3; when the data label is converted from cold data to hot data, the encoding method is switched from RS encoding to LRC encoding, Skip to S4.

S3: LRC code is switched to RS code, for ( k , g ) RS code and ( k , l , g-1 ) LRC , the parameter k _RS = k _LRC and g _RS = g _LRC +1. For the last global redundant block f _g (x) of the ( k , g ) RS code, it is necessary to XOR the l local check blocks of the ( k , l , g-1 ) LRC code, and then ( k , l , g-1 ) l local check blocks of LRC encoding can be deleted. Examples are as follows:

As shown in Figure 7, for the (6, 3, 2) LRC code, when it is necessary to switch to the (6, 3) RS code, the data of the second and third local parity blocks are transmitted to the first local For the check block, f ₃ (x) can be obtained by XOR, and then the second and third partial check blocks can be deleted.

S4: RS code is switched to LRC code, for ( k , g ) RS code and ( k , l , g-1 ) LRC code, parameter k = k _RS = k _LRC and g _RS = g _LRC +1, so ( k , g ) The first g-1 global check blocks of the RS code global check block remain unchanged; the last ( l-1 ) local check blocks are obtained by XORing the corresponding data blocks, and the first local check block The check block is obtained by XORing the last global check block f _g (x) and the next ( l-1 ) local check blocks from all corresponding data blocks. Examples are as follows:

As shown in Figure 8, when the (6,3) RS code is switched to the (6,3,2) LRC code, the first two global check blocks remain unchanged; the last two local check blocks are determined by the corresponding data block XOR is performed, and the first local check block is obtained by XORing the last global check block f ₃ (x) and the last two local check blocks from all corresponding data blocks.

Through the above scheme, when the system needs to convert hot data into cold data, it will switch ( k , l , g-1 ) LRC to ( k , g ) RS code. If the re-encoding algorithm is used, all data blocks need to be obtained before re-encoding, and new redundant blocks are sent to each node after encoding, so at least ( k + g -1) blocks need to be transmitted. If the switching algorithm of the embodiment of the present invention is adopted, it is only necessary to transmit the data of the last l-1 local check blocks encoded by ( k , l , g-1 ) to the first local check block. The data transmission amount of the switching algorithm in the embodiment is ( 1-1 ) blocks.

When the system needs to convert cold data into hot data, the ( k , g ) RS code will be switched to ( k , l , g-1 ) LRC code. If the re-encoding algorithm is used, at least ( k + l + g -1) blocks need to be transmitted. If the algorithm proposed in this paper is adopted, (( k / l ) * ( l-1 )) blocks need to be transmitted. When k =6, l=3 and g =3, the handover algorithm of the embodiment of the present invention requires 4 data transmissions, and the re-encoding requires 11 data transmissions.

Combining the above two switching processes, it can be seen that the high-efficiency switching algorithm proposed by the present invention has obvious optimization compared with the re-encoding algorithm. It is worth noting that, generally speaking, since the data in the system generally increases continuously, and data access has time locality, there should be much more cases where hot data becomes cold data than cold data becomes hot data. Moreover, since the switching algorithm proposed by the present invention has an obvious optimization effect when the hot data becomes cold, the overall performance optimization is quite remarkable.

In terms of computational efficiency, the algorithm proposed in the present invention only needs to perform XOR (exclusive OR) operation, and the scope involved is relatively small compared with the entire set of data, which is also obvious compared to the Galois field operation that requires the entire set of data for the re-encoding algorithm optimization. Although computing time is not the bottleneck of encoding switching, the substantial increase in computing efficiency can effectively help the system save computing resources, thereby allocating more computing resources for upper-layer practical applications and further improving the overall system performance.

Based on the same inventive concept, according to another aspect of the present invention, as shown in FIG. 9 , an embodiment of the present invention also provides a data storage device, including:

A check module 110, the check module 110 is configured to periodically check the storage system data mark, wherein the storage system data mark includes cold data and hot data, and the storage system is configured to store the stored data based on the first code. the data marked as cold data, and store the data marked as hot data based on the second encoding;

The switching module 120, the switching module 120 is configured to switch the detected data mark in response to the detected data mark triggering mark switching condition, and correspondingly switch the encoding mode to store corresponding data based on the switched encoding mode.

In some embodiments, the first encoding comprises RS encoding and the second encoding comprises LRC encoding.

Based on the same inventive concept, according to another aspect of the present invention, as shown in FIG. 10 , the embodiment of the present invention also provides a computer device 30, which includes a processor 310 and a memory 320, and the memory 320 stores There is a computer program 321 that can run on the processor, and the processor 310 executes the steps of the above method when the program is executed.

Wherein, the memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the data storage method in the embodiment of the present application corresponds to program instructions/modules. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory, that is, implements the data storage method of the above method embodiment.

The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function; the data storage area may store data created according to usage of the device, and the like. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory may optionally include memory located remotely from the processor, and these remote memories may be connected to the local module via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Based on the same inventive concept, according to another aspect of the present invention, as shown in FIG. 11 , the embodiment of the present invention also provides a computer-readable storage medium 40. The computer-readable storage medium 40 stores the Computer program 410 as above method.

Finally, it should be noted that those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer programs, and the programs can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium of the program may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM). The foregoing computer program embodiments can achieve the same or similar effects as any of the foregoing method embodiments corresponding thereto.

Those of skill would also appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as software or as hardware depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the functions in various ways for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope disclosed in the embodiments of the present invention.

The above are the exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications can be made without departing from the scope of the disclosed embodiments of the present invention defined in the claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. The serial numbers of the embodiments disclosed in the above-mentioned embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in an individual form, they may also be understood as a plurality unless explicitly limited to a singular number.

It should be understood that as used herein, the singular form "a" and "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.

Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is exemplary only, and is not intended to imply that the disclosed scope (including claims) of the embodiments of the present invention is limited to these examples; under the idea of the embodiments of the present invention , the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in details for the sake of brevity. Therefore, within the spirit and principle of the embodiments of the present invention, any omissions, modifications, equivalent replacements, improvements, etc., shall be included in the protection scope of the embodiments of the present invention.

Claims (19)

1. A method of data storage, comprising:

checking a storage system data tag on a periodic basis, wherein the storage system data tag includes cold data and hot data, the storage system configured to store the data tagged as cold data based on a first encoding and store the data tagged as hot data based on a second encoding, wherein the first encoding is an RS encoding and the second encoding is an LRC encoding;

switching the mark of the first data in response to the detected mark of the first data being hot data and triggering a mark switching condition, and switching the coding mode of the first data after the mark switching from the second coding to the first coding to store the first data based on the switched coding mode, wherein the step of switching the coding mode of the first data after the mark switching from the second coding to the first coding comprises: l encoding the LRC _LRC Performing exclusive-or calculation on the local check blocks, and taking the exclusive-or calculation result as the last global check block of the RS code, wherein l is as follows _LRC Representing the number of partial check blocks in the LRC codeAmount of the components.

2. The method as recited in claim 1, further comprising: and switching the mark of the second data in response to the detected mark of the second data being cold data and triggering a mark switching condition, and switching the coding mode of the second data after the mark switching from the first coding to the second coding to store the second data based on the switched coding mode.

3. The method of claim 2, wherein the RS code is (k _RS ，g _RS ) RS code, the LRC code is (k _LRC ，l _LRC ，g _LRC ) LRC coding, where k _RS Representing the number of data blocks and g in the RS code _RS Representing the number, k, of global check blocks in the RS code _LRC Representing the number of data blocks in the LRC coding, g _LRC Representing the number of global parity chunks in the LRC encoding, where k _RS =k _LRC ，g _RS =g _LRC +1。

4. A method according to claim 3, wherein switching the flag of the first data and switching the coding mode of the first data after the flag switching from the second coding to the first coding in response to the checked flag of the first data being hot data and the flag switching condition being triggered comprises:

in response to the checked flag of the first data being hot data and a flag switching condition being triggered, determining whether the number of accesses of the first data reaches a first threshold;

and switching the mark of the first data from hot data to cold data in response to the access times of the first data not reaching the first threshold, and switching the coding mode of the first data after the mark switching from second coding storage to first coding.

5. A method according to claim 3, wherein switching the flag of the second data and switching the coding scheme of the second data after the flag switching from the first coding to the second coding in response to the checked flag of the second data being cold data and the flag switching condition being triggered comprises:

in response to the detected flag of the second data being cold data and a flag switching condition being triggered, determining whether the number of accesses of the second data reaches a second threshold;

and switching the mark of the second data from cold data to hot data in response to the access times of the second data reaching the second threshold, and switching the coding mode of the second data after the mark switching from the first coding to the second coding.

6. The method of claim 4, wherein switching the coding mode of the first data after the mark switching from the second coding to the first coding comprises:

and switching the coding mode of the first data after the mark switching from the second coding to the first coding based on a first coding switching algorithm.

7. The method of claim 5, wherein switching the coding mode of the second data after the mark switching from the first coding to the second coding comprises:

and switching the coding mode of the second data after the mark switching from the first coding to the second coding based on a second coding switching algorithm.

8. The method of claim 6, wherein switching the coding mode of the first data after the marker switching from the second coding to the first coding based on the first coding switching algorithm comprises:

obtaining the RS-encoded data block based on the LRC-encoded data block;

and obtaining the global check block of the RS code based on the global check block and the local check block of the LRC code.

9. The method of claim 8, wherein deriving the RS-encoded data block based on the LRC-encoded data block comprises:

and acquiring the LRC encoded data block, and taking the acquired data block as the RS encoded data block.

10. The method of claim 8, wherein deriving the RS-encoded global parity block based on the LRC encoded global parity block and a local parity block comprises:

g of the LRC code _LRC Global check block as g of the RS code _RS -1 global check block.

11. The method of claim 7, wherein switching the coding mode of the second data after the marker switching from the first coding to the second coding based on the second coding switching algorithm comprises:

obtaining the LRC encoded data block based on the RS encoded data block;

obtaining the LRC encoded global check block based on the RS encoded global check block;

and obtaining the local check block of the LRC code based on the data block of the RS code and the last global check block.

12. The method of claim 11, wherein deriving the LRC encoded data block based on the RS encoded data block comprises:

and acquiring the RS encoded data block, and taking the acquired data block as the LRC encoded data block.

13. The method of claim 11, wherein deriving the LRC encoded global parity block based on the RS encoded global parity block comprises:

g of the RS code _RS -1 global parity block as g of said LRC encoding _LRC Global parity chunks.

14. The method of claim 11, wherein obtaining the LRC encoded local check block based on the RS encoded data block and a last global check block comprises:

obtaining the post l of the LRC code based on the RS-encoded data block _LRC -1 local check block and associating the last global check block of the RS code with the post l of the LRC code _LRC And carrying out exclusive or on the data blocks corresponding to the 1 local check blocks to obtain the 1 st local check block.

15. The method of claim 14, wherein the LRC encoded post l is derived based on the RS encoded data block _LRC -the 1 partial check blocks comprise:

post k based on the RS code _RS /l _LRC *（l _LRC -1) data blocks to obtain the LRC encoded post l _LRC -1 local check block.

16. A method according to claim 3, wherein the last global parity block of the RS code consists of k _RS And performing exclusive OR calculation on the data blocks.

17. A data storage device, comprising:

a checking module configured to check a storage system data tag on a periodic basis, wherein the storage system data tag includes cold data and hot data, the storage system configured to store the data tagged as cold data based on a first encoding and store the data tagged as hot data based on a second encoding, wherein the first encoding is an RS encoding and the second encoding is an LRC encoding;

a switching module configured to switch the tag of the first data in response to the detected tag of the first data being hot data and triggering a tag switch condition, andthe step of switching the coding mode of the first data after the mark switching from the second coding to the first coding to store the first data based on the switched coding mode, wherein the step of switching the coding mode of the first data after the mark switching from the second coding to the first coding comprises the following steps: l encoding the LRC _LRC Performing exclusive-or calculation on the local check blocks, and taking the exclusive-or calculation result as the last global check block of the RS code, wherein l is as follows _LRC Representing the number of partial parity chunks in the LRC encoding.

18. A computer device, comprising:

at least one processor; and

a memory storing a computer program executable on the processor, wherein the processor performs the steps of the method of any one of claims 1 to 16 when the program is executed.

19. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor performs the steps of the method according to any one of claims 1 to 16.

Images