CN107329909B - A data management method and device - Google Patents

Abstract

The application discloses a data management method, which comprises the following steps: establishing a first mapping for metadata, allocating a first logic block address, and taking the first logic block address as first metadata; establishing a second mapping for the first metadata, allocating a second logical block address, saving the second logical block address at a set position, and taking the second logical block address as second metadata; and processing the second metadata as user data. By the method, the problem of complexity of using two sets of flows because the system treats the user data and the metadata differently is solved. In addition, due to the introduction of multi-level mapping, the data searching efficiency can be improved, and the data amount written and stored when the system is powered off is reduced. The application also discloses a data management device, which has the beneficial effects.

Description

A data management method and device

technical field

The present application relates to the field of data management, and in particular, to a data management method and device.

Background technique

Hard drives can be seen everywhere in life. It is one of the main storage media for computers and consists of one or more aluminum or glass discs. It is mainly divided into Solid State Drive (SSD), Hard Disk Drive (HDD), and Hybrid Hard Drive (HHD). The solid-state hard disk, also known as electronic hard disk or solid-state electronic disk, is a hard disk composed of a control unit and a solid-state storage unit. The solid-state storage unit is generally DRAM (Dynamic Random Access Memory, Chinese name: dynamic random access memory) or Flash (Chinese Name: flash memory) chip.

The data in the reading process of SSD is called user data (host data), and only this data is not enough, it needs to be managed with another data alignment, this kind of data is called meta data (meta data), common Metadata includes l2p table, trim bitmap (block bitmap) and super block info (super block package information).

At present, the management of user data and metadata is generally treated differently in the industry, that is, user data uses a set of special read, write and GC (Garbage Collection, Chinese name: garbage collection) schemes, while metadata uses another set Completely different read, write and GC schemes. This approach is extremely complicated in logic due to the use of two completely different solutions, and the development and testing are also quite difficult.

SUMMARY OF THE INVENTION

In view of this, the present application provides a data management method and device, which solves the complexity problem of using two sets of processes for the system to treat user data and metadata differently, and treats user data and metadata as the same data type for processing. method, using the same process for processes such as read and write and GC. The method is as follows:

The technical scheme provided by the present invention is as follows:

establishing a first mapping for the metadata, assigning a first logical block address, and using the first logical block address as the first metadata;

Create a second mapping for the first metadata, assign a second logical block address, save the second logical block address in a set location, and use the second logical block address as the second metadata ;

The second metadata is processed as user data.

Wherein, processing the second metadata as user data includes:

The second metadata is read out at power-on, and the first metadata is restored through the second metadata.

Wherein, processing the second metadata as user data includes:

The second metadata is flashed to the flash, and the second metadata is saved to the corresponding position of the flash.

Wherein, the metadata includes l2p table or trim bitmap or super block info.

Wherein, saving the second logical block address in the set position includes:

The second logical block address is sequentially stored after the first metadata.

The present application also provides an apparatus for data management, the apparatus comprising:

a first metadata module for establishing a first mapping for metadata, assigning a first logical block address, and using the first logical block address as the first metadata;

The second metadata module establishes a second mapping for the first metadata, allocates a second logical block address, saves the second logical block address in a set location, and stores the second logical block address address as second metadata;

A data management module, configured to process the second metadata as user data.

Wherein, the data management module includes:

A data recovery submodule, configured to read out the second metadata when powered on, and recover the first metadata through the second metadata.

Wherein, the data management module includes:

The data flashing submodule is used for flashing the second metadata to the flash, and saving the second metadata to the corresponding position of the flash.

The data storage module is specifically a module that stores the second logical block address after the first metadata in sequence.

In the present application, by introducing a secondary mapping composed of a first mapping and a second mapping, the system can process meta data and host data as the same data type, which solves the problem that the system in the prior art uses different processing host data and meta data. The complexity of the two sets of processes, and because of the secondary mapping, the search efficiency of host data is improved. Moreover, the present application only needs to save the multi-level mapping, that is, an index table, when the system is powered off, and does not need to save all the data, which reduces the amount of data saved when the system is powered off. The present application also provides a data management device, which has the above beneficial effects, which will not be repeated here.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a flowchart of a data management method provided by an embodiment of the present application;

2 is a schematic diagram of a first mapping relationship provided by an embodiment of the present application;

3 is a schematic diagram of a second mapping relationship provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a data management apparatus provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

Please refer to FIG. 1 , the first embodiment of the present application is described below:

S101: establish a first mapping for metadata, allocate a first logical block address, and use the first logical block address as the first metadata;

Among them, there are many types of metadata, such as l2p table, trim bitmap, super block info, etc. Of course, there are other types of metadata, which will not be repeated here.

The first mapping may have various forms, which may be tables, figures, texts, or other forms, and the application does not limit the mapping forms. The first map can be stored in the 16K space of the super block, and is read out by the Control Mananger module immediately after the initial power-on. It can also be stored in other setting positions, which is not limited in this application. The so-called first mapping refers to the correspondence between the logical block address and the physical block address of the metadata.

The first logical block address refers to the location of the metadata in the system. When reading data, it is necessary to search for the physical block address corresponding to the logical block address.

The first metadata generally refers to the first logical block address, and the first logical block address and metadata may also be collectively referred to as the first metadata.

S102: Establish a second mapping for the first metadata, assign a second logical block address, save the second logical block address in a set location, and use the second logical block address as the second logical block address metadata;

Similarly, the second mapping is similar to the first mapping, and may also have various forms, which will not be repeated here.

Another mapping, that is, a second mapping, is established for the first metadata obtained in S101, and a second logical block address is assigned to it. The so-called second mapping refers to the correspondence between the logical block address and the physical block address of the first metadata.

The logical block address may be stored after the second mapping, or may be sequentially stored after the first metadata, which is not limited in this application.

The second metadata and the first metadata here are both "metadata", but their contents are different. "Metadata" here means that it is not user data, and is different from user data. The first mapping is the mapping of the correspondence between the physical block addresses and the logical block addresses of the metadata, and the second mapping is the mapping of the correspondence between the physical block addresses and the logical block addresses of the first metadata.

S103: Process the second metadata as user data.

When the power is turned off, the second metadata is executed in the same write (host write) process as the user data, and when the power is restored, the second metadata is executed in the same read (host read) process as the user data. Wherein, the read process refers to reading the second metadata at power-on, and restoring the first metadata through the second metadata. GC (Gabage Collection) process, the GC process is to move the valid data in several blocks (blocks) to a new block, and then erase these blocks to generate new available blocks.

Based on the above technical solutions, the data management method provided by the embodiments of the present application establishes a secondary mapping for the first metadata, and treats the metadata as user data, which solves the problem of using two sets of processes to distinguish user data and metadata in the prior art The complex method greatly reduces the difficulty of development and testing.

Referring to FIG. 2 below, FIG. 2 is a mapping relationship diagram of a data management method provided by an embodiment of the present invention. On the basis of the above embodiment, the metadata is specifically an l2p table, and the mapping form is a mapping table. The storage location of user data in the system is identified by lba (Logical Block Address, English abbreviation LBA or lba, Chinese name: logical block address) for the system, and pba (Physics Block Address, English name for SSD) Abbreviation: PBA or pba, Chinese abbreviation: physical block address) to find its location in the cache (Chinese name: cache memory) or flash, so there needs to be a mapping table in the SSD to store the mapping from lba to pba, This is the l2p table.

The specific method is as follows:

S201: A Buffer (buffer) is established when the system is running to save the delta data.

The delta data here is generally the delta data of metadata, such as l2p delta data, or trimbitmap (block bitmap) or superblock info (super block package).

S202: When the buffer is full, start to flash the flash. When the flash is also saturated, flash a certain number of bases. The data generated during the flashing process is also the first-level mapping.

Delta data will be generated during the flashing process, and the corresponding data storage location can be known through the delta data. For example, the delta1 data generated when flashing is written, the corresponding memory can be found to be flash through the delta1 data, and the resulting mapping is a first-level mapping.

S203: The system allocates a new lba to the first-level mapping, and saves the new lba to the set position.

The system allocates an lba to the first-level mapping. The lba can be placed in the l2p table, or placed in the l2p table in order. Of course, it can also be placed in other locations, but it is not recommended. As shown in Figure 2, it is stored after the corresponding Delta data. In this way, the mapping of the lba-l2P table constitutes a new "l2p table", a new metadata, which is the secondary mapping.

S204: When the power is turned off, the new "l2p table" is written to the flash with the same writing process as the user data. On power-up, the new metadata is restored using the same read process as user data.

If the data corresponding to the primary and secondary mappings are called primary and secondary data respectively, then when the power is turned on, after reading the secondary data data, use the secondary data to restore the primary data corresponding to the primary mapping table, and then use the secondary data to restore the primary data corresponding to the primary mapping table. Level data restores base data.

In addition, because the secondary mapping is established, the system does not need to save all the data when the system is powered off, only the primary and secondary mappings can be saved, which reduces the amount of data saved. Moreover, the secondary mapping table can be placed in the 16k space of the super block and read out by the Control Mananger module immediately after the initial power-on.

Taking the 8TB disk capacity as an example, the capacity of each partition (partition) is 2TB, and OP (Over-provisioning, Chinese name: reserved space) is temporarily ignored for the convenience of calculation.

The logical space corresponding to each segment in Figure 2 is:

User data 2TB

Metadata 2GB

Delta data 2GB*1/3＝2/3GB

Level 1 mapping (2GB+2/3GB)*4/4KB=8/3MB

The corresponding LBA ranges are:

User LBA range: [0,512M-1]

Metadata LBA range: [512M, 512M+512K-1]

Delta data LBA range: [512M+512K, 512M+4/3*512K-1]

Level 1 mapping LBA range: [512M+4/3*512K, 512M+4/3*512K+2/3K-1]

Secondary mapping LBA range: greater than 512M+4/3*512K+2/3K

Referring to Fig. 3, when the first mapping and the second mapping have been established and saved as in the above-mentioned embodiment, more levels of mapping can be established according to the size of the data volume, and the third-level mapping can be established by using the l2p table of the second level. , use the third-level l2p table to establish the fourth-level mapping... The mappings at all levels are stored in order. Of course, the l2p table here is only one kind of metadata, and other metadata can also be used according to the actual situation. Each level of mapping is stored in order as shown in Figure 2. When powered on, use the data corresponding to the last level of mapping to restore the data of the previous level, restore level by level, and use multi-level indexing.

Since the method part of this embodiment corresponds to the above-mentioned embodiment, reference is made to the description of the above-mentioned embodiment for the method part of this embodiment, which will not be repeated here.

Please refer to FIG. 4 below, which is a schematic structural diagram of a data management device provided by the application; the device may include:

The first metadata module 101 is used for establishing a first mapping for metadata, assigning a first logical block address, and using the first logical block address as the first metadata;

The second metadata module 102 establishes a second mapping for the first metadata, allocates a second logical block address, saves the second logical block address in a set location, and stores the second logical block address block address as second metadata;

The data management module 103 is configured to process the second metadata as user data.

Based on the above embodiment, the data management module 103 may further include a data refresh submodule 201 and a data recovery submodule 202 .

The data flashing sub-module 201 is used for flashing the second metadata to the flash, and saving the second metadata to a corresponding position of the flash.

The data recovery sub-module 202 is configured to recover the first metadata through the second metadata.

That is, the corresponding workflow is:

S301: The first metadata module establishes a first mapping for metadata, allocates a first logical block address, and uses the first logical block address as the first metadata.

Usually, for an SSD with on-board DRAM (dynamic random access memory), it is only necessary to search for the first mapping in the DRAM, and after obtaining the physical block address, access the flash to obtain host data; for Sandforce (a control chip) ), it needs to check whether the mapping relationship is stored in RAM (random storage memory) first, then read the host data directly according to the mapping relationship, if not, you need to read the mapping relationship from the flash. Read hostdata. Metadata can be l2p tables, superblock info, trim bitmap, etc.

S302: The second metadata module establishes a second mapping for the first metadata, assigns a second logical block address, saves the second logical block address in a set location, and uses the second logical block address as the second metadata .

The process is similar to that in S301, but the set position may be stored in sequence after the first metadata, or may be stored in a certain position in the first metadata, which is not limited in this application. The data storage module is also responsible for saving the second map, for example, it can be stored in the 16k space of the super block, so that it can be read by the Control Manager module immediately after the initial power-on. Of course, it can also be stored in other locations.

S303: The data management module processes the second metadata in the same process as the user data.

The above process includes that the data refresh sub-module 201 of the data management module processes the second metadata in the same manner as the user data write operation when the power is turned off. The flashing process includes flashing the second metadata to the flash, and saving the second metadata to a corresponding position of the flash. When powered on, the data recovery sub-module 202 in the data management module recovers the first metadata through the second metadata, obtains the required user data location and obtains the user data through the first metadata. It can also include a GC unit for data recovery processing. If there is third metadata generated according to the above process, the second metadata is restored through the third metadata, and the first metadata is restored through the second metadata.

The data management method and device provided by the present application have been described in detail above. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of the present application, several improvements and modifications can also be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations. There is no such actual relationship or sequence between operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element

The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

Professionals may further realize that the sub-modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of the two, in order to clearly illustrate the differences between hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

Claims (7)

1. a data management method, is characterized in that, described method comprises: establishing a first mapping for the metadata, assigning a first logical block address, and using the first logical block address as the first metadata; establishing a second mapping for the first metadata, assigning a second logical block address, saving the second logical block address in a set location, and using the second logical block address as the second metadata; processing the second metadata as user data; Wherein, processing the second metadata as user data includes: The second metadata is read out at power-on, and the first metadata is restored through the second metadata. 2. The method according to claim 1, wherein processing the narration second metadata as user data comprises: The second metadata is flashed to the flash, and the second metadata is saved to the corresponding position of the flash. 3. The method according to claim 1, wherein the metadata comprises l2p table or trim bitmap or superblock info. 4. The method according to claim 1, wherein storing the second logical block address in a set position comprises: The second logical block address is sequentially stored after the first metadata. 5. An apparatus for data management, wherein the apparatus comprises: a first metadata module for establishing a first mapping for metadata, assigning a first logical block address, and using the first logical block address as the first metadata; The second metadata module establishes a second mapping for the first metadata, allocates a second logical block address, saves the second logical block address in a set location, and stores the second logical block address address as second metadata; a data management module for processing the second metadata as user data; Wherein, the data management module includes: A data recovery submodule, configured to read out the second metadata when powered on, and recover the first metadata through the second metadata. 6. The apparatus according to claim 5, wherein the data management module comprises: The data flashing submodule is used for flashing the second metadata to the flash, and saving the second metadata to the corresponding position of the flash. 7. The apparatus of claim 6, further comprising: A data storage module, configured to store the second logical block address after the first metadata in sequence.

Images