CN108255432A - Write operation control method, system, device and storage medium based on bedding storage - Google Patents

Abstract

The invention discloses a kind of write operation control method, system, device and storage medium based on bedding storage, including：When receiving write operation requests, the type in the corresponding base ponds of data to be written is judged；If pond is deleted in base ponds to entangle, the type of the corresponding write operation of write operation requests is judged, when write operation is write for covering, be written into data write-in cache ponds, and base ponds are written in data to be written in the form of entangling and delete；If base ponds are copy pond, data write-in cache ponds are written into, and base ponds are written in data to be written in the form of copy.When reaching the preset standard of base ponds type and write operation type in the application, so that this part data to be written preserved in cache ponds and base ponds are consistent, reduce Data Migration pressure when cache ponds reach water level using space, file system is made to keep a smoother performance, improves user experience.

Description

Write operation control method, system, device and storage medium based on hierarchical storage

technical field

The present invention relates to the field of storage technology, in particular to a method, system, device and storage medium for controlling writing operations based on hierarchical storage.

Background technique

With the emergence of high-performance storage devices such as SSDs (Solid State Drives, solid state drives), tiered storage technologies are becoming more and more important in the enterprise storage market. Hierarchical storage improves the overall performance of the distributed file system by using high-performance storage devices as cache pools (cache pools) to store hot data, and low-performance high-capacity storage devices as backup pools (base pools) to store cold data. In writeback mode, for a write request, the client writes the data into the cache pool and returns it. When the used space of the cache pool reaches the preset water level threshold, the data migration operation is performed to flush the cooled data stored in the cache pool to backing pool, and remove them from the buffer pool.

It can be seen that in the prior art, for the writeback mode, the performance of the distributed file system is very high during the initial write, but when the used space of the cache pool reaches the water level, the data must be migrated to the backup pool, and the data migration pressure is high, and then Coupled with the read and write pressure of the client, the access performance is seriously reduced.

Therefore, how to provide a solution to the above-mentioned technical problems is a problem that those skilled in the art need to solve at present.

Contents of the invention

The object of the present invention is to provide a write operation control method, system, device, and storage medium based on hierarchical storage. When the preset standards of the base pool type and the write operation type are reached, the cache pool and the base pool save the part to be stored. The written data is consistent, which reduces the data migration pressure when the cache pool usage space reaches the water level, maintains a smoother performance of the file system, and improves the user experience.

In order to solve the above technical problems, the present invention provides a write operation control method based on hierarchical storage, including:

When receiving a write operation request, determine the type of the base pool corresponding to the data to be written;

If the base pool is an erasure pool, then judge the type of the write operation corresponding to the write operation request, and when the write operation is overwriting, write the data to be written into the cache pool, and write the The data to be written is written to the base pool in the form of erasure correction;

If the base pool is a copy pool, write the data to be written into the cache pool, and write the data to be written into the base pool in the form of a copy.

Preferably, the process of judging the type of the write operation corresponding to the write operation request is specifically:

Call the init_op_flags function in do_op to obtain the type of the write operation corresponding to the write operation request.

Preferably, the process of writing the data to be written into the base pool in the form of erasure correction or copy is specifically:

Call the do_proxy_write function to directly write the data to be written into the base pool in the form of erasure correction or copy.

Preferably, before calling the do_proxy_write function to directly write the data to be written into the base pool in the form of erasure correction or copy, the method also includes:

Call the maybe_handle_cache function to encapsulate the maybe_handle_cache_detail function;

Call the maybe_handle_cache_detail function to introduce proxy write mode.

In order to solve the above technical problems, the present invention also provides a write operation control system based on hierarchical storage, including:

Judging module, used to judge the type of the base pool corresponding to the data to be written when receiving the write operation request; if the base pool is an erasure correction pool, then trigger the erasure correction proxy write unit, if the base pool is Replica pool, trigger the replica agent to write to the unit;

The erasure correction proxy writing unit is used to judge the type of the write operation corresponding to the write operation request, and when the write operation is overwriting, write the data to be written into the cache pool, and write the The data to be written is written to the base pool in the form of erasure correction;

The copy proxy writing unit is configured to write the data to be written into the cache pool, and write the data to be written into the base pool in the form of a copy.

Preferably, the erasure correction proxy writing unit is specifically configured to call the init_op_flags function in the do_op to obtain the type of the write operation corresponding to the write operation request.

Preferably, the erasure correction proxy write unit or the copy proxy write unit is specifically configured to call the do_proxy_write function to directly write the data to be written into the base pool in the form of erasure correction or copy.

Preferably, the system further includes a calling unit, for calling the maybe_handle_cache function to encapsulate the maybe_handle_cache_detail function; calling the maybe_handle_cache_detail function to introduce the agent writing mode.

In order to solve the above technical problems, the present invention also provides a write operation control device based on hierarchical storage, including:

memory for storing computer programs;

A processor, configured to implement the steps of any one of the writing operation control methods described above when executing the computer program.

In order to solve the above-mentioned technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned write operations can be realized The steps of the control method.

The present invention provides a write operation control method based on hierarchical storage, comprising: when receiving a write operation request, judging the type of the base pool corresponding to the data to be written; if the base pool is an erasure pool, judging the write operation The type of write operation corresponding to the request. When the write operation is an overwrite, write the data to be written into the cache pool, and write the data to be written into the base pool in the form of erasure; if the base pool is a copy pool, then Write the data to be written into the cache pool, and write the data to be written into the base pool in the form of a copy.

Compared with the distributed file system in the prior art, the initial performance is high and the follow-up performance is low. In this application, the client initiates a write operation to the cache pool. When the preset standards for the base pool type and write operation type are reached, the data to be written is written to cache pool, and write the data to be written into the base pool at the same time, so that the data stored in the cache pool and the base pool are consistent, thereby reducing the data migration pressure when the space used by the cache pool reaches the water level, and making the distributed file system Maintain a smoother performance and improve user experience.

The present invention also provides a write operation control system, device and storage medium based on layered storage, which have the same beneficial effect as the above write operation control method.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the prior art and the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is a process flow chart of a write operation control method based on hierarchical storage provided by the present invention;

FIG. 2 is a schematic structural diagram of a write operation control system based on hierarchical storage provided by the present invention.

Detailed ways

The core of the present invention is to provide a write operation control method, system, device, and storage medium based on layered storage. The written data is consistent, which reduces the data migration pressure when the cache pool usage space reaches the water level, maintains a smoother performance of the file system, and improves the user experience.

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Please refer to FIG. 1. FIG. 1 is a process flow diagram of a method for controlling write operations based on hierarchical storage provided by the present invention. The method includes:

Step S11: When a write operation request is received, determine the type of the base pool corresponding to the data to be written;

It should be noted that hierarchical storage, also known as hierarchical storage, can divide the storage space into different storage levels, and store data in different storage levels according to the hierarchical storage management strategy to meet the user's requirements for processing speed and transmission bandwidth. In order to achieve the purpose of rationally utilizing storage space, improving storage system access performance, and reducing overall deployment costs.

When configuring the tiering function, you need to create different disk pools to configure different types of storage media. Based on this, a distributed file system with hierarchical storage needs at least 3 pools, 1 metadata pool, 1 backup pool, and 1 cache pool. Among them, 1 pool corresponds to a redundancy rule, and the general metadata pool is The copy pool, the backup pool is an erasure pool or copy pool, and the buffer pool is a copy pool. The cache pool is used as the cache of the backup pool, and the cache pool is invisible to the client.

In addition, the hierarchical storage management strategy here can support the mechanism that the data corresponding to the read operation request can be upgraded according to the latest access time: for the read operation request, if the data is within the latest access time, the data will be upgraded, otherwise it will not be upgraded. This mechanism reduces the frequency of data fetching, reduces system loss caused by unnecessary data fetching, and also improves the service life of high-performance storage devices. Hierarchical storage management strategies can also support the high and low water level method based on storage space. The high and low water level method requires real-time monitoring of the space used by storage devices to determine whether the space used by high-performance storage devices has reached a certain water level threshold. When the used space reaches the water level threshold triggers the execution of data migration operations.

Step S12: If the base pool is an erasure pool, determine the type of the write operation corresponding to the write operation request, and when the write operation is an overwrite, write the data to be written into the cache pool, and write the data to be written as an erasure If the base pool is a copy pool, the data to be written is written to the cache pool, and the data to be written is written to the base pool in the form of a copy.

Specifically, considering that there is a rollback operation for erasure correction, such as erasure correction 4+2, one piece of data is divided into 4 pieces, 2 pieces of verification code are generated, 6 pieces of data will be written to the disks of 6 nodes, and all 6 pieces of data Successful writing is considered successful. If a node fails to write, the entire write operation will fail and must be rolled back to the state before writing. Otherwise, some nodes will have new data and some nodes will have old data. , resulting in inconsistency between the data saved on the disk and the actual data.

Therefore, when the base pool is an erasure pool, it is necessary to determine the type of the write operation corresponding to the received write operation request. The data is written to the base pool in the form of erasure correction. It can be seen that the data stored in the cache pool and the base pool are consistent, thereby reducing the data migration pressure when the cache pool’s usage space reaches the water level, making the distributed file system A smoother performance can be maintained. In addition, in writeback mode, when the cache misses, this application no longer waits for the object to be promoted from the base pool to the cache pool, and then writes it into the cache pool, but directly writes the data into the cache pool and writes it on behalf of The base pool reduces the latency of write operations and improves the performance of write operations. If the write operation is not overwriting, such as modifying, then use the method of the existing technology to determine whether the cache is hit. When the cache hits, directly modify the data to be modified in the cache pool. The pool is promoted to the cache pool, and the data to be modified is modified in the cache pool.

Specifically, the copy does not need to slice the data. What is written is a complete set of data. If one copy of the data is successfully written, it is considered successful. There is no rollback operation, so there is no need to distinguish the type of write operation. Based on this, when the base pool is a copy pool, it directly writes the data to be written into the cache pool and returns it, and writes the data to be written into the base pool as a proxy in the form of a copy. The data stored in the cache pool and the base pool are consistent, which reduces the data migration pressure when the cache pool reaches the water level, enables the distributed file system to maintain a smoother performance, and eases the initial impact of the distributed file system. The defect of high performance and low follow-up performance.

The present invention provides a write operation control method based on hierarchical storage, comprising: when receiving a write operation request, judging the type of the base pool corresponding to the data to be written; if the base pool is an erasure pool, judging the write operation The type of write operation corresponding to the request. When the write operation is an overwrite, write the data to be written into the cache pool, and write the data to be written into the base pool in the form of erasure; if the base pool is a copy pool, then Write the data to be written into the cache pool, and write the data to be written into the base pool in the form of a copy.

Compared with the distributed file system in the prior art, the initial performance is high and the follow-up performance is low. In this application, the client initiates a write operation to the cache pool. When the preset standards for the base pool type and write operation type are reached, the data to be written is written to cache pool, and write the data to be written into the base pool at the same time, so that the data stored in the cache pool and the base pool are consistent, thereby reducing the data migration pressure when the space used by the cache pool reaches the water level, and making the distributed file system Maintain a smoother performance and improve user experience.

On the basis of above-mentioned embodiment:

As a preferred embodiment, the process of judging the type of the write operation corresponding to the write operation request is specifically as follows:

Call the init_op_flags function in do_op to obtain the type of write operation corresponding to the write operation request.

Specifically, when judging the type of the write operation corresponding to the write operation request in this application, the init_op_flags function is directly called in do_op to determine whether the write operation corresponding to the write operation request is an overwrite, and the result of the function call can clearly display the write operation type, so that The corresponding follow-up operations are carried out, and the implementation is simple, the development difficulty is relatively low, and the development time is shortened, thereby reducing the production cost of the enterprise.

As a preferred embodiment, the process of writing the data to be written into the base pool in the form of erasure correction or copy is specifically as follows:

Call the do_proxy_write function to directly write the data to be written to the base pool in the form of erasure correction or copy.

Specifically, if the base pool is an erasure pool, and the write operation is an overwrite, call the do_proxy_write function to directly write the data to be written into the base pool in the form of erasure correction. If the base pool is a copy pool, call the do_proxy_write function to directly write the data to be written into the base pool in the form of a copy. In this way, the do_proxy_write function is called to implement the proxy write mode, which is more flexible and has a wide range of applications.

As a preferred embodiment, before calling the do_proxy_write function to directly write the data to be written into the base pool in the form of erasure or copy, the method also includes:

Call the maybe_handle_cache function to encapsulate the maybe_handle_cache_detail function;

Call the maybe_handle_cache_detail function to introduce proxy write mode.

Specifically, in order to better realize proxy writing, before calling the do_proxy_write function in this application to directly write the data to be written into the base pool in the form of erasure or copy, some initialization work will be performed, specifically, calling the maybe_handle_cache function Encapsulate the maybe_handle_cache_detail function, and then call the maybe_handle_cache_detail function (the entry function written by the agent) to introduce the agent write mode.

Please refer to FIG. 2. FIG. 2 is a schematic structural diagram of a write operation control system based on hierarchical storage provided by the present invention. The system includes:

The judging module 1 is used to judge the type of the base pool corresponding to the data to be written when a write operation request is received; if the base pool is an erasure pool, trigger the erasure proxy writing unit 2; if the base pool is a copy pool , then trigger the copy agent to write to unit 3;

The erasure correction proxy writing unit 2 is used to determine the type of the write operation corresponding to the write operation request. When the write operation is an overwrite, write the data to be written into the cache pool, and write the data to be written in the form of erasure Write to the base pool;

The copy proxy writing unit 3 is used to write the data to be written into the cache pool, and write the data to be written into the base pool in the form of a copy.

As a preferred embodiment, the erasure correction proxy writing unit 2 is specifically configured to call the init_op_flags function in the do_op to obtain the type of the write operation corresponding to the write operation request.

As a preferred embodiment, the erasure correction proxy writing unit 2 or the copy proxy writing unit 3 is specifically configured to call the do_proxy_write function to directly write the data to be written into the base pool in the form of erasure correction or copy.

As a preferred embodiment, the system further includes a calling unit for calling the maybe_handle_cache function to encapsulate the maybe_handle_cache_detail function; calling the maybe_handle_cache_detail function to introduce the agent writing mode.

For the introduction of the system provided by the present invention, please refer to the foregoing method embodiments, and the present invention will not be repeated here.

The present invention also provides a write operation control device based on hierarchical storage, including:

memory for storing computer programs;

A processor, configured to implement the steps of any one of the above writing operation control methods when executing the computer program.

For the introduction of the device provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here.

The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the above-mentioned writing operation control methods are realized.

For the introduction of the computer-readable storage medium provided by the present invention, please refer to the foregoing method embodiments, and the present invention will not be repeated here.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. As for the systems, devices and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and for relevant details, please refer to the description of the methods.

It should also be noted that in this specification, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or order between the operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. a kind of write operation control method based on bedding storage, which is characterized in that including：

When receiving write operation requests, the type in the corresponding base ponds of data to be written is judged；

If pond is deleted in the base ponds to entangle, judge the type of the corresponding write operation of the write operation requests, behaviour is write when described When being write as covering, by the data write-in cache ponds to be written, and institute is written in the data to be written in the form of entangling and delete State base ponds；

If the base ponds are copy pond, the data to be written are written into cache ponds, and the data to be written with The base ponds are written in the form of copy.

2. write operation control method according to claim 1, which is characterized in that described to judge that the write operation requests correspond to The process of type of write operation be specially：

The type of the corresponding write operation of the init_op_flags functions acquisition write operation requests is called in do_op.

3. write operation control method according to claim 1, which is characterized in that described that the data to be written are deleted with entangling Or the process in the form write-in base ponds of copy is specially：

Call do_proxy_write functions directly the data to be written by entangle delete or copy in the form of the write-in base Pond.

4. write operation control method according to claim 3, which is characterized in that the calling do_proxy_write functions Directly the data to be written by entangle delete or copy in the form of before the write-in base ponds, this method further includes：

Call maybe_handle_cache functions encapsulation maybe_handle_cache_detail functions；

The maybe_handle_cache_detail functions introducing is called to act on behalf of write mode.

5. a kind of write operation control system based on bedding storage, which is characterized in that including：

Judgment module, for when receiving write operation requests, judging the type in the corresponding base ponds of data to be written；If institute It states base ponds and deletes pond to entangle, then triggering, which entangles to delete, acts on behalf of writing unit, if the base ponds are copy pond, triggers replica agent Writing unit；

It is described entangle to delete act on behalf of writing unit, for judging the type of the corresponding write operation of the write operation requests, behaviour is write when described When being write as covering, by the data write-in cache ponds to be written, and institute is written in the data to be written in the form of entangling and delete State base ponds；

The replica agent writing unit, for by the data to be written write-in cache ponds, and the data to be written with The base ponds are written in the form of copy.

6. write operation control system according to claim 5, which is characterized in that it is described entangle to delete act on behalf of writing unit and specifically use In the type that the corresponding write operation of the init_op_flags functions acquisition write operation requests is called in do_op.

7. write operation control system according to claim 5, which is characterized in that it is described entangle to delete act on behalf of writing unit or described Replica agent writing unit is specifically used for calling do_proxy_write functions that directly the data to be written are deleted or secondary to entangle The base ponds are written in this form.

8. write operation control system according to claim 7, which is characterized in that the system further includes call unit, is used for Call maybe_handle_cache functions encapsulation maybe_handle_cache_detail functions；Call the maybe_ Write mode is acted on behalf of in the introducing of handle_cache_detail functions.

9. a kind of write operation control device based on bedding storage, which is characterized in that including：

Memory, for storing computer program；

Processor realizes the write operation control method as described in any one of Claims 1-4 during for performing the computer program The step of.

10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium Program, realizing the write operation control method as described in any one of Claims 1-4 when the computer program is executed by processor Step.