CN118656028A - Storage method, device, computer equipment, storage medium and program product - Google Patents

Abstract

The present application relates to a storage method, apparatus, computer device, storage medium, and program product. The method comprises the following steps: and determining a corresponding target resource pool of the received storage request in the cache space, and adjusting the storage space of the target resource pool according to the shared resource pool in the cache under the condition that the target resource pool does not meet the preset storage condition, so that the data requested by the storage request is stored according to the adjusted target resource pool. By establishing the shared resource pool, under the condition that the target resource pool does not meet the preset storage condition, the storage space of the target resource pool is dynamically adjusted, so that the storage space of the target resource pool can be adjusted according to the requirement of the storage request, the storage efficiency of the storage request is improved, the resource utilization rate of the cache space is improved, further, the data requested by the storage request is stored according to the adjusted target resource pool, and the cache performance can be further exerted.

Description

Storage method, device, computer equipment, storage medium and program product

Technical Field

The present application relates to the technical field of data storage, and in particular to a storage method, apparatus, computer equipment, storage medium and program product.

Background Art

With the generation of massive data, high requirements are placed on storage, so large-capacity mechanical hard disks are widely used in storage systems. However, due to the poor performance of mechanical hard disks, the existing storage system can use solid-state drive cache to optimize this problem, which can not only solve the balance between capacity and cost, but also ensure the large space advantage of mechanical hard disk storage and the high performance characteristics of solid-state drives.

However, when storing multi-protocol data, the above storage method combining solid-state hard disks for caching still has the problem of low storage efficiency.

Summary of the invention

Based on this, it is necessary to provide a storage method, device, computer equipment, storage medium and program product that can improve storage efficiency in response to the above technical problems.

In a first aspect, the present application provides a storage method, comprising:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

In one embodiment, the adjusting the storage space of the target resource pool according to the shared resource pool in the cache includes:

Determine the space capacity to be adjusted based on the remaining space capacity in the target resource pool and the preset storage conditions;

Adjust the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted.

In one embodiment, the step of adjusting the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted includes:

If the space capacity of the shared resource pool is less than the space capacity to be adjusted, the storage space of the target resource pool is adjusted according to the current request processing pressure of each type of resource pool;

If the space capacity of the shared resource pool is not less than the space capacity to be adjusted, the space of the space capacity to be adjusted is allocated from the shared resource pool to the target resource pool.

In one embodiment, the above-mentioned adjusting the storage space of the target resource pool according to the current request processing pressure of each type of resource pool includes:

Determine the target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range;

The storage space of each type of resource pool is adjusted so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

In one embodiment, the preset storage condition includes any one of the following:

The amount of data requested by the storage request is less than the remaining space capacity in the target resource pool;

The data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

In one embodiment, the method further comprises:

The local cache space is divided into resource pools, into multiple types of resource pools and shared resource pools.

In one embodiment, storing the data requested by the storage request according to the adjusted target resource pool includes:

Determine the remaining space capacity of the adjusted target resource pool;

If the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, the data requested by the storage request is cached in the adjusted target resource pool;

If the remaining space of the adjusted target resource pool is less than the amount of data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk.

In one embodiment, the method further comprises:

When a read request is received, the data requested by the read request is read from the cache space;

If the read fails, the data requested by the read request is read from the mechanical hard disk.

In a second aspect, the present application further provides a storage device, including:

A determination module, used to determine a target resource pool corresponding to the received storage request in the cache space;

An adjustment module, used to adjust the storage space of the target resource pool according to the shared resource pool in the cache when the target resource pool does not meet the preset storage conditions;

The storage module is used to store the data requested by the storage request according to the adjusted target resource pool.

In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

The above storage method, device, computer equipment, storage medium and program product determine the target resource pool corresponding to the received storage request in the cache space, and when the target resource pool does not meet the preset storage conditions, adjust the storage space of the target resource pool according to the shared resource pool in the cache, so as to store the data requested by the storage request according to the adjusted target resource pool. By establishing a shared resource pool, when the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is dynamically adjusted, so that the storage space of the target resource pool can be adjusted according to the needs of the storage request, thereby improving the storage efficiency of the storage request and the resource utilization of the cache space. Furthermore, the data requested by the storage request is stored according to the adjusted target resource pool, which can further exert the cache performance.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

FIG1 is an application environment diagram of a storage method in one embodiment;

FIG2 is a schematic diagram of a storage method in one embodiment;

FIG3 is a schematic diagram of a storage system architecture in one embodiment;

FIG4 is a schematic diagram of a flow chart of a storage method in another embodiment;

FIG5 is a schematic diagram of a flow chart of a storage method in another embodiment;

FIG6 is a schematic flow chart of a storage method in another embodiment;

FIG7 is a schematic diagram of a flow chart of a storage method in another embodiment;

FIG8 is a schematic diagram of resource partitioning in another embodiment;

FIG9 is a schematic flow chart of a storage method in another embodiment;

FIG10 is a schematic flow chart of a storage method in another embodiment;

FIG11 is a schematic flow chart of a storage method in another embodiment;

FIG12 is a block diagram of a storage device in one embodiment;

FIG. 13 is a diagram showing the internal structure of a computer device in one embodiment.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

With the generation of massive data, high requirements are placed on storage needs. Therefore, large-capacity mechanical hard disks (HDD) are widely used in storage systems. Due to the poor read and write performance of HDD, existing storage systems usually use solid-state drives (SSD) cache to optimize this problem. This not only solves the balance between capacity and cost, but also ensures the large space advantage of mechanical hard disk storage and the high performance characteristics of solid-state drives.

After the storage system is deployed and implemented, usually a storage service of a single storage protocol of a type can provide storage services based on the cache instance corresponding to the storage protocol. Since the user side may have storage requirements for multiple protocols, a distributed storage system that supports multiple storage protocols has emerged. The client layer contains client services corresponding to different storage protocols to provide corresponding services for users. When input/output (IO) requests of different protocols are sent to the storage system, they are converted into corresponding IO requests on the mechanical hard disk through the disk device mapping layer of the volume device and the mechanical hard disk. Different cache instances are configured for different storage protocols. The IO request is first written to the corresponding cache instance, and the processing result of the IO request can be returned to the client. In the cache instance of the read-write mode, the flushing process will flush the written data back to the mechanical hard disk at a fixed time. At this time, the data becomes clean data, and the corresponding space on the solid-state drive can be reused. This cache mode can effectively improve the write performance. At the same time, there is a copy of the data in the solid-state drive, which can provide a read hit rate in the future and speed up the read request performance.

In the existing storage methods that support multiple protocols, each cache instance will use its own memory, threads, hard disk and other resources, which cannot be shared between different cache instances. When the service of a certain protocol is stopped or the load is low, the resources are relatively abundant, and the remaining resources cannot be fully utilized by other cache instances, resulting in a waste of resources and unable to ensure the efficient execution of the cache. Therefore, a storage method that can improve storage efficiency is needed.

The storage method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the client 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 receives the storage request sent by the client 102. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

In one embodiment, as shown in FIG. 2 , a storage method is provided, which is described by taking the method applied to the server in FIG. 1 as an example, including:

S201, determining a target resource pool corresponding to a received storage request in a cache space.

Among them, the storage request may be a write request in an IO request; the cache space may include cache resources of a solid state drive (SSD) and processing resources corresponding to a memory; and the target resource pool may be an independent resource pool of a storage type corresponding to the storage request.

In an embodiment of the present application, the client layer can be protected in the server, and the storage request sent by the client to the server can be received through the client layer. The mapping relationship between the storage request and the mechanical hard disk can be determined through the mapping layer in the server, thereby converting the storage request into a storage request corresponding to the mechanical hard disk (Hard Disk Drive, HDD), and determining the target resource pool corresponding to the storage request in the cache space, so as to process the storage request according to the status of the target resource pool.

Optionally, the target resource pool corresponding to the storage request in the cache space can be determined according to the storage protocol corresponding to the storage request. Among them, the storage protocol can include a block storage protocol, a file storage protocol and an object storage protocol, and accordingly, the independent resource pool can include a block resource pool, a file resource pool and an object resource pool. If the storage protocol corresponding to the storage request is a block storage protocol, the target resource pool is a block resource pool; if the storage protocol corresponding to the storage request is a file storage protocol, the target resource pool is a file resource pool; if the storage protocol corresponding to the storage request is an object storage protocol, the target resource pool is an object resource pool.

Exemplarily, as shown in FIG3 , the server can provide storage services to block storage clients, file storage clients, and object storage clients through the client layer, and the cache resources include a block resource pool, a file resource pool, an object resource pool, a shared resource pool, and a thread resource pool.

S202: When the target resource pool does not meet a preset storage condition, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache.

The preset storage condition includes any one of the following: (1) the amount of data requested by the storage request is less than the remaining space capacity in the target resource pool; (2) the data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

In an embodiment of the present application, when the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted. Optionally, the space capacity of a preset size in the shared resource pool can be allocated to the target resource pool to expand the storage space of the target resource pool; or, a preset proportion of the space capacity in the shared resource pool can be allocated to the target resource pool to expand the storage space of the target resource pool. Exemplarily, the space capacity of the preset size can be 8MB. When the target resource pool does not meet the preset storage conditions, 8MB of space in the shared resource pool can be allocated to the target resource pool; or, the preset proportion can be 10%. If the current space capacity of the shared resource pool is 100MB, 10MB of space in the shared resource pool is allocated to the target resource pool.

Optionally, when the target resource pool meets the preset storage conditions, the data requested by the storage request is stored, that is, the data carried in the storage request is cached to the target resource pool, and the data requested by the storage request is flushed back to the mechanical hard disk according to the preset data flushing method. As an optional implementation, after caching the data carried in the storage request to the target resource pool, the server can return a message to the client indicating that the storage request is successful.

Optionally, the data can be flushed back to the mechanical hard disk when the data is cached in the target resource pool for a preset period of time; or, the data can be flushed back to the mechanical hard disk in a unified manner after the storage space of the target resource pool is full of data.

S203: Store the data requested by the storage request according to the adjusted target resource pool.

In an embodiment of the present application, the storage location of the data requested by the storage request is determined according to the adjusted target resource pool, and the storage location may include any one of a mechanical hard disk and a solid-state hard disk.

As an optional implementation, the data requested by the storage request can be stored in a solid-state drive if the adjusted target resource pool meets the preset storage conditions; the data requested by the storage request can be stored in a mechanical hard drive if the adjusted target resource pool does not meet the preset storage conditions.

As another optional implementation, the heat value of the clean data in the target resource pool may be determined, and the storage location where the clean data with the lowest heat value is located may be determined, and the data requested by the storage request may be stored in the storage location.

In the above storage method, the target resource pool corresponding to the received storage request in the cache space is determined. When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache, so that the data requested by the storage request is stored according to the adjusted target resource pool. By establishing a shared resource pool, when the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is dynamically adjusted, so that the storage space of the target resource pool can be adjusted according to the needs of the storage request, thereby improving the storage efficiency of the storage request and the resource utilization of the cache space. Furthermore, the data requested by the storage request is stored according to the adjusted target resource pool, which can further exert the cache performance.

In one embodiment, an implementation of the above S202 is provided, as shown in FIG4 , wherein the above “adjusting the storage space of the target resource pool according to the shared resource pool in the cache” includes:

S301, determining the space capacity to be adjusted according to the remaining space capacity in the target resource pool and a preset storage condition.

In an embodiment of the present application, when the data amount of the data requested by the storage request is less than the remaining space capacity in the target resource pool, the difference between the data amount of the data requested by the storage request and the remaining space capacity in the target resource pool is determined, and the difference is used as the space capacity to be adjusted; or, when the data flushing efficiency of the target resource pool is less than the cache efficiency, the space capacity to be adjusted is determined to be the data amount of the data requested by the storage request.

S302: Adjust the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted.

In the embodiment of the present application, it is determined whether the space capacity of the shared resource pool is greater than the space capacity to be adjusted, and the storage space of the target resource pool is adjusted according to the determination result.

Optionally, if the space capacity of the shared resource pool is greater than or equal to the space capacity to be adjusted, part of the space can be allocated from the shared resource pool to the target resource pool to expand the storage space of the target resource pool; if the space capacity of the shared resource pool is less than the space capacity to be adjusted, the storage space of the target resource pool may not be adjusted, and the data requested by the storage request can be stored on the mechanical hard disk.

The method described in this embodiment first determines the capacity of the space to be adjusted, and then adjusts the storage space of the target resource pool according to the space capacity of the shared resource pool and the capacity of the space to be adjusted, so that the adjustment of the storage space of the target resource pool is more accurate and the probability of storage resource waste is reduced.

In one embodiment, an implementation of the above S302 is provided, as shown in FIG5 , wherein the above “adjusting the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted” includes:

S401: If the space capacity of the shared resource pool is less than the space capacity to be adjusted, the storage space of the target resource pool is adjusted according to the current request processing pressure of each type of resource pool.

Each type of resource pool is an independent resource pool corresponding to each storage type, and each type of resource pool includes a target resource pool. Exemplarily, each type of resource pool may include a block resource pool, a file resource pool, and an object resource pool.

In an embodiment of the present application, when the space capacity of the shared resource pool is less than the space capacity to be adjusted, the current request processing pressure of each type of resource pool is determined, and the storage space of each type of resource pool and the shared resource pool is adjusted according to the current request processing pressure of each type of resource pool.

Optionally, a pressure threshold may be preset, and if the current request processing pressure of a resource pool of a certain type is less than the pressure threshold, a space of a preset size in the resource pool is divided into the shared resource pool.

Optionally, a minimum space capacity of each type of resource pool may be pre-set, and the minimum space capacity of each type of resource pool may be the same or different, and the space capacity of each type of resource pool is always greater than the minimum value corresponding to the resource pool; or, a minimum proportion of the space capacity of each type of resource pool to the total cache resources may be pre-set, and the minimum proportion of the space capacity of each type of resource pool to the total cache resources may be the same or different, and the space capacity of each type of resource pool to the total cache resources is always greater than the minimum proportion corresponding to the resource pool.

S402: If the space capacity of the shared resource pool is not less than the space capacity to be adjusted, the space of the space capacity to be adjusted is allocated from the shared resource pool to the target resource pool.

In an embodiment of the present application, when the space capacity of the shared resource pool is not less than the space capacity to be adjusted and the space capacity in the shared resource pool is sufficient, the space of the space capacity to be adjusted is directly allocated from the shared resource pool to the target resource pool to expand the storage space of the target resource pool.

The method described in this embodiment adjusts the storage space of the target resource pool according to the current request processing pressure of each type of resource pool when the space capacity of the shared resource pool is in different situations, or divides the space with the space capacity to be adjusted from the shared resource pool to the target resource pool, and reasonably allocates the space capacity of the shared resource pool and the target resource pool, thereby improving the cache efficiency of the server.

In one embodiment, an implementation of the above S401 is provided, as shown in FIG6 , wherein the above “adjusting the storage space of the target resource pool according to the current request processing pressure of each type of resource pool” includes:

S501, determining target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range.

In an embodiment of the present application, when the space capacity of the shared resource pool is less than the space capacity to be adjusted, the space capacity of the shared resource pool is insufficient, and the current request processing pressure of each type of resource pool is determined, and then the ratio of the current request processing pressure of each type of resource pool is determined based on the current request processing pressure of each type of resource pool, and then the target space capacity of each type of resource pool is determined based on the ratio of the current request processing pressure of each type.

Optionally, the current request processing pressure of each type of resource pool can be determined based on the number of storage requests of each type received in a preset time period. For example, if the number of storage requests corresponding to the block storage type received in the preset time period is 10, the number of storage requests corresponding to the file storage type received is 15, and the number of storage requests corresponding to the file storage type received is 25, then the ratio of the current request processing pressure of each type is 2:3:5, and the target space capacity of each type of resource pool is also 2:3:5, that is, the target space capacity of the block storage resource pool accounts for 20% of the space capacity of the total cache resource pool, the target space capacity of the file storage resource pool accounts for 30% of the space capacity of the total cache resource pool, and the target space capacity of the object storage resource pool accounts for 50% of the space capacity of the total cache resource pool.

In an embodiment of the present application, the target space capacity is within a preset capacity range. For example, if the preset capacity range of the space capacity of the object storage resource pool is 10% to 40%, and the target space capacity of the object storage resource pool determined based on the current request processing pressure of each type of resource pool accounts for 50% of the space capacity of the total cache resource pool, then the target space capacity of the object storage resource pool is adjusted to 40% of the space capacity of the total cache resource pool.

S502, adjusting the storage space of each type of resource pool so that the adjusted space capacity of each type of resource pool reaches the target space capacity.

In the embodiment of the present application, the target space capacity of each type of resource pool adjusts the storage space of each type of resource pool.

As an optional implementation, the resource pool whose current space capacity is greater than the target capacity space can be adjusted first, and the first difference between the current space capacity of the resource pool and the target space capacity can be determined, thereby dividing the space of the first difference from the resource pool to the shared resource pool; then the resource pool whose current space capacity is greater than the target capacity space can be adjusted, and the second difference between the target space capacity of the resource pool and the current space capacity can be determined, and the space of the second difference can be divided from the shared resource pool to the resource pool.

As another optional implementation, the space in a resource pool that is larger than the target capacity space may be divided into resource pools that are smaller than the target capacity space, so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

Optionally, since each target space capacity is within the preset capacity range, after adjusting the storage space of each type of resource pool, there may be space capacity in the shared resource pool. The space capacity in the shared resource pool can be reserved, or the space capacity in the shared resource pool can be allocated to a resource pool that does not exceed the preset capacity range.

The method described in this embodiment adjusts the storage space of each type of resource pool according to the current request processing pressure of each type of resource pool when the space capacity of the shared resource pool is insufficient, so as to make the resource allocation of each resource pool more reasonable and maximize the resource utilization.

In one embodiment, as shown in FIG7 , the storage method further includes:

S204, dividing the local cache space into resource pools, into multiple types of resource pools and a shared resource pool.

In an embodiment of the present application, as shown in the resource partitioning diagram of Figure 8, the local cache space is divided into resource pools according to a preset partitioning rule to divide multiple types of resource pools and shared resource pools. Optionally, the preset partitioning rule can be the initial proportions of the multiple types of resource pools and the shared resource pool in the local cache space. Exemplarily, the initial proportions may include each type of resource pool occupying 20% of the local cache space, and the shared resource pool occupying 40% of the local cache space; or, the initial proportions may also include the block storage resource pool occupying 20% of the local cache space, the file storage resource pool occupying 20% of the local cache space, and the object storage resource pool occupying 30% of the local cache space, and the shared resource pool occupying 30% of the local cache space.

The method described in this embodiment pre-divides multiple types of resource pools and shared resource pools, so that when each type of resource pool is short of capacity space, it can request space in the shared resource pool for replenishment, making the server more flexible in utilizing capacity space when processing storage requests, thereby improving storage efficiency.

In one embodiment, an implementation of the above S203 is provided, as shown in FIG8 , the above “storing the data requested by the storage request according to the adjusted target resource pool” includes:

S601: Determine the remaining space capacity of the adjusted target resource pool.

In an embodiment of the present application, the use data of the resource pool can be collected in real time through a preset tool to determine the remaining space capacity of the adjusted target resource pool, which includes the space capacity of the independent resource pool corresponding to the type and the space capacity of a portion of the shared resource pool, and determines the remaining space capacity of the adjusted target resource pool, that is, the space capacity of uncached data and/or the space capacity of cached clean data in the adjusted target resource pool. Further, determine whether the remaining space capacity of the adjusted target resource pool is less than the data volume of the data requested by the storage request.

S602: If the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, cache the data requested by the storage request to the adjusted target resource pool.

In an embodiment of the present application, if the remaining space capacity of the adjusted target resource pool is greater than or equal to the data amount requested by the storage request, the remaining space capacity of the current target resource pool meets the preset storage conditions, and the data requested by the storage request is cached to the adjusted target resource pool.

S603: If the remaining space of the adjusted target resource pool is smaller than the amount of data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk.

In an embodiment of the present application, if the remaining space of the adjusted target resource pool is less than the amount of data requested by the storage request, the remaining space capacity of the current target resource pool still does not meet the preset storage conditions, and the data requested by the storage request cannot be cached to the solid-state drive, and the data requested by the storage request is stored in the mechanical hard disk.

The method described in this embodiment determines the remaining space capacity of the adjusted target resource pool to determine whether the data requested by the storage request can be cached to the adjusted target resource pool, thereby fully utilizing the space capacity of the target resource pool and improving the data storage speed.

In one embodiment, as shown in FIG9 , the storage method further includes:

S205: When a read request is received, data requested by the read request is read from the cache space.

In the embodiment of the present application, when a read request is received, the server can protect the client layer, receive the read request sent by the client to the server through the client layer, and determine the mapping relationship between the read request and the mechanical hard disk through the mapping layer in the server, so as to convert the storage request into a read request corresponding to the mechanical hard disk, and determine the resource pool corresponding to the read request in the cache space. Further, the data requested by the read request is searched from the resource pool, and if the search is successful, the data requested by the read request is read from the resource pool, and the data requested by the read request is sent to the client.

Optionally, if the read is successful, the heat of the storage location where the data is located can be increased, and the data can continue to be stored in the cache as hot data.

S206: If the reading fails, the data requested by the read request is read from the mechanical hard disk.

In an embodiment of the present application, if the data requested by the read request is not cached in the cache space, the data requested by the read request is read from the resource pool, and the data requested by the read request is sent to the client.

The method described in this embodiment can directly read the data requested by the read request from the cache space if the data requested by the read request exists in the cache space, thereby improving the data reading speed; if the data requested by the read request does not exist in the cache space, the data requested by the read request is read from the mechanical hard disk, thereby ensuring the data reading success rate.

Based on all the above embodiments, a storage method is also provided, as shown in FIG10 , the method includes:

S1, divide the local cache space into resource pools, and divide them into multiple types of resource pools and shared resource pools.

S2, determining a target resource pool corresponding to the received storage request in the cache space.

S3, when the target resource pool does not meet the preset storage conditions, determines the space capacity to be adjusted based on the remaining space capacity in the target resource pool and the preset storage conditions; the preset storage conditions include: the amount of data requested by the storage request is less than the remaining space capacity in the target resource pool; or, the data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

S4, if the space capacity of the shared resource pool is less than the space capacity to be adjusted, execute S5-S6; if the space capacity of the shared resource pool is not less than the space capacity to be adjusted, execute S7.

S5, determining the target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range.

S6, adjusting the storage space of each type of resource pool so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

S7, allocating the space whose capacity is to be adjusted from the shared resource pool to the target resource pool.

S8, determining the remaining space capacity of the adjusted target resource pool.

S9, if the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, the data requested by the storage request is cached to the adjusted target resource pool; if the remaining space of the adjusted target resource pool is less than the data volume of the data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk.

S10, when a read request is received, the data requested by the read request is read from the cache space; if the read fails, the data requested by the read request is read from the mechanical hard disk.

In the above storage method, the target resource pool corresponding to the received storage request in the cache space is determined. When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache, so that the data requested by the storage request is stored according to the adjusted target resource pool. By establishing a shared resource pool, when the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is dynamically adjusted, so that the storage space of the target resource pool can be adjusted according to the needs of the storage request, thereby improving the storage efficiency of the storage request and the resource utilization of the cache space. Furthermore, the data requested by the storage request is stored according to the adjusted target resource pool, which can further exert the cache performance.

The methods described in the above steps are all described in the above embodiments. Please refer to the above description for details and will not be repeated here.

It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application also provides a storage device for implementing the storage method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more storage device embodiments provided below can refer to the limitations on the storage method above, and will not be repeated here.

In one embodiment, as shown in FIG11 , a storage device is provided, including: a determination module 10, an adjustment module and a storage module, wherein:

The determination module 10 is used to determine the target resource pool corresponding to the received storage request in the cache space.

The adjustment module 11 is used to adjust the storage space of the target resource pool according to the shared resource pool in the cache when the target resource pool does not meet the preset storage condition.

The storage module 12 is used to store the data requested by the storage request according to the adjusted target resource pool.

In one embodiment, the adjustment module 11 includes: a first determination unit and an adjustment unit, wherein:

The first determination unit is used to determine the space capacity to be adjusted based on the remaining space capacity in the target resource pool and preset storage conditions; the preset storage conditions include any one of the following: the amount of data requested by the storage request is less than the remaining space capacity in the target resource pool; the data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

The adjustment unit is used to adjust the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted.

In one embodiment, the above-mentioned adjustment unit is specifically used to adjust the storage space of the target resource pool according to the current request processing pressure of each type of resource pool when the space capacity of the shared resource pool is less than the space capacity to be adjusted; and to allocate the space of the space capacity to be adjusted from the shared resource pool to the target resource pool when the space capacity of the shared resource pool is not less than the space capacity to be adjusted.

In one embodiment, the above-mentioned adjustment unit is specifically used to determine the target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range; and the storage space of each type of resource pool is adjusted so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

In one embodiment, the storage device further includes: a partitioning module, configured to partition the local cache space into resource pools, and divide the resource pools into multiple types of resource pools and a shared resource pool.

In one embodiment, the storage module 12 includes: a second determination unit, a first storage unit, and a second storage unit, wherein:

The second determining unit is used to determine the remaining space capacity of the adjusted target resource pool.

The first storage unit is used to cache the data requested by the storage request to the adjusted target resource pool when the remaining space capacity of the adjusted target resource pool is not less than the data amount of the data requested by the storage request.

The second storage unit is used to store the data requested by the storage request in the mechanical hard disk when the remaining space of the adjusted target resource pool is smaller than the data amount of the data requested by the storage request.

In one embodiment, the storage device further includes: a first reading module and a second reading module, wherein:

The first reading module is used to read the data requested by the read request from the cache space when receiving the read request.

The second reading module is used to read the data requested by the read request from the mechanical hard disk when the reading fails.

Each module in the above storage device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG12. The computer device includes a processor, a memory, an input/output interface (Input/Output, referred to as I/O) and a communication interface. The processor, the memory and the input/output interface are connected via a system bus, and the communication interface is connected to the system bus via the input/output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store storage data. The input/output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a storage method is implemented.

Those skilled in the art will understand that the structure shown in FIG. 12 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Determine the space capacity to be adjusted based on the remaining space capacity in the target resource pool and the preset storage conditions;

Adjust the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

If the space capacity of the shared resource pool is less than the space capacity to be adjusted, the storage space of the target resource pool is adjusted according to the current request processing pressure of each type of resource pool;

If the space capacity of the shared resource pool is not less than the space capacity to be adjusted, the space of the space capacity to be adjusted is allocated from the shared resource pool to the target resource pool.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Determine the target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range;

The storage space of each type of resource pool is adjusted so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

The amount of data requested by the storage request is less than the remaining space capacity in the target resource pool;

The data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

The local cache space is divided into resource pools, into multiple types of resource pools and shared resource pools.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

Determine the remaining space capacity of the adjusted target resource pool;

If the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, the data requested by the storage request is cached in the adjusted target resource pool;

If the remaining space of the adjusted target resource pool is less than the amount of data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk.

In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

When a read request is received, the data requested by the read request is read from the cache space;

If the read fails, the data requested by the read request is read from the mechanical hard disk.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

Determine the space capacity to be adjusted based on the remaining space capacity in the target resource pool and the preset storage conditions;

Adjust the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

If the space capacity of the shared resource pool is less than the space capacity to be adjusted, the storage space of the target resource pool is adjusted according to the current request processing pressure of each type of resource pool;

If the space capacity of the shared resource pool is not less than the space capacity to be adjusted, the space of the space capacity to be adjusted is allocated from the shared resource pool to the target resource pool.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

Determine the target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range;

The storage space of each type of resource pool is adjusted so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

The amount of data requested by the storage request is less than the remaining space capacity in the target resource pool;

The data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

The local cache space is divided into resource pools, into multiple types of resource pools and shared resource pools.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

Determine the remaining space capacity of the adjusted target resource pool;

If the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, the data requested by the storage request is cached in the adjusted target resource pool;

If the remaining space of the adjusted target resource pool is less than the amount of data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

When a read request is received, the data requested by the read request is read from the cache space;

If the read fails, the data requested by the read request is read from the mechanical hard disk.

In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

Determine a target resource pool corresponding to the received storage request in the cache space;

When the target resource pool does not meet the preset storage conditions, the storage space of the target resource pool is adjusted according to the shared resource pool in the cache;

The data requested by the storage request is stored according to the adjusted target resource pool.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

Determine the space capacity to be adjusted based on the remaining space capacity in the target resource pool and the preset storage conditions;

Adjust the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

If the space capacity of the shared resource pool is less than the space capacity to be adjusted, the storage space of the target resource pool is adjusted according to the current request processing pressure of each type of resource pool;

If the space capacity of the shared resource pool is not less than the space capacity to be adjusted, the space of the space capacity to be adjusted is allocated from the shared resource pool to the target resource pool.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

Determine the target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range;

The storage space of each type of resource pool is adjusted so that the space capacity of each type of resource pool after adjustment reaches the target space capacity.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

The amount of data requested by the storage request is less than the remaining space capacity in the target resource pool;

The data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

The local cache space is divided into resource pools, into multiple types of resource pools and shared resource pools.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

Determine the remaining space capacity of the adjusted target resource pool;

If the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, the data requested by the storage request is cached in the adjusted target resource pool;

If the remaining space of the adjusted target resource pool is less than the amount of data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk.

In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

When a read request is received, the data requested by the read request is read from the cache space;

If the read fails, the data requested by the read request is read from the mechanical hard disk.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims (12)

1. A storage method, characterized in that the method comprises: Determine a target resource pool corresponding to the received storage request in the cache space; When the target resource pool does not meet the preset storage condition, adjusting the storage space of the target resource pool according to the shared resource pool in the cache; The data requested by the storage request is stored according to the adjusted target resource pool. 2. The method according to claim 1, wherein adjusting the storage space of the target resource pool according to the shared resource pool in the cache comprises: Determining the space capacity to be adjusted according to the remaining space capacity in the target resource pool and the preset storage condition; The storage space of the target resource pool is adjusted according to the space capacity of the shared resource pool and the space capacity to be adjusted. 3. The method according to claim 2, characterized in that the adjusting the storage space of the target resource pool according to the space capacity of the shared resource pool and the space capacity to be adjusted comprises: If the space capacity of the shared resource pool is less than the space capacity to be adjusted, adjusting the storage space of the target resource pool according to the current request processing pressure of each type of resource pool; If the space capacity of the shared resource pool is not less than the space capacity to be adjusted, the space of the space capacity to be adjusted is allocated from the shared resource pool to the target resource pool. 4. The method according to claim 3, characterized in that the adjusting the storage space of the target resource pool according to the current request processing pressure of each type of resource pool comprises: Determining target space capacity of each type of resource pool according to the current request processing pressure of each type of resource pool; the target space capacity is within a preset capacity range; The storage space of each type of resource pool is adjusted so that the space capacity of each type of resource pool after adjustment reaches the target space capacity. 5. The method according to any one of claims 1 to 4, characterized in that the preset storage condition includes any one of the following: The amount of data requested by the storage request is less than the remaining space capacity in the target resource pool; The data flushing efficiency of the target resource pool is greater than or equal to the cache efficiency. 6. The method according to any one of claims 1 to 4, characterized in that the method further comprises: The local cache space is divided into resource pools to obtain multiple types of resource pools and the shared resource pool. 7. The method according to any one of claims 1 to 4, characterized in that storing the data requested by the storage request according to the adjusted target resource pool comprises: Determining the remaining space capacity of the adjusted target resource pool; If the remaining space capacity of the adjusted target resource pool is not less than the data volume of the data requested by the storage request, caching the data requested by the storage request to the adjusted target resource pool; If the remaining space of the adjusted target resource pool is smaller than the amount of data requested by the storage request, the data requested by the storage request is stored in the mechanical hard disk. 8. The method according to any one of claims 1 to 4, characterized in that the method further comprises: When receiving a read request, reading the data requested by the read request from the cache space; If the read fails, the data requested by the read request is read from the mechanical hard disk. 9. A storage device, characterized in that the device comprises: A determination module, used to determine a target resource pool corresponding to the received storage request in the cache space; An adjustment module, configured to adjust the storage space of the target resource pool according to the shared resource pool in the cache when the target resource pool does not meet the preset storage condition; The storage module is used to store the data requested by the storage request according to the adjusted target resource pool. 10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 8 when executing the computer program. 11. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented. 12. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.