[go: up one dir, main page]

CN118897655A - Data request processing method, device, computer equipment and storage medium - Google Patents

Data request processing method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN118897655A
CN118897655A CN202411367369.6A CN202411367369A CN118897655A CN 118897655 A CN118897655 A CN 118897655A CN 202411367369 A CN202411367369 A CN 202411367369A CN 118897655 A CN118897655 A CN 118897655A
Authority
CN
China
Prior art keywords
data
cache
data block
request
cleared
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202411367369.6A
Other languages
Chinese (zh)
Other versions
CN118897655B (en
Inventor
王永刚
李大生
仇锋利
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Priority to CN202411367369.6A priority Critical patent/CN118897655B/en
Publication of CN118897655A publication Critical patent/CN118897655A/en
Application granted granted Critical
Publication of CN118897655B publication Critical patent/CN118897655B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/08Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0877Cache access modes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • G06F3/0641De-duplication techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656Data buffering arrangements

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

The invention relates to the technical field of computers, and discloses a data request processing method, a device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring the occupied water level of the cache; under the condition that the occupied water level of the cache is higher than a preset threshold value, determining a data block to be cleaned, and releasing the cache space occupied by the data block to be cleaned; and under the condition that the data request is received from the host, determining a target cache space from the idle cache space in the cache, storing the data corresponding to the data request into the target cache space, and responding to the data request. The problem that in the process of processing the data request, if the buffer space is full, the time delay for processing the data request is increased due to replacement of the existing data in the buffer is solved.

Description

数据请求处理方法、装置、计算机设备及存储介质Data request processing method, device, computer equipment and storage medium

技术领域Technical Field

本发明涉及计算机技术领域,具体涉及一种数据请求处理方法、装置、计算机设备及存储介质。The present invention relates to the field of computer technology, and in particular to a data request processing method, device, computer equipment and storage medium.

背景技术Background Art

在存储系统响应数据请求的过程中,将数据请求对应的IO(Input/Output,输入/输出)数据缓存至DRAM(Dynamic Random Access Memory,动态随机存取存储器)可以提升存储系统性能,例如:对于写请求,数据写入缓存就可以响应主机完成,写入缓存的数据在后台刷写到HDD(Hard Disk Drive,机械硬盘)或者SSD(Solid State Disk,固态硬盘)等相对于DRAM慢速的非易失介质上,降低写请求的响应时间。对于读请求,如果读取的数据在缓存中,就无需从HDD或者SSD等存储介质读取数据,可以直接从缓存中读取数据返回给主机,降低读请求响应时间。In the process of the storage system responding to data requests, caching the IO (Input/Output) data corresponding to the data request to DRAM (Dynamic Random Access Memory) can improve the performance of the storage system. For example, for a write request, the data written to the cache can be responded to by the host. The data written to the cache is flushed to non-volatile media such as HDD (Hard Disk Drive) or SSD (Solid State Disk) that are slower than DRAM in the background, reducing the response time of the write request. For a read request, if the read data is in the cache, there is no need to read the data from storage media such as HDD or SSD. The data can be directly read from the cache and returned to the host, reducing the response time of the read request.

然而,在处理数据请求的过程中,如果缓存空间已满,需要通过缓存置换算法,把缓存中已有的数据置换出取,再向缓存中存入新的数据,这些流程的耗时是包含在I/O流程的处理时间中的,而缓存空间有限,缓存空间满是存储系统使用过程中的常态,因此每一个涉及缓存数据的数据请求,在处理过程中都需要处理缓存替换的流程,导致处理数据请求的时延增加,在存储系统使用过程可以累积大量数据请求的处理时延。However, in the process of processing data requests, if the cache space is full, it is necessary to use the cache replacement algorithm to replace the existing data in the cache and then store new data in the cache. The time consumption of these processes is included in the processing time of the I/O process. The cache space is limited, and it is normal for the cache space to be full during the use of the storage system. Therefore, every data request involving cached data needs to handle the cache replacement process during the processing, which increases the delay in processing data requests. The processing delay of a large number of data requests can accumulate during the use of the storage system.

因此,相关技术存在处理数据请求的过程中,如果缓存空间已满,置换缓存中已有的数据会导致处理数据请求的时延增加的问题。Therefore, the related art has a problem that during the process of processing a data request, if the cache space is full, replacing the existing data in the cache will cause an increase in the delay of processing the data request.

发明内容Summary of the invention

有鉴于此,本发明提供了一种数据请求处理方法、装置、计算机设备及存储介质,以解决处理数据请求的过程中,如果缓存空间已满,置换缓存中已有的数据会导致处理数据请求的时延增加的问题。In view of this, the present invention provides a data request processing method, apparatus, computer equipment and storage medium to solve the problem that in the process of processing data requests, if the cache space is full, replacing the existing data in the cache will cause an increase in the delay in processing the data request.

第一方面,本发明提供了一种数据请求处理方法,包括:In a first aspect, the present invention provides a data request processing method, comprising:

获取缓存占用水位;Get the cache occupancy level;

在缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放待清理数据块占用的缓存空间;When the cache occupancy level is higher than a preset threshold, determine the data blocks to be cleaned up, and release the cache space occupied by the data blocks to be cleaned up;

在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求。When a data request is received from the host, a target cache space is determined from free cache space in the cache, data corresponding to the data request is saved in the target cache space, and the data request is responded to.

本实施例提供的数据请求处理方法,在缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放待清理数据块占用的缓存空间。使缓存替换的处理过程不包含在处理数据请求的流程中,并且缓存替换的处理时间也不包含在处理数据请求的时间内。并且,通过及时释放清理数据块占用的缓存空间,使处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间,可以降低数据请求处理的响应时间。降低了数据请求的处理时延,提高了存储系统读写性能。解决了处理数据请求的过程中,如果缓存空间已满,置换缓存中已有的数据会导致处理数据请求的时延增加的问题。The data request processing method provided in this embodiment determines the data blocks to be cleared and releases the cache space occupied by the data blocks to be cleared when the cache occupancy level is higher than a preset threshold. The cache replacement process is not included in the process of processing data requests, and the cache replacement processing time is not included in the time of processing data requests. In addition, by timely releasing the cache space occupied by the cleared data blocks, the free cache space in the cache can be directly used in the process of processing each data request, which can reduce the response time of data request processing. The processing delay of data requests is reduced, and the read and write performance of the storage system is improved. The problem that in the process of processing data requests, if the cache space is full, replacing the existing data in the cache will increase the delay in processing data requests is solved.

在一些可选的实施方式中,在获取缓存占用水位之前,方法还包括:In some optional implementations, before obtaining the cache occupancy watermark, the method further includes:

获取处理缓存替换任务的线程数;Get the number of threads that handle cache replacement tasks;

根据线程数,生成第一预设数量个数据块处理队列,其中,数据块处理队列用于确定并释放待清理数据块。A first preset number of data block processing queues are generated according to the number of threads, wherein the data block processing queues are used to determine and release data blocks to be cleared.

在本实施方式中,根据处理缓存替换任务的线程数,生成第一预设数量个数据块处理队列,利用数据块处理队列对数据块进行管理,确定并释放待清理数据块,使处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间,可以降低数据请求处理的响应时间。In this embodiment, a first preset number of data block processing queues are generated according to the number of threads processing cache replacement tasks, and the data block processing queues are used to manage the data blocks, determine and release the data blocks to be cleaned, so that the free cache space in the cache can be directly used in the process of processing each data request, which can reduce the response time of data request processing.

在一些可选的实施方式中,在生成第一预设数量个数据块处理队列之后,方法还包括:In some optional implementations, after generating a first preset number of data block processing queues, the method further includes:

确定缓存中数据块归属的逻辑卷的标识;Determine the identity of the logical volume to which the data block in the cache belongs;

根据逻辑卷的标识和第一预设数量,确定数据块对应的数据块处理队列,并将数据块放入对应的数据块处理队列。According to the identifier of the logical volume and the first preset number, a data block processing queue corresponding to the data block is determined, and the data block is put into the corresponding data block processing queue.

在本实施方式中,确定数据块对应的数据块处理队列,并将数据块放入对应的数据块处理队列,利用数据块处理队列对数据块进行管理,便于对数据块进行清理并释放数据块占用的缓存空间。In this embodiment, the data block processing queue corresponding to the data block is determined, and the data block is placed in the corresponding data block processing queue. The data block is managed by the data block processing queue to facilitate cleaning of the data block and release of the cache space occupied by the data block.

在一些可选的实施方式中,在将数据块放入对应的数据块处理队列之后,方法还包括:In some optional implementations, after placing the data block into the corresponding data block processing queue, the method further includes:

确定数据块的第一替换优先级;Determining a first replacement priority of the data block;

根据第一替换优先级,在第一预设数量个数据块处理队列中生成数据块的序号。According to the first replacement priority, sequence numbers of data blocks are generated in a first preset number of data block processing queues.

在本实施方式中,根据第一替换优先级,在第一预设数量个数据块处理队列中生成数据块的序号,便于对数据块进行管理,根据序号确定并释放待清理数据块吗,准确高效。In this embodiment, according to the first replacement priority, the sequence numbers of the data blocks are generated in the first preset number of data block processing queues, which is convenient for managing the data blocks, and the data blocks to be cleared are determined and released according to the sequence numbers, which is accurate and efficient.

在一些可选的实施方式中,在缓存占用水位高于预设阈值的情况下,确定待清理数据块,包括:In some optional implementations, when the cache occupancy level is higher than a preset threshold, determining the data block to be cleared includes:

在缓存占用水位高于第一预设阈值且低于第二预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第一预设值的数据块作为待清理数据块;When the cache occupancy level is higher than a first preset threshold and lower than a second preset threshold, data blocks with sequence numbers smaller than a first preset value in a first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第二预设阈值且低于第三预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第二预设值的数据块作为待清理数据块;When the cache occupancy level is higher than the second preset threshold and lower than the third preset threshold, data blocks with sequence numbers smaller than the second preset value in the first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第三预设阈值且低于第四预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第三预设值的数据块作为待清理数据块;When the cache occupancy level is higher than the third preset threshold and lower than the fourth preset threshold, data blocks with sequence numbers smaller than the third preset value in the first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第四预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第四预设值的数据块作为待清理数据块。When the cache occupancy level is higher than a fourth preset threshold, data blocks with sequence numbers less than a fourth preset value in the first preset number of data block processing queues are taken as data blocks to be cleared.

在本实施方式中,将缓存占用水位分别于第一预设阈值至第四预设阈值进行比较,根据比较结果确定数据块处理队列中待清理数据块的数量,修改缓存替换的速度,合理利用系统的计算资源确保处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间。In this embodiment, the cache occupancy level is compared with the first preset threshold to the fourth preset threshold, and the number of data blocks to be cleared in the data block processing queue is determined based on the comparison results, the speed of cache replacement is modified, and the computing resources of the system are reasonably utilized to ensure that the free cache space in the cache can be directly used during the processing of each data request.

在一些可选的实施方式中,在确定待清理数据块,并释放待清理数据块占用的缓存空间之后,方法还包括:In some optional implementations, after determining the data blocks to be cleared and releasing the cache space occupied by the data blocks to be cleared, the method further includes:

确定第一预设数量个数据块处理队列中剩余数据块的第二替换优先级;Determining a second replacement priority of remaining data blocks in the first preset number of data block processing queues;

根据第二替换优先级,更新剩余数据块的序号。According to the second replacement priority, the sequence numbers of the remaining data blocks are updated.

在一些可选的实施方式中,在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求,包括:In some optional implementations, when a data request is received from a host, determining a target cache space from free cache space in a cache, saving data corresponding to the data request to the target cache space, and responding to the data request include:

在数据请求为读请求的情况下,如果缓存中不存在读请求对应的数据块,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a read request, if the data block corresponding to the read request does not exist in the cache, the target cache space is determined from the free cache space in the cache;

从预设存储介质中获取读请求对应的数据,并将读请求对应的数据写入目标缓存空间,生成读请求对应的读请求数据块;Acquire data corresponding to the read request from a preset storage medium, and write the data corresponding to the read request into a target cache space to generate a read request data block corresponding to the read request;

将读请求数据块中的数据返回给主机。Returns the data in the read request data block to the host.

在本实施方式中,通过将缓存替换的处理过程与处理数据请求的流程进行分离,确保处理每个读请求过程中都能够直接使用缓存中空闲的缓存空间,降低了读请求的处理时延。In this embodiment, by separating the cache replacement process from the data request processing process, it is ensured that free cache space in the cache can be directly used in the process of processing each read request, thereby reducing the processing delay of the read request.

在一些可选的实施方式中,释放待清理数据块占用的缓存空间,包括:In some optional implementations, releasing the cache space occupied by the data block to be cleared includes:

在待清理数据块为读请求数据块的情况下,删去数据块处理队列中的待清理数据块;In the case where the data block to be cleaned is a read request data block, the data block to be cleaned in the data block processing queue is deleted;

利用预设算法调换数据块处理队列中剩余的数据块的位置。The positions of the remaining data blocks in the data block processing queue are swapped using a preset algorithm.

在一些可选的实施方式中,在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求,包括:In some optional implementations, when a data request is received from a host, determining a target cache space from free cache space in a cache, saving data corresponding to the data request to the target cache space, and responding to the data request include:

在数据请求为写请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a write request, determining a target cache space from free cache space in the cache;

将写请求对应的数据写入目标缓存空间,生成写请求对应的写请求数据块,并响应主机。Write the data corresponding to the write request into the target cache space, generate a write request data block corresponding to the write request, and respond to the host.

在本实施方式中,通过将缓存替换的处理过程与处理数据请求的流程进行分离,确保处理每个写请求过程中都能够直接使用缓存中空闲的缓存空间,降低了写请求的处理时延。In this embodiment, by separating the cache replacement process from the data request processing process, it is ensured that the free cache space in the cache can be directly used in the process of processing each write request, thereby reducing the processing delay of the write request.

在一些可选的实施方式中,释放待清理数据块占用的缓存空间,包括:In some optional implementations, releasing the cache space occupied by the data block to be cleared includes:

在待清理数据块为写请求数据块的情况下,判断待清理数据块的数据是否已写入预设存储介质;In the case where the data block to be cleared is a write request data block, determining whether the data of the data block to be cleared has been written into a preset storage medium;

如果已写入,删去数据块处理队列中的待清理数据块;If it has been written, delete the data block to be cleaned up in the data block processing queue;

如果未写入,将待清理数据块的数据写入预设存储介质,删去数据块处理队列中的待清理数据块。If not written, the data of the data block to be cleared is written into a preset storage medium, and the data block to be cleared in the data block processing queue is deleted.

在一些可选的实施方式中,获取缓存占用水位,包括:In some optional implementations, obtaining the cache occupancy level includes:

启用预设定时器,并在预设定时器的时长为预设时长时,调用定时器处理函数;Enable the preset timer, and call the timer processing function when the preset timer duration is the preset duration;

利用定时器处理函数,获取缓存占用水位。Use the timer processing function to obtain the cache occupancy level.

在本实施方式中,在预设定时器的时长为预设时长时,获取缓存占用水位,在确保及时获取缓存占用水位并启动缓存替换流程的同时,避免实时获取缓存占用水位占用过多计算资源。In this embodiment, when the preset timer duration is the preset duration, the cache occupancy level is obtained, ensuring that the cache occupancy level is obtained in time and the cache replacement process is started, while avoiding excessive computing resources from being occupied by obtaining the cache occupancy level in real time.

在一些可选的实施方式中,获取缓存占用水位,包括:In some optional implementations, obtaining the cache occupancy level includes:

获取缓存的已使用缓存空间和总缓存空间;Get the used cache space and total cache space of the cache;

将已使用缓存空间和总缓存空间的比值作为缓存占用水位。The ratio of used cache space to total cache space is used as the cache occupancy watermark.

第二方面,本发明提供了一种数据请求处理装置,包括:In a second aspect, the present invention provides a data request processing device, comprising:

获取模块,用于获取缓存占用水位;An acquisition module is used to obtain the cache occupancy level;

释放模块,用于在缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放待清理数据块占用的缓存空间;A release module is used to determine the data blocks to be cleaned up and release the cache space occupied by the data blocks to be cleaned up when the cache occupancy level is higher than a preset threshold;

处理模块,用于在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求。The processing module is used to determine the target cache space from the free cache space in the cache when receiving a data request from the host, save the data corresponding to the data request to the target cache space, and respond to the data request.

第三方面,本发明提供了一种计算机设备,包括:存储器和处理器,存储器和处理器之间互相通信连接,存储器中存储有计算机指令,处理器通过执行计算机指令,从而执行上述第一方面或其对应的任一实施方式的数据请求处理方法。In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data request processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

第四方面,本发明提供了一种计算机可读存储介质,该计算机可读存储介质上存储有计算机指令,计算机指令用于使计算机执行上述第一方面或其对应的任一实施方式的数据请求处理方法。In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data request processing method of the first aspect or any corresponding embodiment thereof.

第五方面,本发明提供了一种计算机程序产品,包括计算机指令,计算机指令用于使计算机执行上述第一方面或其对应的任一实施方式的数据请求处理方法。In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data request processing method of the first aspect or any corresponding embodiment thereof.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

为了更清楚地说明本发明具体实施方式或相关技术中的技术方案,下面将对具体实施方式或相关技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本发明的一些实施方式,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

图1是根据本发明实施例的利用传统方案处理读请求的流程图;1 is a flow chart of processing a read request using a conventional solution according to an embodiment of the present invention;

图2是根据本发明实施例的数据请求处理方法的流程示意图;FIG2 is a schematic flow chart of a method for processing a data request according to an embodiment of the present invention;

图3是根据本发明实施例的数据块处理队列的示意图;FIG3 is a schematic diagram of a data block processing queue according to an embodiment of the present invention;

图4是根据本发明实施例的处理读请求的流程示意图;FIG4 is a schematic diagram of a flow chart of processing a read request according to an embodiment of the present invention;

图5是根据本发明实施例的数据请求处理装置的结构框图;5 is a structural block diagram of a data request processing device according to an embodiment of the present invention;

图6是本发明实施例的计算机设备的硬件结构示意图。FIG. 6 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention.

具体实施方式DETAILED DESCRIPTION

为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

在存储系统中作为缓存使用的DRAM,比非易失存储设备例如:HDD、SSD等成本更高,所以在存储系统中配置的DRAM容量有限,无法把所有数据请求对应的数据都缓存到DRAM中,因此在处理数据请求的过程中,DRAM的存储空间用满后,如果需要缓存新的数据,就需要把DRAM中已有的数据替换出来用来缓存新的数据,当前常用的缓存替换算法有FIFO(First In First Out,先进先出)、LRU(Least Recently Used,最近最久未使用)、LFU(Least Frequently Used,最近最少使用)等。当前传统方案处理数据请求的流程如图1所示,以一个读请求处理流程为例,在一个读请求的处理过程,将读请求下发到缓存模块,查询是否能够命中缓存,如果能够命中的话,就从缓存读取数据,返回给主机,结束;如果不能命中缓存,就需要从后端数据盘读取数据,把读取的数据返回给缓存模块,判断缓存空间是否已满,如果否,则读取的数据放入缓存中,之后返回给主机,如果是,按照替换算法,失效部分数据,释放缓存空间,读取的数据放入缓存中,之后返回给主机,结束。同样,对于写请求,数据在写入缓存时,如果缓存空间已满,同样需要通过缓存置换算法把已有的数据置换出来,写入新的数据。DRAM used as cache in storage systems is more expensive than non-volatile storage devices such as HDD and SSD. Therefore, the DRAM capacity configured in the storage system is limited, and it is impossible to cache all data requests corresponding to the data in the DRAM. Therefore, in the process of processing data requests, when the DRAM storage space is full, if new data needs to be cached, it is necessary to replace the existing data in the DRAM to cache the new data. Currently, commonly used cache replacement algorithms include FIFO (First In First Out), LRU (Least Recently Used), LFU (Least Frequently Used), etc. The process of processing data requests in the current traditional solution is shown in Figure 1. Taking a read request processing process as an example, in the process of processing a read request, the read request is sent to the cache module to check whether it can hit the cache. If it can hit, the data is read from the cache and returned to the host. If it cannot hit the cache, it is necessary to read the data from the backend data disk and return the read data to the cache module to determine whether the cache space is full. If not, the read data is placed in the cache and then returned to the host. If so, according to the replacement algorithm, some data is invalidated to release the cache space. The read data is placed in the cache and then returned to the host. Similarly, for write requests, when data is written to the cache, if the cache space is full, it is also necessary to replace the existing data with the new data through the cache replacement algorithm.

在上述传统方案中,处理数据请求的过程,如果需要把数据放入缓存且缓存空间已满,需要通过缓存置换算法,把缓存中已有的数据置换出取,在存入新的数据,这个过程中涉及到缓存置换算法的处理,缓存资源的释放,为数据请求申请缓存资源等处理流程,这些处理流程的耗时是包含在缓存请求的处理时间中的,并且缓存空间有限,缓存空间满是存储系统使用过程中的常态,因此每一个涉及缓存数据的数据请求,在处理过程中都需要执行缓存替换的流程,导致延长每个数据请求的处理时延,导致系统性能降低。In the above-mentioned traditional scheme, in the process of processing data requests, if data needs to be put into the cache and the cache space is full, it is necessary to use the cache replacement algorithm to replace the existing data in the cache and then store the new data. This process involves the processing of the cache replacement algorithm, the release of cache resources, and the application of cache resources for data requests. The time consumption of these processing flows is included in the processing time of the cache request, and the cache space is limited. It is normal for the cache space to be full during the use of the storage system. Therefore, every data request involving cache data needs to execute the cache replacement process during the processing process, which prolongs the processing delay of each data request and causes the system performance to degrade.

基于上述内容,本发明实施例提供了一种数据请求处理方法,使缓存替换的处理过程不包含在处理数据请求的流程中,当缓存占用水位高于预设阈值时,确定待清理数据块,并释放待清理数据块占用的缓存空间,上述缓存替换的处理时间不包含在处理数据请求的时间内;并且,使处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间,可以降低数据请求处理的响应时间。以达到处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间,降低数据请求的处理时延,提高了存储系统读写性能的技术效果。Based on the above content, an embodiment of the present invention provides a data request processing method, so that the cache replacement processing process is not included in the process of processing data requests. When the cache occupancy level is higher than the preset threshold, the data block to be cleared is determined, and the cache space occupied by the data block to be cleared is released. The above cache replacement processing time is not included in the time of processing data requests; and in the process of processing each data request, the free cache space in the cache can be directly used, which can reduce the response time of data request processing. In this way, the free cache space in the cache can be directly used in the process of processing each data request, the processing delay of the data request is reduced, and the read and write performance of the storage system is improved.

根据本发明实施例,提供了一种数据请求处理实施例,需要说明的是,在附图的流程图示出的步骤可以在具有数据处理能力的计算机设备中执行,例如:电脑、服务器等,并且,虽然在流程图中示出了逻辑顺序,但是在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤。According to an embodiment of the present invention, a data request processing embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer device with data processing capabilities, such as a computer, a server, etc., and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

在本实施例中提供了一种数据请求处理方法,图2是根据本发明实施例的数据请求处理方法的流程图,如图2所示,该流程包括如下步骤:In this embodiment, a data request processing method is provided. FIG. 2 is a flow chart of the data request processing method according to an embodiment of the present invention. As shown in FIG. 2 , the flow chart includes the following steps:

步骤S201,获取缓存占用水位。Step S201, obtaining the cache occupancy level.

具体地,DRAM为存储系统中的缓存。获取缓存占用水位,缓存占用水位例如:DRAM中的已使用空间大小除以DRAM的总空间大小。通过缓存占用水位,可以确定DRAM的使用情况,确定DRAM是否有足够的空闲缓存空间供之后处理数据请求的过程使用。Specifically, DRAM is a cache in the storage system. The cache occupancy level is obtained. The cache occupancy level is, for example, the size of the used space in the DRAM divided by the total size of the DRAM. The cache occupancy level can be used to determine the usage of the DRAM and whether the DRAM has enough free cache space for the subsequent processing of data requests.

步骤S202,在缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放待清理数据块占用的缓存空间。Step S202: when the cache occupancy level is higher than a preset threshold, determine the data blocks to be cleared, and release the cache space occupied by the data blocks to be cleared.

具体地,预设阈值例如:70%、80%、90%等,具体数值根据实际需求设定。如果缓存占用水位高于预设阈值,表示DRAM中可供之后处理数据请求的过程使用的空闲缓存空间不足,需要进行缓存替换。Specifically, the preset thresholds are, for example, 70%, 80%, 90%, etc., and the specific values are set according to actual needs. If the cache occupancy level is higher than the preset threshold, it means that there is insufficient free cache space in the DRAM for subsequent processing of data requests, and cache replacement is required.

在DRAM中确定待清理数据块,并释放待清理数据块占用的缓存空间,需要说明的是,本实施例处理数据请求过程中,会将数据请求对应的数据以数据块的形式保存在DRAM中,便于高效地响应发送数据请求的主机。The data blocks to be cleaned are determined in the DRAM, and the cache space occupied by the data blocks to be cleaned is released. It should be noted that during the data request processing in this embodiment, the data corresponding to the data request will be stored in the DRAM in the form of data blocks, so as to efficiently respond to the host sending the data request.

另外,可以设置多个预设阈值,在缓存占用水位高于不同预设阈值时,采用不同的缓存替换的速度。In addition, multiple preset thresholds may be set, and when the cache occupancy level is higher than different preset thresholds, different cache replacement speeds may be used.

步骤S203,在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求。Step S203, when a data request is received from the host, a target cache space is determined from the free cache space in the cache, data corresponding to the data request is saved in the target cache space, and the data request is responded to.

具体地,数据请求例如:IO请求。在从主机接收到数据请求的情况下,包括写请求和读请求,从DRAM中空闲的缓存空间确定目标缓存空间,目标缓存空间的大小大于或等于数据请求的数据大小,并且,目标缓存空间最好是连续的一段空间,这样响应发送数据请求的主机时,仅用发起一次数据传输,提高响应速度。将数据请求对应的数据以数据块的形式保存到目标缓存空间,并将该数据块中的数据返回主机,实现响应数据请求。Specifically, a data request is, for example, an IO request. When a data request is received from a host, including a write request and a read request, a target cache space is determined from the free cache space in the DRAM. The size of the target cache space is greater than or equal to the data size of the data request, and the target cache space is preferably a continuous space, so that when responding to the host sending the data request, only one data transmission needs to be initiated, thereby improving the response speed. The data corresponding to the data request is saved in the target cache space in the form of a data block, and the data in the data block is returned to the host to achieve the response to the data request.

本实施例提供的数据请求处理方法,在缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放待清理数据块占用的缓存空间。使缓存替换的处理过程不包含在处理数据请求的流程中,并且缓存替换的处理时间也不包含在处理数据请求的时间内。并且,通过及时释放清理数据块占用的缓存空间,使处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间,可以降低数据请求处理的响应时间。降低了数据请求的处理时延,提高了存储系统读写性能。解决了处理数据请求的过程中,如果缓存空间已满,置换缓存中已有的数据会导致处理数据请求的时延增加的问题。The data request processing method provided in this embodiment determines the data blocks to be cleared and releases the cache space occupied by the data blocks to be cleared when the cache occupancy level is higher than a preset threshold. The cache replacement process is not included in the process of processing data requests, and the cache replacement processing time is not included in the time of processing data requests. In addition, by timely releasing the cache space occupied by the cleared data blocks, the free cache space in the cache can be directly used in the process of processing each data request, which can reduce the response time of data request processing. The processing delay of data requests is reduced, and the read and write performance of the storage system is improved. The problem that in the process of processing data requests, if the cache space is full, replacing the existing data in the cache will increase the delay in processing data requests is solved.

在一些可选的实施方式中,在获取缓存占用水位之前,方法还包括:In some optional implementations, before obtaining the cache occupancy watermark, the method further includes:

获取处理缓存替换任务的线程数;Get the number of threads that handle cache replacement tasks;

根据线程数,生成第一预设数量个数据块处理队列,其中,数据块处理队列用于确定并释放待清理数据块。A first preset number of data block processing queues are generated according to the number of threads, wherein the data block processing queues are used to determine and release data blocks to be cleared.

具体地,本实施例处理数据请求过程中,会将数据请求对应的数据以数据块的形式保存在DRAM中,便于高效地响应发送数据请求的主机。Specifically, in the process of processing a data request in this embodiment, the data corresponding to the data request is stored in the DRAM in the form of a data block, so as to efficiently respond to the host sending the data request.

本实施例将DRAM中的数据块保存在多个数据块处理队列中,便于对数据块进行管理,包括:确定并释放待清理数据块。因此,需要先生成第一预设数量个数据块处理队列,流程包括:获取处理缓存替换任务的线程数m。根据线程数,生成第一预设数量个数据块处理队列,第一预设数量可以等于线程数m,即有多少个线程处理缓存替换任务就有多少个双向队列,另外,如果实际使用过程中有需要,第一预设数量也可以为线程数m的整数倍,例如:第一预设数量为m、2m、3m等。This embodiment stores the data blocks in the DRAM in multiple data block processing queues to facilitate the management of the data blocks, including: determining and releasing the data blocks to be cleaned. Therefore, it is necessary to first generate a first preset number of data block processing queues, and the process includes: obtaining the number of threads m for processing cache replacement tasks. According to the number of threads, a first preset number of data block processing queues is generated. The first preset number can be equal to the number of threads m, that is, there are as many bidirectional queues as there are threads processing cache replacement tasks. In addition, if necessary during actual use, the first preset number can also be an integer multiple of the number of threads m, for example: the first preset number is m, 2m, 3m, etc.

以第一预设数量为线程数m为例,结合图3对本实施例进行说明。处理缓存替换任务的线程数有m个,对应创建m个双向队列作为数据块处理队列,如图3所示,创建包含任务队列0、任务队列1、…、任务队列m-1共m个任务队列,任务队列即数据块处理队列,每个任务队列中会存放多个缓存数据块。Taking the first preset number as the number of threads m as an example, this embodiment is described in conjunction with Figure 3. There are m threads processing cache replacement tasks, and m bidirectional queues are correspondingly created as data block processing queues. As shown in Figure 3, m task queues are created, including task queue 0, task queue 1, ..., task queue m-1, and a total of m task queues are created. The task queues are data block processing queues, and each task queue stores multiple cache data blocks.

在本实施方式中,根据处理缓存替换任务的线程数,生成第一预设数量个数据块处理队列,利用数据块处理队列对数据块进行管理,确定并释放待清理数据块,使处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间,可以降低数据请求处理的响应时间。In this embodiment, a first preset number of data block processing queues are generated according to the number of threads processing cache replacement tasks, and the data block processing queues are used to manage the data blocks, determine and release the data blocks to be cleaned, so that the free cache space in the cache can be directly used in the process of processing each data request, which can reduce the response time of data request processing.

在一些可选的实施方式中,在生成第一预设数量个数据块处理队列之后,方法还包括:In some optional implementations, after generating a first preset number of data block processing queues, the method further includes:

确定缓存中数据块归属的逻辑卷的标识;Determine the identity of the logical volume to which the data block in the cache belongs;

根据逻辑卷的标识和第一预设数量,确定数据块对应的数据块处理队列,并将数据块放入对应的数据块处理队列。According to the identifier of the logical volume and the first preset number, a data block processing queue corresponding to the data block is determined, and the data block is put into the corresponding data block processing queue.

具体地,确定缓存中数据块归属的逻辑卷(Logical Unit Number,LUN)的标识,其中,LUN为逻辑单元号,在存储系统中为逻辑卷。Specifically, an identifier of a logical volume (Logical Unit Number, LUN) to which the data block in the cache belongs is determined, wherein LUN is a logical unit number, which is a logical volume in a storage system.

确定缓存中数据块归属的逻辑卷的标识(LUN ID)。以第一预设数量为线程数m为例,缓存中数据块放入哪个队列根据缓存数据块归属的LUN(逻辑卷)来确定,计算方法可以为:队列ID=LUN ID % m,%表示求余计算。通过上述方法确定数据块对应的数据块处理队列ID,例如:即根据数据归属的LUN的编号对m取余即为队列编号,即LUN 0的数据放置在队列0中、LUN 1的数据放置在队列1中,…,LUN m-1数据放置在队列m-1中、LUN m的数据放置在队列0中,以此类推。如图3所示,缓存数据块0-1、缓存数据块0-2、…、缓存数据块0-n对应的数据块处理队列为任务队列0,将缓存数据块0-1、缓存数据块0-2、…、缓存数据块0-n放入任务队列0,其它任务队列如图3所示,此处不再赘述。Determine the identifier (LUN ID) of the logical volume to which the data block in the cache belongs. Taking the first preset number as the number of threads m as an example, which queue the data block in the cache is placed in is determined according to the LUN (logical volume) to which the cache data block belongs, and the calculation method can be: queue ID = LUN ID % m, % represents the remainder calculation. The data block processing queue ID corresponding to the data block is determined by the above method, for example: the remainder of m according to the number of the LUN to which the data belongs is the queue number, that is, the data of LUN 0 is placed in queue 0, the data of LUN 1 is placed in queue 1, ..., the data of LUN m-1 is placed in queue m-1, the data of LUN m is placed in queue 0, and so on. As shown in Figure 3, the data block processing queue corresponding to cache data blocks 0-1, cache data blocks 0-2, ..., cache data blocks 0-n is task queue 0, and cache data blocks 0-1, cache data blocks 0-2, ..., cache data blocks 0-n are placed in task queue 0. Other task queues are shown in Figure 3 and will not be repeated here.

根据数据块处理队列ID确定数据块对应的数据块处理队列,将数据块放入对应的数据块处理队列。The data block processing queue corresponding to the data block is determined according to the data block processing queue ID, and the data block is placed in the corresponding data block processing queue.

在本实施方式中,确定数据块对应的数据块处理队列,并将数据块放入对应的数据块处理队列,利用数据块处理队列对数据块进行管理,便于对数据块进行清理并释放数据块占用的缓存空间。In this embodiment, the data block processing queue corresponding to the data block is determined, and the data block is placed in the corresponding data block processing queue. The data block is managed by the data block processing queue to facilitate cleaning of the data block and release of the cache space occupied by the data block.

在一些可选的实施方式中,在将数据块放入对应的数据块处理队列之后,方法还包括:In some optional implementations, after placing the data block into the corresponding data block processing queue, the method further includes:

确定数据块的第一替换优先级;Determining a first replacement priority of the data block;

根据第一替换优先级,在第一预设数量个数据块处理队列中生成数据块的序号。According to the first replacement priority, sequence numbers of data blocks are generated in a first preset number of data block processing queues.

具体地,在每一个数据块处理队列中,根据对应的缓存替换算法,把优先被替换的数据块放在数据块处理队列的头部,其它数据块按照替换顺序依次存储在链表中。Specifically, in each data block processing queue, according to the corresponding cache replacement algorithm, the data block to be replaced first is placed at the head of the data block processing queue, and other data blocks are stored in a linked list in sequence according to the replacement order.

替换顺序可以根据数据块的第一替换优先级确定,例如:根据数据块加入数据块处理队列的时间确定每个数据块处理队列中数据块的第一替换优先级,加入时间越小,第一替换优先级越高;根据数据块中数据的使用频率、命中率确定数据块的第一替换优先级,在业务周期内,数据的使用频率越低、命中率越低的数据块,第一替换优先级越高。The replacement order can be determined according to the first replacement priority of the data block. For example, the first replacement priority of the data block in each data block processing queue is determined according to the time when the data block joins the data block processing queue. The smaller the joining time, the higher the first replacement priority; the first replacement priority of the data block is determined according to the usage frequency and hit rate of the data in the data block. During the business cycle, the lower the data usage frequency and the lower the hit rate of the data block, the higher the first replacement priority.

根据第一替换优先级,在第一预设数量个数据块处理队列中生成数据块的序号,其中,第一替换优先级越高,生成的序号越小越靠前,后续序号越小的数据块越容易被确定为待清理数据块;或者,反过来,第一替换优先级越高,生成的序号越大越靠后,后续序号越大的数据块越容易被确定为待清理数据块。According to the first replacement priority, serial numbers of data blocks are generated in a first preset number of data block processing queues, wherein the higher the first replacement priority, the smaller and more forward the generated serial numbers are, and the subsequent data blocks with smaller serial numbers are more likely to be determined as data blocks to be cleaned; or, conversely, the higher the first replacement priority, the larger and more forward the generated serial numbers are, and the subsequent data blocks with larger serial numbers are more likely to be determined as data blocks to be cleaned.

在本实施方式中,根据第一替换优先级,在第一预设数量个数据块处理队列中生成数据块的序号,便于对数据块进行管理,根据序号确定并释放待清理数据块吗,准确高效。In this embodiment, according to the first replacement priority, the sequence numbers of the data blocks are generated in the first preset number of data block processing queues, which is convenient for managing the data blocks, and the data blocks to be cleared are determined and released according to the sequence numbers, which is accurate and efficient.

在一些可选的实施方式中,在缓存占用水位高于预设阈值的情况下,确定待清理数据块,包括:In some optional implementations, when the cache occupancy level is higher than a preset threshold, determining the data block to be cleared includes:

在缓存占用水位高于第一预设阈值且低于第二预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第一预设值的数据块作为待清理数据块;When the cache occupancy level is higher than a first preset threshold and lower than a second preset threshold, data blocks with sequence numbers smaller than a first preset value in a first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第二预设阈值且低于第三预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第二预设值的数据块作为待清理数据块;When the cache occupancy level is higher than the second preset threshold and lower than the third preset threshold, data blocks with sequence numbers smaller than the second preset value in the first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第三预设阈值且低于第四预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第三预设值的数据块作为待清理数据块;When the cache occupancy level is higher than the third preset threshold and lower than the fourth preset threshold, data blocks with sequence numbers smaller than the third preset value in the first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第四预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第四预设值的数据块作为待清理数据块。When the cache occupancy level is higher than a fourth preset threshold, data blocks with sequence numbers less than a fourth preset value in the first preset number of data block processing queues are taken as data blocks to be cleared.

具体地,为了把缓存替换的过程不放入处理数据请求的过程中,因此把缓存替换的处理过程放置在后台执行,为了前台的处理数据请求的过程不会因为DRAM的缓存空间满而阻塞,因此后台的缓存替换任务不能在缓存空间满的时候才开始执行,因此设定一个预设阈值,在缓存占用水位高于该预设阈值时,就开始启动缓存的替换处理流程。Specifically, in order to prevent the cache replacement process from being included in the process of processing data requests, the cache replacement process is placed in the background for execution. In order to prevent the foreground process of processing data requests from being blocked due to the full DRAM cache space, the background cache replacement task cannot be started when the cache space is full. Therefore, a preset threshold is set. When the cache occupancy level is higher than the preset threshold, the cache replacement process is started.

本实施例通过设置不同的预设阈值,根据缓存占用水位超出的预设阈值动态地调整缓存替换速度。不同的预设阈值包括:第一预设阈值例如:70%;第二预设阈值例如:80%;第三预设阈值例如:90%;第四预设阈值例如:95%,上述预设值的具体数值可以根据实际需求调整,此处不作限制。以第一预设数量为线程数m为例。This embodiment dynamically adjusts the cache replacement speed according to the preset threshold value exceeded by the cache occupancy level by setting different preset threshold values. The different preset threshold values include: a first preset threshold value, for example, 70%; a second preset threshold value, for example, 80%; a third preset threshold value, for example, 90%; a fourth preset threshold value, for example, 95%. The specific values of the above preset values can be adjusted according to actual needs and are not limited here. Take the first preset number as the number of threads m as an example.

在缓存占用水位高于第一预设阈值且低于第二预设阈值的情况下,表示DRAM的缓存空间开始不足,需要开始进行缓存替换流程,m个用于处理缓存替换任务的线程同时处理m个数据块处理队列中的缓存数据,将每个数据块处理队列中序号小于第一预设值的数据块作为待清理数据块,例如:第一预设值为2,即将每个队列中序号为1的数据块作为待清理数据块,每个线程把对应数据块处理队列的首部的一个数据块作为待清理数据块从缓存中失效掉,释放待清理数据块的缓存空间,结束本次处理。When the cache occupancy level is higher than the first preset threshold and lower than the second preset threshold, it indicates that the DRAM cache space is insufficient and the cache replacement process needs to be started. The m threads for handling the cache replacement task simultaneously process the cache data in the m data block processing queues, and the data blocks with sequence numbers less than the first preset value in each data block processing queue are taken as data blocks to be cleaned. For example, the first preset value is 2, that is, the data block with sequence number 1 in each queue is taken as the data block to be cleaned. Each thread invalidates a data block at the head of the corresponding data block processing queue as the data block to be cleaned from the cache, releases the cache space of the data block to be cleaned, and ends this processing.

在缓存占用水位高于第二预设阈值且低于第三预设阈值的情况下,表示DRAM的缓存空间比较不足,需要快速进行缓存替换流程,m个用于处理缓存替换任务的线程同时处理m个数据块处理队列中的缓存数据,将每个数据块处理队列中序号小于第二预设值的数据块作为待清理数据块,例如:第二预设值为3,即将每个队列中序号为1和2的数据块作为待清理数据块,每个线程当此处理从对应的队列中失效队列前面的两个数据块,释放待清理数据块的缓存空间,结束本次处理。When the cache occupancy level is higher than the second preset threshold and lower than the third preset threshold, it indicates that the DRAM cache space is insufficient and a cache replacement process needs to be performed quickly. m threads for processing cache replacement tasks simultaneously process cache data in m data block processing queues, and use data blocks with sequence numbers less than the second preset value in each data block processing queue as data blocks to be cleaned. For example, the second preset value is 3, that is, data blocks with sequence numbers 1 and 2 in each queue are used as data blocks to be cleaned. When each thread processes this, it invalidates the first two data blocks in the queue from the corresponding queue, releases the cache space for the data blocks to be cleaned, and ends this processing.

在缓存占用水位高于第三预设阈值且低于第四预设阈值的情况下,表示DRAM的缓存空间非常不足,需要更快进行缓存替换流程,m个用于处理缓存替换任务的线程同时处理m个数据块处理队列中的缓存数据,将每个数据块处理队列中序号小于第三预设值的数据块作为待清理数据块,例如:第三预设值为5,即将每个队列中序号为1至4的数据块作为待清理数据块,每个线程当此处理从对应的队列中失效队列前面的四个数据块,释放待清理数据块的缓存空间,结束本次处理。When the cache occupancy level is higher than the third preset threshold and lower than the fourth preset threshold, it means that the DRAM cache space is very insufficient and the cache replacement process needs to be performed faster. m threads for processing cache replacement tasks simultaneously process the cache data in m data block processing queues, and use the data blocks with sequence numbers less than the third preset value in each data block processing queue as data blocks to be cleaned. For example, the third preset value is 5, that is, the data blocks with sequence numbers 1 to 4 in each queue are used as data blocks to be cleaned. When each thread processes this, it invalidates the first four data blocks in the queue from the corresponding queue, releases the cache space for the data blocks to be cleaned, and ends this processing.

在缓存占用水位高于第四预设阈值的情况下,表示DRAM的缓存空间严重不足,需要以最快速度进行缓存替换流程,m个用于处理缓存替换任务的线程同时处理m个数据块处理队列中的缓存数据,将每个数据块处理队列中序号小于第四预设值的数据块作为待清理数据块,例如:第四预设值为9,即将每个队列中序号为1至8的数据块作为待清理数据块,每个线程当此处理从对应的队列中失效队列前面的八个数据块,释放待清理数据块的缓存空间,结束本次处理。When the cache occupancy level is higher than the fourth preset threshold, it means that the DRAM cache space is seriously insufficient and the cache replacement process needs to be performed at the fastest speed. m threads for processing cache replacement tasks simultaneously process the cache data in m data block processing queues, and take the data blocks with sequence numbers less than the fourth preset value in each data block processing queue as data blocks to be cleaned. For example: the fourth preset value is 9, that is, the data blocks with sequence numbers 1 to 8 in each queue are taken as data blocks to be cleaned. When each thread processes this, it invalidates the first eight data blocks in the queue from the corresponding queue, releases the cache space of the data blocks to be cleaned, and ends this processing.

在本实施方式中,将缓存占用水位分别于第一预设阈值至第四预设阈值进行比较,根据比较结果确定数据块处理队列中待清理数据块的数量,修改缓存替换的速度,合理利用系统的计算资源确保处理每个数据请求过程中,都能够直接使用缓存中空闲的缓存空间。In this embodiment, the cache occupancy level is compared with the first preset threshold to the fourth preset threshold, and the number of data blocks to be cleared in the data block processing queue is determined based on the comparison results, the speed of cache replacement is modified, and the computing resources of the system are reasonably utilized to ensure that the free cache space in the cache can be directly used during the processing of each data request.

在一些可选的实施方式中,在释放待清理数据块的过程中,为了避免直接删除待清理数据块可能会影响后续用户的读写业务需求,可以先暂时锁定待清理数据块,并获取接下来一段时间的读写业务情况,如果不影响用户的读写业务需求,再删除待清理数据块,并释放待清理数据块的缓存空间。具体流程包括步骤A1至步骤A4。In some optional implementations, in the process of releasing the data blocks to be cleared, in order to avoid directly deleting the data blocks to be cleared which may affect the subsequent user's read and write service needs, the data blocks to be cleared can be temporarily locked first, and the read and write service conditions for the next period of time can be obtained. If the read and write service needs of the user are not affected, the data blocks to be cleared can be deleted and the cache space of the data blocks to be cleared can be released. The specific process includes steps A1 to A4.

步骤A1,将确定的待清理数据块锁定。Step A1: Lock the determined data block to be cleared.

具体地,将待清理数据块锁定后,无法读取待清理数据块也无法向待清理数据块中写入数据。Specifically, after the data block to be cleaned is locked, data cannot be read from the data block to be cleaned and data cannot be written into the data block to be cleaned.

步骤A2,判断在预设时间段内与用户对应的传输带宽是否下降。Step A2: determine whether the transmission bandwidth corresponding to the user decreases within a preset time period.

具体地,判断传输带宽是否下降允许有5%的误差,即5%以下的影响可以视为误差。如果传输带宽在预设时间段内没有下降,可以确定删除待清理数据块不会影响后续用户的读写业务需求,可以删除待清理数据块,并释放待清理数据块的缓存空间。Specifically, a 5% error is allowed for determining whether the transmission bandwidth has decreased, that is, an impact of less than 5% can be regarded as an error. If the transmission bandwidth has not decreased within the preset time period, it can be determined that deleting the data block to be cleared will not affect the subsequent user's read and write service needs, and the data block to be cleared can be deleted, and the cache space of the data block to be cleared can be released.

步骤A3,如果传输带宽在预设时间段内未下降,删去数据块处理队列中的待清理数据,并释放待清理数据的缓存空间。Step A3: if the transmission bandwidth does not decrease within the preset time period, the data to be cleared in the data block processing queue is deleted, and the cache space of the data to be cleared is released.

具体地,预设时间段例如两周,可以检测两周时间,如果传输带宽在预设时间段内未下降,确定用户的读写业务效率未受影响,删去数据块处理队列中的待清理数据,并释放待清理数据的缓存空间。需要说明的是,预设时间段可以根据实际需求设定,例如,可以更长或更短。Specifically, the preset time period is, for example, two weeks. If the transmission bandwidth does not decrease within the preset time period, it is determined that the user's read and write service efficiency is not affected, the data to be cleared in the data block processing queue is deleted, and the cache space of the data to be cleared is released. It should be noted that the preset time period can be set according to actual needs, for example, it can be longer or shorter.

步骤A4,如果传输带宽在预设时间段内下降,将待清理数据块解锁并重新确定待清理数据块。Step A4: if the transmission bandwidth decreases within a preset time period, the data blocks to be cleared are unlocked and the data blocks to be cleared are re-determined.

具体地,如果传输带宽在一段时间内下降比大于阈值,确定锁定待清理数据块会对用户的读写业务产生影响,将待清理数据块解锁。并循环检测其它数据块,重新确定待清理数据块,以此来回收冗余的缓存空间,使得缓存资源最大化被利用。Specifically, if the transmission bandwidth decreases by more than a threshold within a period of time, it is determined that locking the data block to be cleared will affect the user's read and write services, and the data block to be cleared is unlocked. Other data blocks are detected cyclically and the data block to be cleared is re-determined, so as to reclaim redundant cache space and maximize the utilization of cache resources.

在本实施方式中,先暂时锁定待清理数据块,并获取接下来一段时间的读写业务情况,如果不影响用户的读写业务需求,再删除待清理数据块,并释放待清理数据块的缓存空间,使得缓存资源最大化被利用。In this embodiment, the data block to be cleaned is temporarily locked first, and the read and write business conditions in the next period of time are obtained. If it does not affect the user's read and write business needs, the data block to be cleaned is deleted and the cache space of the data block to be cleaned is released to maximize the utilization of cache resources.

在一些可选的实施方式中,在确定待清理数据块,并释放待清理数据块占用的缓存空间之后,方法还包括:In some optional implementations, after determining the data blocks to be cleared and releasing the cache space occupied by the data blocks to be cleared, the method further includes:

确定第一预设数量个数据块处理队列中剩余数据块的第二替换优先级;Determining a second replacement priority of remaining data blocks in the first preset number of data block processing queues;

根据第二替换优先级,更新剩余数据块的序号。According to the second replacement priority, the sequence numbers of the remaining data blocks are updated.

具体地,在将待清理数据块在数据块处理队列中删除并释放待清理数据块占用的缓存空间之后,数据块处理队列中数据块的序号不连贯,例如:图3中任务队列1中,删除了缓存数据块1-1,此时任务队列1中数据块的序号从1-2开始,不利于后续确定待清理数据块。Specifically, after the data blocks to be cleaned are deleted from the data block processing queue and the cache space occupied by the data blocks to be cleaned is released, the sequence numbers of the data blocks in the data block processing queue are discontinuous. For example, in task queue 1 in Figure 3, cached data block 1-1 is deleted. At this time, the sequence numbers of the data blocks in task queue 1 start from 1-2, which is not conducive to the subsequent determination of the data blocks to be cleaned.

因此,重新确定第一预设数量个数据块处理队列中剩余数据块的第二替换优先级,根据第二替换优先级,更新剩余数据块的序号,例如:数据块的第二替换优先级与第一替换优先级相同,均是利用数据块加入数据块处理队列的时间生成的,直接将,如图3所示,任务队列1中,删除了缓存数据块1-1,此时任务队列1中数据块的序号从1-2开始,将序号1-2修改为1-1,将序号1-3修改为1-2,以此类推。如果数据块的第二替换优先级与第一替换优先级不相同,第一替换优先级是利用数据块加入数据块处理队列的时间生成的,第二替换优先级是利用数据块中数据的使用频率、命中率生成的,则直接根据第二替换优先级重新生成剩余数据块的序号。根据更新后的序号动态调换数据块处理队列中剩余数据块的位置。Therefore, the second replacement priority of the remaining data blocks in the first preset number of data block processing queues is re-determined, and the sequence numbers of the remaining data blocks are updated according to the second replacement priority. For example, the second replacement priority of the data block is the same as the first replacement priority, and both are generated using the time when the data block joins the data block processing queue. As shown in FIG3 , in task queue 1, cached data block 1-1 is deleted. At this time, the sequence numbers of the data blocks in task queue 1 start from 1-2, and the sequence number 1-2 is modified to 1-1, and the sequence number 1-3 is modified to 1-2, and so on. If the second replacement priority of the data block is different from the first replacement priority, the first replacement priority is generated using the time when the data block joins the data block processing queue, and the second replacement priority is generated using the usage frequency and hit rate of the data in the data block, then the sequence numbers of the remaining data blocks are directly regenerated according to the second replacement priority. The positions of the remaining data blocks in the data block processing queue are dynamically swapped according to the updated sequence numbers.

在一些可选的实施方式中,在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求,包括:In some optional implementations, when a data request is received from a host, determining a target cache space from free cache space in a cache, saving data corresponding to the data request to the target cache space, and responding to the data request include:

在数据请求为读请求的情况下,如果缓存中不存在读请求对应的数据块,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a read request, if the data block corresponding to the read request does not exist in the cache, the target cache space is determined from the free cache space in the cache;

从预设存储介质中获取读请求对应的数据,并将读请求对应的数据写入目标缓存空间,生成读请求对应的读请求数据块;Acquire data corresponding to the read request from a preset storage medium, and write the data corresponding to the read request into a target cache space to generate a read request data block corresponding to the read request;

将读请求数据块中的数据返回给主机。Returns the data in the read request data block to the host.

具体地,在本实施例中,处理读请求的流程中无需再处理缓存替换的处理逻辑,可以直接从DRAM中空闲的缓存空间确定目标缓存空间。Specifically, in this embodiment, there is no need to process the processing logic of cache replacement in the process of processing the read request, and the target cache space can be directly determined from the free cache space in the DRAM.

判断DRAM中是否存在读请求对应的数据块,即判断读请求是否命中缓存,如果存在,则确定读请求读取的数据在DRAM中,就无需再从HDD、SSD等存储介质读取读请求对应的数据,可以直接从DRAM中读取读请求对应的数据返回给主机,降低读请求响应时间。Determine whether there is a data block corresponding to the read request in DRAM, that is, determine whether the read request hits the cache. If so, it is determined that the data read by the read request is in DRAM, so there is no need to read the data corresponding to the read request from storage media such as HDD and SSD. The data corresponding to the read request can be directly read from DRAM and returned to the host, reducing the response time of the read request.

如果缓存中不存在读请求对应的数据块,确定读请求读取的数据不在DRAM中,从DRAM中空闲的缓存空间确定目标缓存空间。从预设存储介质中获取读请求对应的数据,预设存储介质例如:HDD、SSD等相对于DRAM慢速的非易失存储介质。获取到读请求对应的数据之后,将读取到的数据写入DRAM的目标缓存空间,并生成读请求对应的读请求数据块,便于后续请求能够命中缓存。最后,把读请求数据块中的数据返回给主机。If the data block corresponding to the read request does not exist in the cache, it is determined that the data read by the read request is not in the DRAM, and the target cache space is determined from the free cache space in the DRAM. The data corresponding to the read request is obtained from a preset storage medium, such as a non-volatile storage medium such as HDD or SSD that is slower than DRAM. After obtaining the data corresponding to the read request, the read data is written to the target cache space of the DRAM, and a read request data block corresponding to the read request is generated so that subsequent requests can hit the cache. Finally, the data in the read request data block is returned to the host.

上述过程如图4所示,将读请求下发到缓存模块,判断读请求是否命中缓存,如果是,则从缓存读取数据,返回给主机;如果否,从后端数据盘读取数据,把读取的数据返回给缓存模块,读取的数据放入缓存中,之后返回给主机,结束。The above process is shown in Figure 4. The read request is sent to the cache module to determine whether the read request hits the cache. If so, the data is read from the cache and returned to the host. If not, the data is read from the backend data disk and returned to the cache module. The read data is placed in the cache and then returned to the host, and the process ends.

在本实施方式中,通过将缓存替换的处理过程与处理数据请求的流程进行分离,确保处理每个读请求过程中都能够直接使用缓存中空闲的缓存空间,降低了读请求的处理时延。In this embodiment, by separating the cache replacement process from the data request processing process, it is ensured that free cache space in the cache can be directly used in the process of processing each read request, thereby reducing the processing delay of the read request.

在一些可选的实施方式中,释放待清理数据块占用的缓存空间,包括:In some optional implementations, releasing the cache space occupied by the data block to be cleared includes:

在待清理数据块为读请求数据块的情况下,删去数据块处理队列中的待清理数据块;In the case where the data block to be cleaned is a read request data block, the data block to be cleaned in the data block processing queue is deleted;

利用预设算法调换数据块处理队列中剩余的数据块的位置。The positions of the remaining data blocks in the data block processing queue are swapped using a preset algorithm.

具体地,在待清理数据块为读请求数据块的情况下,可以直接删去数据块处理队列中的待清理数据块。Specifically, when the data block to be cleared is a read request data block, the data block to be cleared in the data block processing queue can be directly deleted.

在将待清理数据块在数据块处理队列中删除并释放待清理数据块占用的缓存空间之后,数据块处理队列中数据块的序号不连贯,因此,利用预设算法动态调换数据块处理队列中剩余的数据块的位置,并更新剩余的数据块的序号,例如:利用上述第二替换优先级,更新剩余数据块的序号,并根据更新后的序号动态调换数据块处理队列中剩余的数据块的位置。After the data blocks to be cleared are deleted from the data block processing queue and the cache space occupied by the data blocks to be cleared is released, the sequence numbers of the data blocks in the data block processing queue are discontinuous. Therefore, a preset algorithm is used to dynamically swap the positions of the remaining data blocks in the data block processing queue, and the sequence numbers of the remaining data blocks are updated. For example, the above-mentioned second replacement priority is used to update the sequence numbers of the remaining data blocks, and the positions of the remaining data blocks in the data block processing queue are dynamically swapped according to the updated sequence numbers.

在一些可选的实施方式中,在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求,包括:In some optional implementations, when a data request is received from a host, determining a target cache space from free cache space in a cache, saving data corresponding to the data request to the target cache space, and responding to the data request include:

在数据请求为写请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a write request, determining a target cache space from free cache space in the cache;

将写请求对应的数据写入目标缓存空间,生成写请求对应的写请求数据块,并响应主机。Write the data corresponding to the write request into the target cache space, generate a write request data block corresponding to the write request, and respond to the host.

具体地,在数据请求为写请求的情况下,从DRAM中空闲的缓存空间确定目标缓存空间,目标缓存空间的大小大于或等于写请求的数据大小,并且,目标缓存空间最好是连续的一段空间,这样将写请求的数据写入目标缓存空间时,只需要发起一次数据传输,提高数据请求的处理速度。Specifically, when the data request is a write request, the target cache space is determined from the free cache space in the DRAM. The size of the target cache space is greater than or equal to the data size of the write request, and the target cache space is preferably a continuous space. In this way, when writing the data of the write request to the target cache space, only one data transfer needs to be initiated, thereby improving the processing speed of the data request.

对于写请求,将写请求的数据写入DRAM后就可以响应主机数据请求处理完成。写入DRAM的数据在后台刷写到HDD或者SSD等相对于DRAM慢速的非易失存储介质上,降低系统对写请求的响应时间。因此,将写请求对应的数据写入目标缓存空间,生成写请求对应的写请求数据块,并响应主机。For write requests, the data of the write request is written into DRAM and the host data request processing is completed. The data written into DRAM is flushed to non-volatile storage media such as HDD or SSD, which are slower than DRAM, in the background, reducing the system's response time to write requests. Therefore, the data corresponding to the write request is written into the target cache space, the write request data block corresponding to the write request is generated, and the host is responded to.

在本实施方式中,通过将缓存替换的处理过程与处理数据请求的流程进行分离,确保处理每个写请求过程中都能够直接使用缓存中空闲的缓存空间,降低了写请求的处理时延。In this embodiment, by separating the cache replacement process from the data request processing process, it is ensured that the free cache space in the cache can be directly used in the process of processing each write request, thereby reducing the processing delay of the write request.

在一些可选的实施方式中,释放待清理数据块占用的缓存空间,包括:In some optional implementations, releasing the cache space occupied by the data block to be cleared includes:

在待清理数据块为写请求数据块的情况下,判断待清理数据块的数据是否已写入预设存储介质;In the case where the data block to be cleared is a write request data block, determining whether the data of the data block to be cleared has been written into a preset storage medium;

如果已写入,删去数据块处理队列中的待清理数据块;If it has been written, delete the data block to be cleaned up in the data block processing queue;

如果未写入,将待清理数据块的数据写入预设存储介质,删去数据块处理队列中的待清理数据块。If not written, the data of the data block to be cleared is written into a preset storage medium, and the data block to be cleared in the data block processing queue is deleted.

具体地,写入DRAM的数据在后台刷写到预设存储介质上,预设存储介质例如:HDD、SSD等相对于DRAM慢速的非易失存储介质。后台刷写的流程与释放待清理数据块占用的缓存空间的流程同时进行,因此,可能出现某一写请求数据块被确认为待清理数据块时,该写请求数据块的数据还未刷写到HDD或者SSD等非易失存储介质上,如果直接删除该写请求数据块,会导致数据丢失。Specifically, the data written to DRAM is flushed to a preset storage medium in the background, such as a non-volatile storage medium such as HDD or SSD that is slower than DRAM. The process of background flushing is carried out simultaneously with the process of releasing the cache space occupied by the data block to be cleared. Therefore, when a certain write request data block is confirmed as a data block to be cleared, the data of the write request data block has not been flushed to a non-volatile storage medium such as HDD or SSD. If the write request data block is directly deleted, data loss will result.

判断待清理数据块的数据是否已写入预设存储介质,如果已写入,直接删去数据块处理队列中的待清理数据块,如果未写入,将待清理数据块的数据写入预设存储介质后,再删去数据块处理队列中的待清理数据块。Determine whether the data of the data block to be cleaned has been written into the preset storage medium. If so, directly delete the data block to be cleaned in the data block processing queue. If not, write the data of the data block to be cleaned into the preset storage medium and then delete the data block to be cleaned in the data block processing queue.

在一些可选的实施方式中,获取缓存占用水位,包括:In some optional implementations, obtaining the cache occupancy level includes:

启用预设定时器,并在预设定时器的时长为预设时长时,调用定时器处理函数;Enable the preset timer, and call the timer processing function when the preset timer duration is the preset duration;

利用定时器处理函数,获取缓存占用水位。Use the timer processing function to obtain the cache occupancy level.

具体地,如果实时获取缓存占用水位,过于占用系统的计算资源,因此,可以设置定时器,每间隔一段时间,获取一次缓存占用水位,判断是否需要进行缓存替换流程。Specifically, if the cache occupancy level is obtained in real time, it will occupy too much computing resources of the system. Therefore, a timer can be set to obtain the cache occupancy level once every certain period of time to determine whether a cache replacement process is needed.

启用预设定时器,预设定时器例如:一个100ms的定时器。预设时长例如:100ms。在预设定时器的时长为预设时长时,调用定时器处理函数,在定时器处理函数中,获取缓存占用水位,判断当前的缓存占用水位是否达到阈值,例如:70%,如果未达到,则什么都不做,如果达到了,则开始替换缓存中数据,即把对应的数据失效掉,释放对应的缓存空间。Enable the preset timer, for example, a 100ms timer. The preset duration is, for example, 100ms. When the preset timer duration is the preset duration, call the timer processing function. In the timer processing function, get the cache occupancy level and determine whether the current cache occupancy level has reached the threshold, for example, 70%. If it has not reached, do nothing. If it has reached, start replacing the data in the cache, that is, invalidate the corresponding data and release the corresponding cache space.

在本实施方式中,在预设定时器的时长为预设时长时,获取缓存占用水位,在确保及时获取缓存占用水位并启动缓存替换流程的同时,避免实时获取缓存占用水位占用过多计算资源。In this embodiment, when the preset timer duration is the preset duration, the cache occupancy level is obtained, ensuring that the cache occupancy level is obtained in time and the cache replacement process is started, while avoiding excessive computing resources from being occupied by obtaining the cache occupancy level in real time.

在一些可选的实施方式中,获取缓存占用水位,包括:In some optional implementations, obtaining the cache occupancy level includes:

获取缓存的已使用缓存空间和总缓存空间;Get the used cache space and total cache space of the cache;

将已使用缓存空间和总缓存空间的比值作为缓存占用水位。The ratio of used cache space to total cache space is used as the cache occupancy watermark.

具体地,为了把缓存替换的过程不放入处理数据请求的过程中,因此把缓存替换的处理过程放置在后台执行,为了前台的处理数据请求的过程不会因为DRAM的缓存空间满而阻塞,因此后台的缓存替换任务不能在缓存空间满的时候才开始执行,因此设定一个预设阈值,在缓存占用水位高于该预设阈值时,就开始启动缓存的替换处理流程。Specifically, in order to prevent the cache replacement process from being included in the process of processing data requests, the cache replacement process is placed in the background for execution. In order to prevent the foreground process of processing data requests from being blocked due to the full DRAM cache space, the background cache replacement task cannot be started when the cache space is full. Therefore, a preset threshold is set. When the cache occupancy level is higher than the preset threshold, the cache replacement process is started.

DRAM为存储系统中的缓存。获取缓存占用水位,缓存占用水位例如:DRAM的已使用缓存空间除以DRAM的总缓存空间,已使用缓存空间为DRAM中所有数据块的缓存空间之和。通过缓存占用水位,可以确定DRAM的使用情况,确定DRAM是否有足够的空闲缓存空间供之后处理数据请求的过程使用。DRAM is the cache in the storage system. Get the cache occupancy level. For example, the cache occupancy level is the used cache space of DRAM divided by the total cache space of DRAM. The used cache space is the sum of the cache spaces of all data blocks in DRAM. The cache occupancy level can be used to determine the usage of DRAM and whether DRAM has enough free cache space for the subsequent data request processing process.

在本实施例中还提供了一种数据请求处理装置,该装置用于实现上述实施例及优选实施方式,已经进行过说明的不再赘述。如以下所使用的,术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现,但是硬件,或者软件和硬件的组合的实现也是可能并被构想的。In this embodiment, a data request processing device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and/or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

本实施例提供一种数据请求处理装置,如图5所示,包括:This embodiment provides a data request processing device, as shown in FIG5 , including:

获取模块501,用于获取缓存占用水位;Acquisition module 501, used to acquire the cache occupancy level;

释放模块502,用于在缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放待清理数据块占用的缓存空间;A release module 502 is used to determine the data blocks to be cleared and release the cache space occupied by the data blocks to be cleared when the cache occupancy level is higher than a preset threshold;

处理模块503,用于在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求。The processing module 503 is used to determine the target cache space from the free cache space in the cache when receiving a data request from the host, save the data corresponding to the data request to the target cache space, and respond to the data request.

在一些可选的实施方式中,在获取缓存占用水位之前,该装置用于:In some optional implementations, before obtaining the cache occupancy water level, the device is used to:

获取处理缓存替换任务的线程数;Get the number of threads that handle cache replacement tasks;

根据线程数,生成第一预设数量个数据块处理队列,其中,数据块处理队列用于确定并释放待清理数据块。A first preset number of data block processing queues are generated according to the number of threads, wherein the data block processing queues are used to determine and release data blocks to be cleared.

在一些可选的实施方式中,在生成第一预设数量个数据块处理队列之后,该装置还用于:In some optional implementations, after generating a first preset number of data block processing queues, the apparatus is further configured to:

确定缓存中数据块归属的逻辑卷的标识;Determine the identity of the logical volume to which the data block in the cache belongs;

根据逻辑卷的标识和第一预设数量,确定数据块对应的数据块处理队列,并将数据块放入对应的数据块处理队列。According to the identifier of the logical volume and the first preset number, a data block processing queue corresponding to the data block is determined, and the data block is put into the corresponding data block processing queue.

在一些可选的实施方式中,在将数据块放入对应的数据块处理队列之后,该装置还用于:In some optional implementations, after placing the data block into the corresponding data block processing queue, the device is further configured to:

确定数据块的第一替换优先级;Determining a first replacement priority of the data block;

根据第一替换优先级,在第一预设数量个数据块处理队列中生成数据块的序号。According to the first replacement priority, sequence numbers of data blocks are generated in a first preset number of data block processing queues.

在一些可选的实施方式中,释放模块502在缓存占用水位高于预设阈值的情况下,确定待清理数据块,包括:In some optional implementations, when the cache occupancy level is higher than a preset threshold, the release module 502 determines the data block to be cleared, including:

在缓存占用水位高于第一预设阈值且低于第二预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第一预设值的数据块作为待清理数据块;When the cache occupancy level is higher than a first preset threshold and lower than a second preset threshold, data blocks with sequence numbers smaller than a first preset value in a first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第二预设阈值且低于第三预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第二预设值的数据块作为待清理数据块;When the cache occupancy level is higher than the second preset threshold and lower than the third preset threshold, data blocks with sequence numbers smaller than the second preset value in the first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第三预设阈值且低于第四预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第三预设值的数据块作为待清理数据块;When the cache occupancy level is higher than the third preset threshold and lower than the fourth preset threshold, data blocks with sequence numbers smaller than the third preset value in the first preset number of data block processing queues are used as data blocks to be cleared;

在缓存占用水位高于第四预设阈值的情况下,将第一预设数量个数据块处理队列中序号小于第四预设值的数据块作为待清理数据块。When the cache occupancy level is higher than a fourth preset threshold, data blocks with sequence numbers less than a fourth preset value in the first preset number of data block processing queues are taken as data blocks to be cleared.

在一些可选的实施方式中,在确定待清理数据块,并释放待清理数据块占用的缓存空间之后,释放模块502还用于:In some optional implementations, after determining the data blocks to be cleared and releasing the cache space occupied by the data blocks to be cleared, the release module 502 is further used to:

确定第一预设数量个数据块处理队列中剩余数据块的第二替换优先级;Determining a second replacement priority of remaining data blocks in the first preset number of data block processing queues;

根据第二替换优先级,更新剩余数据块的序号。According to the second replacement priority, the sequence numbers of the remaining data blocks are updated.

在一些可选的实施方式中,处理模块503在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求,包括:In some optional implementations, when the processing module 503 receives a data request from the host, it determines the target cache space from the free cache space in the cache, saves the data corresponding to the data request to the target cache space, and responds to the data request, including:

在数据请求为读请求的情况下,如果缓存中不存在读请求对应的数据块,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a read request, if the data block corresponding to the read request does not exist in the cache, the target cache space is determined from the free cache space in the cache;

从预设存储介质中获取读请求对应的数据,并将读请求对应的数据写入目标缓存空间,生成读请求对应的读请求数据块;Acquire data corresponding to the read request from a preset storage medium, and write the data corresponding to the read request into a target cache space to generate a read request data block corresponding to the read request;

将读请求数据块中的数据返回给主机。Returns the data in the read request data block to the host.

在一些可选的实施方式中,处理模块503释放待清理数据块占用的缓存空间,包括:In some optional implementations, the processing module 503 releases the cache space occupied by the data block to be cleared, including:

在待清理数据块为读请求数据块的情况下,删去数据块处理队列中的待清理数据块;In the case where the data block to be cleaned is a read request data block, the data block to be cleaned in the data block processing queue is deleted;

利用预设算法调换数据块处理队列中剩余的数据块的位置。The positions of the remaining data blocks in the data block processing queue are swapped using a preset algorithm.

在一些可选的实施方式中,处理模块503在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将数据请求对应的数据保存到目标缓存空间,并响应数据请求,包括:In some optional implementations, when the processing module 503 receives a data request from the host, it determines the target cache space from the free cache space in the cache, saves the data corresponding to the data request to the target cache space, and responds to the data request, including:

在数据请求为写请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a write request, determining a target cache space from free cache space in the cache;

将写请求对应的数据写入目标缓存空间,生成写请求对应的写请求数据块,并响应主机。Write the data corresponding to the write request into the target cache space, generate a write request data block corresponding to the write request, and respond to the host.

在一些可选的实施方式中,处理模块503释放待清理数据块占用的缓存空间,包括:In some optional implementations, the processing module 503 releases the cache space occupied by the data block to be cleared, including:

在待清理数据块为写请求数据块的情况下,判断待清理数据块的数据是否已写入预设存储介质;In the case where the data block to be cleared is a write request data block, determining whether the data of the data block to be cleared has been written into a preset storage medium;

如果已写入,删去数据块处理队列中的待清理数据块;If it has been written, delete the data block to be cleaned up in the data block processing queue;

如果未写入,将待清理数据块的数据写入预设存储介质,删去数据块处理队列中的待清理数据块。If not written, the data of the data block to be cleared is written into a preset storage medium, and the data block to be cleared in the data block processing queue is deleted.

在一些可选的实施方式中,获取模块501获取缓存占用水位,包括:In some optional implementations, the acquisition module 501 acquires the cache occupancy level, including:

启用预设定时器,并在预设定时器的时长为预设时长时,调用定时器处理函数;Enable the preset timer, and call the timer processing function when the preset timer duration is the preset duration;

利用定时器处理函数,获取缓存占用水位。Use the timer processing function to obtain the cache occupancy level.

在一些可选的实施方式中,获取模块501获取缓存占用水位,包括:In some optional implementations, the acquisition module 501 acquires the cache occupancy level, including:

获取缓存的已使用缓存空间和总缓存空间;Get the used cache space and total cache space of the cache;

将已使用缓存空间和总缓存空间的比值作为缓存占用水位。The ratio of used cache space to total cache space is used as the cache occupancy watermark.

上述各个模块和单元的更进一步的功能描述与上述对应实施例相同,在此不再赘述。The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

本实施例中的数据请求处理装置是以功能单元的形式来呈现,这里的单元是指ASIC(Application Specific Integrated Circuit,专用集成电路)电路,执行一个或多个软件或固定程序的处理器和存储器,和/或其他可以提供上述功能的器件。The data request processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and/or other devices that can provide the above functions.

本发明实施例还提供一种计算机设备,具有上述图5所示的数据请求处理装置。An embodiment of the present invention further provides a computer device having the data request processing device shown in FIG. 5 .

请参阅图6,图6是本发明可选实施例提供的一种计算机设备的结构示意图,如图6所示,该计算机设备包括:一个或多个处理器10、存储器20,以及用于连接各部件的接口,包括高速接口和低速接口。各个部件利用不同的总线互相通信连接,并且可以被安装在公共主板上或者根据需要以其它方式安装。处理器可以对在计算机设备内执行的指令进行处理,包括存储在存储器中或者存储器上以在外部输入/输出装置(诸如,耦合至接口的显示设备)上显示GUI的图形信息的指令。在一些可选的实施方式中,若需要,可以将多个处理器和/或多条总线与多个存储器和多个存储器一起使用。同样,可以连接多个计算机设备,各个设备提供部分必要的操作(例如,作为服务器阵列、一组刀片式服务器、或者多处理器系统)。图6中以一个处理器10为例。Please refer to Figure 6, which is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention. As shown in Figure 6, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input/output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and/or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). In Figure 6, a processor 10 is taken as an example.

处理器10可以是中央处理器,网络处理器或其组合。其中,处理器10还可以进一步包括集成电路,可编程逻辑器件或其组合。上述可编程逻辑器件可以是复杂可编程逻辑器件,现场可编程逻辑门阵列,通用阵列逻辑或其任意组合。The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

其中,存储器20存储有可由至少一个处理器10执行的指令,以使至少一个处理器10执行实现上述实施例示出的方法。The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

存储器20可以包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需要的应用程序;存储数据区可存储根据计算机设备的使用所创建的数据等。此外,存储器20可以包括高速随机存取存储器,还可以包括非瞬时存储器,例如至少一个磁盘存储器件、闪存器件、或其他非瞬时固态存储器件。在一些可选的实施方式中,存储器20可选包括相对于处理器10远程设置的存储器,这些远程存储器可以通过网络连接至该计算机设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

存储器20可以包括易失性存储器,例如,随机存取存储器;存储器也可以包括非易失性存储器,例如,快闪存储器,硬盘或固态硬盘;存储器20还可以包括上述种类的存储器的组合。The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

该计算机设备还包括通信接口30,用于该计算机设备与其他设备或通信网络通信。The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

本发明实施例还提供了一种计算机可读存储介质,上述根据本发明实施例的方法可在硬件、固件中实现,或者被实现为可记录在存储介质,或者被实现通过网络下载的原始存储在远程存储介质或非暂时机器可读存储介质中并将被存储在本地存储介质中的计算机代码,从而在此描述的方法可被存储在使用通用计算机、专用处理器或者可编程或专用硬件的存储介质上的这样的软件处理。其中,存储介质可为磁碟、光盘、只读存储记忆体、随机存储记忆体、快闪存储器、硬盘或固态硬盘等;进一步地,存储介质还可以包括上述种类的存储器的组合。可以理解,计算机、处理器、微处理器控制器或可编程硬件包括可存储或接收软件或计算机代码的存储组件,当软件或计算机代码被计算机、处理器或硬件访问且执行时,实现上述实施例示出的方法。The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

本发明的一部分可被应用为计算机程序产品,例如计算机程序指令,当其被计算机执行时,通过该计算机的操作,可以调用或提供根据本发明的方法和/或技术方案。本领域技术人员应能理解,计算机程序指令在计算机可读介质中的存在形式包括但不限于源文件、可执行文件、安装包文件等,相应地,计算机程序指令被计算机执行的方式包括但不限于:该计算机直接执行该指令,或者该计算机编译该指令后再执行对应的编译后程序,或者该计算机读取并执行该指令,或者该计算机读取并安装该指令后再执行对应的安装后程序。在此,计算机可读介质可以是可供计算机访问的任意可用的计算机可读存储介质或通信介质。A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and/or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

虽然结合附图描述了本发明的实施例,但是本领域技术人员可以在不脱离本发明的精神和范围的情况下做出各种修改和变型,这样的修改和变型均落入本申请所限定的范围之内。Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined in this application.

Claims (15)

1.一种数据请求处理方法,其特征在于,所述方法包括:1. A data request processing method, characterized in that the method comprises: 获取缓存占用水位;Get the cache occupancy level; 在所述缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放所述待清理数据块占用的缓存空间;When the cache occupancy level is higher than a preset threshold, determining a data block to be cleared, and releasing the cache space occupied by the data block to be cleared; 在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将所述数据请求对应的数据保存到所述目标缓存空间,并响应所述数据请求。In case of receiving a data request from the host, a target cache space is determined from free cache space in the cache, data corresponding to the data request is saved in the target cache space, and the data request is responded to. 2.根据权利要求1所述的方法,其特征在于,在所述获取缓存占用水位之前,所述方法还包括:2. The method according to claim 1, characterized in that before obtaining the cache occupancy watermark, the method further comprises: 获取处理缓存替换任务的线程数;Get the number of threads that handle cache replacement tasks; 根据所述线程数,生成第一预设数量个数据块处理队列,其中,所述数据块处理队列用于确定并释放所述待清理数据块。A first preset number of data block processing queues are generated according to the number of threads, wherein the data block processing queues are used to determine and release the data blocks to be cleared. 3.根据权利要求2所述的方法,其特征在于,在所述生成第一预设数量个数据块处理队列之后,所述方法还包括:3. The method according to claim 2, characterized in that after generating the first preset number of data block processing queues, the method further comprises: 确定所述缓存中数据块归属的逻辑卷的标识;Determining an identifier of a logical volume to which the data block in the cache belongs; 根据所述逻辑卷的标识和所述第一预设数量,确定所述数据块对应的数据块处理队列,并将所述数据块放入对应的数据块处理队列。According to the identifier of the logical volume and the first preset number, a data block processing queue corresponding to the data block is determined, and the data block is put into the corresponding data block processing queue. 4.根据权利要求3所述的方法,其特征在于,在所述将所述数据块放入对应的数据块处理队列之后,所述方法还包括:4. The method according to claim 3, characterized in that after placing the data block into the corresponding data block processing queue, the method further comprises: 确定所述数据块的第一替换优先级;determining a first replacement priority of the data block; 根据所述第一替换优先级,在所述第一预设数量个数据块处理队列中生成所述数据块的序号。According to the first replacement priority, sequence numbers of the data blocks are generated in the first preset number of data block processing queues. 5.根据权利要求4所述的方法,其特征在于,所述在所述缓存占用水位高于预设阈值的情况下,确定待清理数据块,包括:5. The method according to claim 4, characterized in that when the cache occupancy level is higher than a preset threshold, determining the data block to be cleared comprises: 在所述缓存占用水位高于第一预设阈值且低于第二预设阈值的情况下,将所述第一预设数量个数据块处理队列中序号小于第一预设值的数据块作为所述待清理数据块;When the cache occupancy level is higher than a first preset threshold and lower than a second preset threshold, data blocks with sequence numbers smaller than a first preset value in the first preset number of data block processing queues are used as the data blocks to be cleared; 在所述缓存占用水位高于第二预设阈值且低于第三预设阈值的情况下,将所述第一预设数量个数据块处理队列中序号小于第二预设值的数据块作为所述待清理数据块;When the cache occupancy level is higher than the second preset threshold and lower than the third preset threshold, the data blocks with sequence numbers smaller than the second preset value in the first preset number of data block processing queues are used as the data blocks to be cleared; 在所述缓存占用水位高于第三预设阈值且低于第四预设阈值的情况下,将所述第一预设数量个数据块处理队列中序号小于第三预设值的数据块作为所述待清理数据块;When the cache occupancy level is higher than a third preset threshold and lower than a fourth preset threshold, the data blocks with sequence numbers smaller than the third preset value in the first preset number of data block processing queues are used as the data blocks to be cleared; 在所述缓存占用水位高于所述第四预设阈值的情况下,将所述第一预设数量个数据块处理队列中序号小于第四预设值的数据块作为所述待清理数据块。When the cache occupancy level is higher than the fourth preset threshold, data blocks with sequence numbers less than a fourth preset value in the first preset number of data block processing queues are used as the data blocks to be cleared. 6.根据权利要求4所述的方法,其特征在于,在所述确定待清理数据块,并释放所述待清理数据块占用的缓存空间之后,所述方法还包括:6. The method according to claim 4, characterized in that after determining the data blocks to be cleared and releasing the cache space occupied by the data blocks to be cleared, the method further comprises: 确定所述第一预设数量个数据块处理队列中剩余数据块的第二替换优先级;Determining a second replacement priority of remaining data blocks in the first preset number of data block processing queues; 根据所述第二替换优先级,更新所述剩余数据块的序号。According to the second replacement priority, the sequence numbers of the remaining data blocks are updated. 7.根据权利要求2所述的方法,其特征在于,所述在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将所述数据请求对应的数据保存到所述目标缓存空间,并响应所述数据请求,包括:7. The method according to claim 2, wherein when a data request is received from a host, determining a target cache space from free cache space in a cache, saving data corresponding to the data request to the target cache space, and responding to the data request comprises: 在所述数据请求为读请求的情况下,如果所述缓存中不存在所述读请求对应的数据块,从缓存中空闲的缓存空间确定所述目标缓存空间;In the case where the data request is a read request, if the data block corresponding to the read request does not exist in the cache, determining the target cache space from free cache space in the cache; 从预设存储介质中获取所述读请求对应的数据,并将所述读请求对应的数据写入所述目标缓存空间,生成所述读请求对应的读请求数据块;Acquire data corresponding to the read request from a preset storage medium, and write the data corresponding to the read request into the target cache space to generate a read request data block corresponding to the read request; 将所述读请求数据块中的数据返回给所述主机。The data in the read request data block is returned to the host. 8.根据权利要求7所述的方法,其特征在于,所述释放所述待清理数据块占用的缓存空间,包括:8. The method according to claim 7, wherein releasing the cache space occupied by the data block to be cleared comprises: 在所述待清理数据块为所述读请求数据块的情况下,删去所述数据块处理队列中的所述待清理数据块;In the case where the data block to be cleared is the read request data block, deleting the data block to be cleared in the data block processing queue; 利用预设算法调换所述数据块处理队列中剩余的数据块的位置。The positions of the remaining data blocks in the data block processing queue are swapped using a preset algorithm. 9.根据权利要求2所述的方法,其特征在于,所述在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将所述数据请求对应的数据保存到所述目标缓存空间,并响应所述数据请求,包括:9. The method according to claim 2, wherein when a data request is received from a host, determining a target cache space from free cache space in a cache, saving data corresponding to the data request to the target cache space, and responding to the data request comprises: 在所述数据请求为写请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间;In the case where the data request is a write request, determining a target cache space from free cache space in the cache; 将所述写请求对应的数据写入所述目标缓存空间,生成所述写请求对应的写请求数据块,并响应所述主机。The data corresponding to the write request is written into the target cache space, a write request data block corresponding to the write request is generated, and the host is responded to. 10.根据权利要求9所述的方法,其特征在于,所述释放所述待清理数据块占用的缓存空间,包括:10. The method according to claim 9, wherein releasing the cache space occupied by the data block to be cleared comprises: 在所述待清理数据块为所述写请求数据块的情况下,判断所述待清理数据块的数据是否已写入预设存储介质;In the case where the data block to be cleared is the write request data block, determining whether the data of the data block to be cleared has been written into a preset storage medium; 如果已写入,删去所述数据块处理队列中的所述待清理数据块;If it has been written, deleting the data block to be cleared in the data block processing queue; 如果未写入,将所述待清理数据块的数据写入预设存储介质,删去所述数据块处理队列中的所述待清理数据块。If not written, the data of the data block to be cleared is written into a preset storage medium, and the data block to be cleared in the data block processing queue is deleted. 11.根据权利要求1所述的方法,其特征在于,所述获取缓存占用水位,包括:11. The method according to claim 1, wherein obtaining the cache occupancy level comprises: 启用预设定时器,并在所述预设定时器的时长为预设时长时,调用定时器处理函数;Enable a preset timer, and when the duration of the preset timer is the preset duration, call a timer processing function; 利用所述定时器处理函数,获取缓存占用水位。The timer processing function is used to obtain the cache occupancy level. 12.根据权利要求1或11所述的方法,其特征在于,所述获取缓存占用水位,包括:12. The method according to claim 1 or 11, characterized in that the obtaining of the cache occupancy level comprises: 获取所述缓存的已使用缓存空间和总缓存空间;Obtaining the used cache space and the total cache space of the cache; 将所述已使用缓存空间和所述总缓存空间的比值作为所述缓存占用水位。The ratio of the used cache space to the total cache space is used as the cache occupancy watermark. 13.一种数据请求处理装置,其特征在于,所述装置包括:13. A data request processing device, characterized in that the device comprises: 获取模块,用于获取缓存占用水位;An acquisition module is used to obtain the cache occupancy level; 释放模块,用于在所述缓存占用水位高于预设阈值的情况下,确定待清理数据块,并释放所述待清理数据块占用的缓存空间;A release module, used to determine the data blocks to be cleared and release the cache space occupied by the data blocks to be cleared when the cache occupancy level is higher than a preset threshold; 处理模块,用于在从主机接收到数据请求的情况下,从缓存中空闲的缓存空间确定目标缓存空间,将所述数据请求对应的数据保存到所述目标缓存空间,并响应所述数据请求。The processing module is used to determine the target cache space from the free cache space in the cache when receiving a data request from the host, save the data corresponding to the data request to the target cache space, and respond to the data request. 14.一种计算机设备,其特征在于,包括:14. A computer device, comprising: 存储器和处理器,所述存储器和所述处理器之间互相通信连接,所述存储器中存储有计算机指令,所述处理器通过执行所述计算机指令,从而执行权利要求1至12中任一项所述的数据请求处理方法。A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data request processing method according to any one of claims 1 to 12 by executing the computer instructions. 15.一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机指令,所述计算机指令用于使计算机执行权利要求1至12中任一项所述的数据请求处理方法。15. A computer-readable storage medium, characterized in that computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to enable a computer to execute the data request processing method according to any one of claims 1 to 12.
CN202411367369.6A 2024-09-29 2024-09-29 Data request processing method, device, computer equipment and storage medium Active CN118897655B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411367369.6A CN118897655B (en) 2024-09-29 2024-09-29 Data request processing method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411367369.6A CN118897655B (en) 2024-09-29 2024-09-29 Data request processing method, device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN118897655A true CN118897655A (en) 2024-11-05
CN118897655B CN118897655B (en) 2025-01-28

Family

ID=93266633

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411367369.6A Active CN118897655B (en) 2024-09-29 2024-09-29 Data request processing method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN118897655B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004006101A1 (en) * 2002-07-04 2004-01-15 Sony Corporation Cache device, cache data management method, and computer program
CN112015343A (en) * 2020-08-27 2020-12-01 杭州宏杉科技股份有限公司 Cache space management method and device of storage volume and electronic equipment
WO2022247070A1 (en) * 2021-05-24 2022-12-01 北京工业大学 High-performance-oriented intelligent cache replacement policy adaptive to prefetching
CN115543938A (en) * 2021-06-30 2022-12-30 腾讯科技(深圳)有限公司 Data processing method and device, electronic equipment and storage medium
CN116366657A (en) * 2023-05-31 2023-06-30 天翼云科技有限公司 Data request scheduling method and system of cache server

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004006101A1 (en) * 2002-07-04 2004-01-15 Sony Corporation Cache device, cache data management method, and computer program
CN112015343A (en) * 2020-08-27 2020-12-01 杭州宏杉科技股份有限公司 Cache space management method and device of storage volume and electronic equipment
WO2022247070A1 (en) * 2021-05-24 2022-12-01 北京工业大学 High-performance-oriented intelligent cache replacement policy adaptive to prefetching
CN115543938A (en) * 2021-06-30 2022-12-30 腾讯科技(深圳)有限公司 Data processing method and device, electronic equipment and storage medium
CN116366657A (en) * 2023-05-31 2023-06-30 天翼云科技有限公司 Data request scheduling method and system of cache server

Also Published As

Publication number Publication date
CN118897655B (en) 2025-01-28

Similar Documents

Publication Publication Date Title
US7024512B1 (en) Compression store free-space management
US10860494B2 (en) Flushing pages from solid-state storage device
US10241722B1 (en) Proactive scheduling of background operations for solid state drives
CN106547476B (en) Method and apparatus for data storage system
US10897517B2 (en) Distributed cache live migration
WO2014000300A1 (en) Data buffer device, data storage system and method
JP5840594B2 (en) Method and apparatus for memory management by storage system
CN112631520B (en) Distributed block storage system, method, apparatus, device and medium
CN103019962A (en) Data cache processing method, device and system
CN107632784A (en) The caching method of a kind of storage medium and distributed memory system, device and equipment
CN107615254A (en) Cache Architecture and Algorithms for Hybrid Object Storage Devices
US8019939B2 (en) Detecting data mining processes to increase caching efficiency
JP2018531471A6 (en) Distributed cache live migration
US10705977B2 (en) Method of dirty cache line eviction
CN104375954A (en) Implementing dynamic cache enabling and disabling based upon workload
WO2017016380A1 (en) Advance cache allocator
CN116795736A (en) Data pre-reading method, device, electronic equipment and storage medium
CN117311621A (en) Cache disk space allocation method and device, computer equipment and storage medium
WO2024108825A1 (en) Memory backup acceleration method, apparatus and device, and non-volatile readable storage medium
JP7011156B2 (en) Storage controller and program
CN118312102A (en) IO request processing method, device, storage device and storage medium
JP2023536237A (en) Acquiring cache resources for an expected write to a track in the writeset after the cache resources for the tracks in the writeset have been freed
US9164904B2 (en) Accessing remote memory on a memory blade
CN118897655A (en) Data request processing method, device, computer equipment and storage medium
CN113093994A (en) Data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant