CN108182151A - Sequential flow caches method, apparatus, storage medium and the terminal device with detection - Google Patents
Sequential flow caches method, apparatus, storage medium and the terminal device with detection Download PDFInfo
- Publication number
- CN108182151A CN108182151A CN201810068469.7A CN201810068469A CN108182151A CN 108182151 A CN108182151 A CN 108182151A CN 201810068469 A CN201810068469 A CN 201810068469A CN 108182151 A CN108182151 A CN 108182151A
- Authority
- CN
- China
- Prior art keywords
- data
- sequential flow
- sequential
- address
- caching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
- G06F12/0615—Address space extension
- G06F12/063—Address space extension for I/O modules, e.g. memory mapped I/O
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of sequential stream detection and method, apparatus, storage medium and the terminal device of caching, the method includes：The access request of I/O data is received, and records the address information of the I/O data in the skip list of address；According to described address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；Judge whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value；And if so, be recorded in the attribute information in the fixed length array according to described address skip list and the sequential flow, to the I/O data of the sequential flow that is currently received, piecemeal caches on the basis of IO queue depths.Using the present invention, sequence flow data can be efficiently identified and piecemeal is cached on the basis of IO queue depths.
Description
Technical field
The present invention relates to the method, apparatus of field of computer technology more particularly to a kind of sequential flow caching with detection, storages
Medium and terminal device.
Background technology
Within the storage system, IO (Input/Output, input/output) scheduler module needs pre-read current IO,
And targetedly handled and optimized, to using disk performance, IO performances be allowed to be optimal as possible.Among these to sequence
The optimization of IO is extremely important, particularly following several scenes：
(1) to a large amount of small optimization for being sequentially written in IO；
(2) optimization of IO is read in a large amount of small sequence；
(3) to the optimization for being sequentially written in IO of non-address alignment.
Wherein, by taking scene 1 as an example, for a data volume it is larger write IO, may be because of service logic, operating system
Or a variety of causes such as processing in file system are split as the soft small letter IO of multiple data volumes, but if disk directly one by one
It if handling write request, can cause to write the amplification of IO, and reduce disk performance.Illustrated with scene 2, for a data volume compared with
IO is greatly read, may be split as because of a variety of causes such as the processing in service logic, operating system or file system
Multiple smaller IO of data volume if disk IO smaller to each data volume directly handles reading request, can cause reading IO to put
Greatly, disk performance is reduced.It is illustrated with scene 3, for continuously writing IO, can understood because of service logic, operating system or file
The a variety of causes such as the processing in system are misaligned, then actually write by cutting again if writing IO after cutting with disk block address
During IO, disk can read the data of head and the tail block first, and after merging in memory with the data of write-in, be then written to disk, this
Sample can lead to additional reading IO, and cause and write IO amplifications, and reduce disk performance.These types of scene is required for scheduler module energy
Under the scene of various complexity particularly in the case where Multi-Client Concurrency Access and a large amount of sequences and random IO mix, identification
Go out the IO sequential flows of multipath concurrence.A kind of therefore it provides general method that multipath concurrence sequence IO is detected and is dispatched
It is very important.And there are following defects for the scheme of detection IO sequential flows that the prior art provides：
1st, the purposes of the identification of the sequential flow IO of the prior art is mainly that IO is pre-read, and there is no the write-ins of optimization write-in IO
Journey；
2nd, IO queue depths is not supported to detect, reason is mainly that the purposes of existing technical solution is to pre-read, but for writing
IO if IO depth is (IO is that sequence is initiated rather than parallel) if 1, cannot actually be cached, it is no it will cause
IO blocks；
3rd, can not detection ordering stream IO when terminate, be that can cause to pre-read in vain for reading IO, for writing IO
It is that tail portion IO is caused to block or generate larger delay；
4th, most of sequential flow IO recognition methods is all based on the piecemeal of regular length (primarily to being looked by address Hash
Look for), then this type method is to address space overlap, non-critical dull continuous (such as continuous I/O sequence majority IO addresses list in address
Adjust a small amount of IO addresses inverted sequence in the case of being incremented by), the endless Total continuity in address (being spaced among such as part IO) situations such as write
Enter to handle existing defects.
Invention content
The embodiment of the present invention provides a kind of method, apparatus of speech polling, storage medium and terminal device, to solve or delay
Solve Yi Shang technical problem of the prior art.
In a first aspect, an embodiment of the present invention provides a kind of sequential stream detection and the method for caching, including：
The access request of I/O data is received, and records the address information of the I/O data in the skip list of address；
According to described address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record in
Fixed length array；Wherein, the attribute information includes the IO quantity of the I/O data of the sequential flow being currently received；
Judge whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value；And
If so, the attribute information in the fixed length array is recorded according to described address skip list and the sequential flow, to working as
Before the I/O data of the sequential flow that the receives piecemeal on the basis of IO queue depths cached.
Further, described address information includes the offset address of the I/O data, and the attribute information further includes described
The offset address of renewal time and the sequential flow of the attribute information of sequential flow in the fixed length array；And
It is described according to described address skip list and the I/O data, the attribute of the affiliated sequential flow of I/O data described in more new record letter
It ceases in fixed length array, including：
Judge whether the I/O data and the history I/O data that described address skip list is recorded are related；
If so, from the fixed length array extraction with the relevant history I/O data of the I/O data with identical offset
Location and the attribute information of renewal time nearest sequential flow, and the attribute information according to extraction and the I/O data, more new record
The attribute information of the affiliated sequential flow of I/O data is in fixed length array；And
If it is not, according to the address information of the I/O data, create the attribute information of sequential flow belonging to the I/O data in
The fixed length array.
Further, described address information includes the data length of the I/O data；And the judgement I/O data
It is whether related to the described address skip list history I/O data recorded, including：
According to the storage address section of the I/O data, the storage for the history I/O data recorded with described address skip list is judged
Address section is with the presence or absence of overlapping or continuous relationship；Wherein, the storage address section by the I/O data offset address
The section obtained with data length computation；
If so, judge that the I/O data is related to the history I/O data that described address skip list is recorded；And
If it is not, judge that the I/O data and the history I/O data that described address skip list is recorded are uncorrelated.
Further, the attribute information further includes data length；And
The attribute information according to extraction and the I/O data, the attribute of the affiliated sequential flow of I/O data described in more new record
Information in fixed length array, including：
Add one to the IO quantity in the attribute information of extraction, and the numerical value after one to be added to update sequence belonging to the I/O data
The IO quantity of stream；
With the sum of the data length of the I/O data and the data length in the attribute information of extraction the update I/O data
The data length of affiliated sequential flow；
The renewal time of the affiliated sequential flow of I/O data is updated with current time；And
The attribute information of the updated affiliated sequential flow of the I/O data is recorded in the fixed length array.
Further, the address information according to the I/O data creates the category of the sequential flow belonging to the I/O data
Property information in the fixed length array, including：
Using numerical value one as the IO quantity of the affiliated sequential flow of the I/O data；
Using the data length of the I/O data as the data length of the affiliated sequential flow of the I/O data；
Using current time as the renewal time of the affiliated sequential flow of the I/O data；
Using the offset address of the I/O data as the offset address of the affiliated sequential flow of the I/O data；And
The attribute information of the affiliated sequential flow of the I/O data of generation is recorded in the fixed length array.
Further, the fixed length array is known including leader；And record the attribute letter of the affiliated sequential flow of the I/O data
It ceases in the fixed length array, including：
The attribute information for recording the affiliated sequential flow of the I/O data is directed toward in the address that the leader is known in the fixed length number
Position in group；And
The cyclic address change known to the leader.
Further, the method further includes：
When the I/O data and the uncorrelated history I/O data that described address skip list is recorded, monitoring the I/O data is
It is no to be buffered in equipment caching and start to be performed refresh operation in equipment caching；The refresh operation is included in
The I/O data and the operation by I/O data write-in disk are removed in the equipment caching；
If so, the execution state for recording the sequential flow belonging to the I/O data is unfinished, and record the sequential flow
The initial value of concurrent quantity is one；And
When the refresh operation is completed in the I/O data executed, the execution state of the sequential flow is recorded to complete,
And the IO queue depths are updated according to the numerical value of the concurrent quantity.
Further, the method further includes：
When the I/O data is related to the history I/O data that described address skip list is recorded, judge belonging to the I/O data
Whether the execution state of sequential flow is unfinished；If so, one is added to the concurrent quantity.
Further, it is described that the IO queue depths are updated according to the numerical value of the concurrent quantity, including：
Judge whether the numerical value of the concurrent quantity is more than the numerical value of current IO queue depths；And
If so, using the numerical value of the concurrent quantity as the numerical value of the updated IO queue depths.
Further, the method further includes：
When the I/O data of the sequential flow is cached in equipment caching, the equipment is inquired according to described address skip list and is delayed
It whether there is and the relevant I/O data of the sequential flow in depositing；
If nothing, timing is started to the sequential flow；
When the I/O data being buffered in the sequential flow in the equipment caching performs refresh operation, to the clocking value
It resets；
Judge whether the corresponding clocking value of the sequential flow reaches timing threshold value；And
When the clocking value reaches the timing threshold value, the sequential flow is buffered in all in the equipment caching
I/O data performs refresh operation and the sequential flow is recorded in all letters in described address skip list and the fixed length array
Breath is removed.
Further, the largest buffered length is the mean data length of I/O data and multiplying for the IO queue depths
Product, the largest buffered quantity of the equipment caching are not more than the IO queue depths；And the method further includes：
Judge whether the quantity for the I/O data that the sequential flow is buffered in the equipment caching reaches the largest buffered
Quantity and judge the I/O data that the sequential flow is buffered in equipment caching total data length whether reach it is described most
Big buffer storage length；And
When the quantity reaches the largest buffered quantity or the total data length reaches the largest buffered length,
All I/O datas being buffered in the sequential flow in the equipment caching perform refresh operation.
Further, the method further includes：
According to described address skip list and the fixed length array, the IO that the sequential flow is buffered in the equipment caching is judged
Whether the storage address of data crosses over the memory block of storage device；And
If so, be buffered in the sequential flow in the equipment caching and storage address is the outland across memory block
All I/O datas before location perform refresh operation.
Further, the method further includes：
When the I/O data being buffered in the sequential flow in the equipment caching starts to perform refresh operation, according to described
Sequential flow is recorded in the renewal time of the fixed length array, records the refresh time of the sequential flow；And according to the fixed length
It is described fixed to judge whether the offset address for performing first I/O data of the refresh operation is recorded in the sequential flow for array
Whether the offset address in long array is consistent；
If so, the initial value for setting the refreshing quantity of the sequential flow is one；
If it is not, the refreshing quantity to the sequential flow adds one；
At the end of the refresh operation, subtract one to the refreshing quantity of the sequential flow；
Judge it is described refresh quantity numerical value whether be zero and judge the refresh time in period threshold value whether
There is update；
If no, all I/O datas being buffered in the sequential flow in the equipment caching perform refresh operation.
Second aspect, the embodiment of the present invention provide a kind of sequential stream detection and the device of caching, including：
Access request receiving module for receiving the access request of I/O data, and records the address information of the I/O data
In the skip list of address；
Fixed length array update module, for according to described address skip list and the I/O data, I/O data described in more new record
The attribute information of affiliated sequential flow is in fixed length array；Wherein, the attribute information includes the sequential flow being currently received
The IO quantity of I/O data；
Queue depth's judgment module, for judging whether the IO quantity meets IO queue depths；And
Sequential flow cache module, for if so, being recorded in the fixed length number according to described address skip list and the sequential flow
Attribute information in group caches the I/O data of the sequential flow being currently received.
Further, described address information includes the offset address of the I/O data, and the attribute information further includes described
The offset address of renewal time and the sequential flow of the attribute information of sequential flow in the fixed length array；And
The fixed length array update module includes：
Data correlated judgment unit, for whether judging the history I/O data of the I/O data and the record of described address skip list
It is related；
First attribute updating unit, for if so, being extracted and the relevant history of the I/O data from the fixed length array
I/O data has the attribute information of identical offset address and renewal time nearest sequential flow, and according to the attribute information of extraction
With the attribute information of the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；And
Second attribute updating unit, for if it is not, according to the address information of the I/O data, creating belonging to the I/O data
Sequential flow attribute information in the fixed length array.
The function of described device by hardware can also be performed corresponding software and be realized by hardware realization.It is described
Hardware or software include the one or more and corresponding module of above-mentioned function.
In a possible design, sequential stream detection and the structure of caching include processor and memory, described to deposit
Reservoir supports the device of sequential stream detection and caching to perform sequential stream detection and the side of caching in above-mentioned first aspect for storing
The program of method, the processor are configurable for performing the program stored in the memory.The sequential stream detection is with delaying
The device deposited can also include communication interface, lead to for sequential stream detection with the device cached with other equipment or communication network
Letter.
The third aspect, an embodiment of the present invention provides a kind of computer readable storage medium, for sequential stream detection with delaying
The computer software instructions used in device deposited, including for performing sequential stream detection in above-mentioned first aspect and the side of caching
Program of the method involved by the device of sequential stream detection and caching.
Any one technical solution in above-mentioned technical proposal has the following advantages that or advantageous effect：
The embodiment of the present invention recorded while the access request for receiving I/O data the address information of the I/O data in
In the skip list of address；Then the I/O data according to address skip list and the self attributes information of I/O data received come more new record
The attribute information of affiliated sequential flow is in fixed length array so that the serial relation belonged between the I/O data of same sequential flow passes through
Address skip list and fixed length array connect and since fixed-length data update the quantity of sequential flow received so that only
It receives and meets the sequential flow of certain IO quantity and can disposably piecemeal be cached, piecemeal is based on IO queues and caches depth.
In addition, above-mentioned skip list and fixed length array can identify that sequential flow facilitates further cache and progress to refresh accordingly into disk
Operation.
Above-mentioned general introduction is merely to illustrate that the purpose of book, it is not intended to be limited in any way.Except foregoing description
Schematical aspect, except embodiment and feature, by reference to attached drawing and the following detailed description, the present invention is further
Aspect, embodiment and feature will be what is be readily apparent that.
Description of the drawings
In the accompanying drawings, unless specified otherwise herein, otherwise represent the same or similar through the identical reference numeral of multiple attached drawings
Component or element.What these attached drawings were not necessarily to scale.It should be understood that these attached drawings are depicted only according to the present invention
Some disclosed embodiments, and should not serve to limit the scope of the present invention.
Fig. 1 is the flow diagram of sequential stream detection provided by the invention and one embodiment of the method for caching；
Fig. 2 is newer one of the attribute information of sequential stream detection provided by the invention and fixed length array in the method for caching
The flow diagram of embodiment；
Fig. 3 is the schematic diagram of skip list provided by the invention and one embodiment of the index relative of fixed length array；
Fig. 4 is the flow diagram of one embodiment of the newer method of IO queue depths provided by the invention；
Fig. 5 is the schematic diagram of another embodiment of the newer method of IO queue depths provided by the invention；
Fig. 6 is the stream of sequential stream detection provided by the invention and first embodiment of the flush buffers of the method for caching
Journey schematic diagram；
Fig. 7 is the stream of sequential stream detection provided by the invention and second embodiment of the flush buffers of the method for caching
Journey schematic diagram；
Fig. 8 is the stream of sequential stream detection provided by the invention and the third embodiment of the flush buffers of the method for caching
Journey schematic diagram；
Fig. 9 is the stream of sequential stream detection provided by the invention and the 4th embodiment of the flush buffers of the method for caching
Journey schematic diagram；
Figure 10 is the structure diagram of sequential stream detection provided by the invention and one embodiment of the device of caching；
Figure 11 is the structure diagram of one embodiment of terminal device provided by the invention.
Specific embodiment
Hereinafter, certain exemplary embodiments are simply just described.As one skilled in the art will recognize that
Like that, without departing from the spirit or scope of the present invention, described embodiment can be changed by various different modes.
Therefore, attached drawing and description are considered essentially illustrative rather than restrictive.
The present invention is directed to propose a kind of sequential stream detection and the mechanism of caching, each IO received by skip list record
One fixed length array of the address informations of data and setting, each element in array represent the attribute letter of a sequential flow
Whether then breath often receives an I/O data, related according to the I/O data that address skip list inquires the I/O data to historical reception arrives,
And then determine the sequential flow belonging to it, it updates the attribute information of the sequential flow that records in fixed length array, i.e., further cache or writes
Can sequential flow be identified by skip list and fixed length array when entering or reading sequential flow and operated accordingly.The embodiment of the present invention can
To be widely used in I/O system, the including but not limited to modules such as local file system, Network File System, disk drive can have
Effect reduces the IO number write out, optimizes IO forms, improves the performance of IO, can also be in flash memory (including but not limited to SSD)
When, it can effectively reduce and write amplification number, improve the service life of flash memory and when for file system, can effectively subtract
The quantity of few disk fragments.Technical scheme of the present invention is described into a point embodiment below：
Embodiment one
Referring to Fig. 1, an embodiment of the present invention provides a kind of sequential stream detection and the method for caching, by disk scheduling module
It performs, it is including step S110 to step S140, specific as follows：
S110, receives the access request of I/O data, and records the address information of I/O data in the skip list of address.
In order to realize the record of I/O data and caching, present invention uses the information of fixed length storage of array history I/O data,
Actually the attribute information of sequential flow and use address skip list as index record I/O data compile move address offset row
Sequence information, address skip list are ranked up using the offset address offset of I/O data as key, the ground of the I/O data of same serial relation
Location information arrangement is specific as shown in Figure 3 after the address information of previous I/O data.
S120, according to address skip list and I/O data, the attribute information of the affiliated sequential flow of the current I/O data of more new record is in fixed
Long array；Wherein, the attribute information includes the IO quantity of the I/O data of the sequential flow being currently received.
Wherein, each element of fixed length array is that (such as sequential flow 1 is ctx_ for the record of the attribute information of a sequential flow
1), which includes newer time (up_time) of the attribute information in the fixed length array, is currently received
Data length (the seq_ of the IO quantity (seq_num) of the I/O data of the sequential flow, the I/O data for the sequential flow being currently received
The information such as len).
So, for a specific example of step S120, as shown in Fig. 2, can be as follows：
S121 judges whether the I/O data being currently received and the history I/O data that described address skip list is recorded are related.
Since skip list record in address has the offset address offset of each I/O data and data length len, then accordingly
The address section (offset_begin, offset_end) of each I/O data can be obtained, offset_begin is the IO numbers
According to offset address offset, offset_end is the offset address offset of the I/O data plus the ground after data length len
Location.It is and then whether related to history I/O data in the I/O data for judging to be currently received, you can associated sequential flow is searched,
Specifically：
According to the storage address section of current I/O data, the storage address for the history I/O data recorded with address skip list is judged
Section is with the presence or absence of overlapping or continuous relationship；If so, judge the history I/O data phase that current I/O data is recorded with address skip list
It closes；If it is not, judge that current I/O data and the history I/O data that address skip list is recorded are uncorrelated.So for address space weight
Situations such as endless Total continuity in the non-critical dull continuous, address in folded, address, can also identify, correspondingly be recorded.So,
It is as follows for the operation for subsequently updating fixed length array：
S122, if judgement is related, the extraction history I/O data tool relevant with the I/O data from the fixed length array
There are the attribute information of identical offset address and renewal time nearest sequential flow, and the attribute information according to extraction and the IO
The attribute information of the affiliated sequential flow of I/O data described in data, more new record is in fixed length array.
S123 if it is determined that related, according to the address information of the I/O data, creates sequential flow belonging to the I/O data
Attribute information is in the fixed length array.
In fact, as shown in figure 3, the first half is skip list, lower half is fixed length array, and fixed length array can also record sequential flow
Offset address, the address is identical with the offset address of first I/O data of sequential flow, and address skip list also recorded it is each
The offset address of a I/O data, then can be by the index for address information of the I/O data recorded in the skip list of address by offset address
To the attribute information of sequential flow.Assume that current I/O data is related to the I/O data of history due to aforementioned, then jump according to address
Table can extract the address information of all relevant I/O datas, and the address of the sequential flow of another aspect fixed length array record is always
It is the offset address of first I/O data, the address information of extraction and extraction has the attribute information of same offset address, is
The attribute information of the current affiliated sequential flow of I/O data, then according to extraction and recent renewal attribute information and current IO numbers
According to, you can be updated the attribute information of sequential flow.Assuming that current I/O data and the I/O data of history are relevant premises
Under, it, specifically can be as follows for this renewal process：
Add one to the IO quantity in the attribute information of extraction, and the numerical value after one to be added to update sequence belonging to current I/O data
The IO quantity of stream；
With the current I/O data of the sum of the data length of current I/O data and the data length in the attribute information of extraction update
The data length of affiliated sequential flow；
The renewal time of the current affiliated sequential flow of I/O data is updated with current time；
Using the offset address in the attribute information of extraction as the offset address of the affiliated sequential flow of current I/O data；With with
The attribute information of the updated affiliated sequential flow of current I/O data is recorded in the fixed length array.
Wherein, there are one leaders to know (head_index) for fixed length array, for recording time nearest IO serial numbers, this when
Between can be absolute time or a relative time, known by this leader, it is equivalent that fixed-length data is changed into one
The queue of first in first out (FIFO).So the above-mentioned fixed length array that is recorded in can be：Record the affiliated sequential flow of current I/O data
It is directed toward the position in the fixed length array in the address that attribute information is known in the leader；And the address known to the leader adds
One.
On the other hand, it is assumed that current I/O data is uncorrelated to the I/O data of history, then it is actually regarded as one
First I/O data of new sequential flow, then the specific implementation process of step S23 can be as follows：
Using numerical value one as the IO quantity of the affiliated sequential flow of the I/O data；
Using the data length of the I/O data as the data length of the affiliated sequential flow of the I/O data；
Using current time as the renewal time of the affiliated sequential flow of the I/O data；
Using the offset address of the I/O data as the offset address of the affiliated sequential flow of the I/O data；And
The attribute information of the affiliated sequential flow of the I/O data of generation is recorded in the fixed length array.Optionally, it records
It is directed toward the position in the fixed length array in the address that the attribute information of the affiliated sequential flow of current I/O data is known in the leader；And
The cyclic address change known to the leader.
S130, judges whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value.
In the present embodiment, the I/O data number of requests of current sequential flow meets preset quantity threshold value or the present count
When measuring the integral multiple of threshold value, you can the I/O data of the sequential flow being currently received to be cached.The preset quantity threshold
The integral multiple of value or preset quantity threshold value equipment according to the performance of different disk or interface.It can during caching
The sequential flow to be cached to be identified by aforementioned address skip list and fixed-length data the data for the sequential flow to be cached, i.e.,
Step S140.On the other hand, since fixed length array can also record the data length for the sequential flow being currently received, thus can also lead to
It crosses and judges whether the data length of record meets the integral multiple of preset data length threshold or the preset data length threshold, if
It is to meet, also can perform step S140.
S140 is recorded in the attribute information in the fixed length array according to described address skip list and the sequential flow, to working as
Before the I/O data of the sequential flow that the receives piecemeal on the basis of IO queue depths cached.
In a specific example, for same sequential flow, it is assumed that preset quantity threshold value is 10, is currently received
The continuous IO quantity of the sequential flow be 10, then the IO quantity of current record is integral multiple and the vacation of 10 or 10
If the IO queue depths detected are 5, then for the I/O data of 10 same sequential flows received, with every piece or every group not
Each piece or each group of I/O data are cached on the basis of more than 5 I/O datas.
Embodiment two
Referring to Fig. 4, dependent on IO queue depths, i.e. IO teams during the caching sequential flow that previous embodiment is related to
The detection process of row depth is also quite important for the caching for writing IO, and the present embodiment will be described in the detection of IO queue depths
Process：
First, to first I/O data of each sequential flow, following monitoring process can be performed：
S210, if current I/O data and the history I/O data that described address skip list is recorded are uncorrelated, i.e. the current data
First I/O data of the sequential flow new for one, monitoring first I/O data I/O data whether be buffered in equipment caching in
And start to be performed refresh operation in equipment caching；The refresh operation, which is included in the equipment caching, removes IO numbers
The operation of disk is written according to this and by I/O data；
S220, if first I/O data I/O data is buffered in equipment caching and starts quilt in equipment caching
Refresh operation is performed, if so, the execution state of the sequential flow belonging to first I/O data I/O data of record is unfinished, and remember
The initial value for recording the concurrent quantity of the sequential flow is one；And
S230 if the refresh operation is completed in the first I/O data executed, records the execution state of the sequential flow to complete,
And the IO queue depths are updated according to the numerical value of the concurrent quantity.
S240, if current I/O data is related to the history I/O data that address skip list is recorded, that is, the current data is with the
The I/O data that the affiliated sequential flow of one I/O data is connected, whether the execution state for judging the sequential flow belonging to the I/O data is not complete
Into；If so, one is added to the concurrent quantity.
Selectively, it is described that the IO queue depths are updated according to the numerical value of the concurrent quantity, including：
Judge whether the numerical value of the concurrent quantity is more than the numerical value of current IO queue depths；And
If so, using the numerical value of the concurrent quantity as the numerical value of the updated IO queue depths.
Citing description is carried out to the present embodiment below with reference to Fig. 5：
First, the attribute information of the first I/O data IO_1 of the new sequential flow of the most junior one is identified as (IO-ctx_1),
State is not complete (done=false), and concurrent quantity is 1 (co_num=1), then (is included but not limited to storage service
Disk, iSCSI interfaces etc.) initiate IO_1 request；
Then, when IO_2 continuous with the first I/O data IO_1 is reached, it will the information of more new record (IO-ctx_1),
If state is not still to complete (done=false), then concurrent quantity adds 1 (co_num+1), it is assumed that continues continuous n-th of IO and reaches
Then ctx_1 is unfinished state, then concurrent quantity is n；
And when IO_1 asks to complete, the information of ctx_1 will be updated, state is set as to complete (done=true),
The concurrent quantity (co_num) detected at this time can regard as IO queue depths (io_depth)；
In addition, when new sequential flow reaches, if it find that state is not complete (done=true), then can reset
State is not complete (done=false), and concurrent quantity is 1 (co_num=1), carries out next round detection, if next round is visited
The IO queue depths (io_depth) of survey can then update the value of IO queue depths (io_depth) more than original value.
Embodiment three
Fig. 6 is referred to, the present embodiment flushes to the I/O data described during equipment is cached after being cached to during equipment caches
Process in specific disk or interface, can be in a sequential flow of current cache for the I/O data for being written or reading
After data reach time threshold, then can the execution refresh operation of this current sequential flow be removed it and be cached in equipment
In data and be stored in the memory of disk or interface, it is specific as follows：
When the I/O data of sequential flow is cached in equipment caching, the equipment is inquired according to described address skip list by S310
It whether there is and the relevant I/O data of the sequential flow in caching；
S320 if nothing, starts timing to the sequential flow；
S330, when the I/O data being buffered in the sequential flow in the equipment caching performs refresh operation, to the meter
Duration is reset；
Due to refresh process be by one piece of one piece flushes in disk of data block, then it is long there are sequential flow, several times
Refresh operation is flushed in disk, then instant non-triggering timing threshold value and the refresh operation that performs should also carry out clocking value
It resets.
S340, judges whether the corresponding clocking value of the sequential flow reaches timing threshold value；And
S350, when clocking value reaches timing threshold value, all IO numbers for being buffered in the sequential flow in equipment caching
It is clear according to all information for performing refresh operation and the sequential flow being recorded in described address skip list and the fixed length array
It removes.
Example IV
Fig. 7 is referred to, the present embodiment on the basis of previous embodiment, can provide the strategy of second of flush buffers：
First, largest buffered length is set for the mean data length of I/O data and the product of the IO queue depths, institute
The largest buffered quantity for stating caching is not more than the IO queue depths.
In in the present embodiment, the spatial cache in being cached for equipment can be with：Set the mean data length of I/O data
(avg_len) value of IO queue depths (io_depth) is multiplied by as cache size, for example, for reading IO, it can be directly with caching
Size (cache_size) by after address align as read buffer value (rd_cache_size) carry out flush buffers operation,
In the way of IO is write, the operation of flush buffers can be carried out.For writing IO, I/O latency can be led to by writing IO due to caching
(latency) it is elongated, and then IO throughputs is caused to decline, it is therefore desirable to the performance of I/O device is tested, to obtain a conjunction
Suitable maximum write buffer quantity (wr_cache_num) and maximum write buffer length (wr_cache_size), but each sequential flow
Maximum write buffer quantity (wr_cache_num) is not greater than the most capitalization of IO queue depths (io_depth) or each sequential flow
Buffer storage length is not greater than the mean data length of I/O data and the product of the IO queue depths, thus reasoning IO queue depths
IO streams for 1 (io_depth=1) should not be cached.
Then in the present embodiment, the strategy of flush buffers can be：
S410, judges whether the quantity for the I/O data that the sequential flow is buffered in the equipment caching reaches the maximum
It caches quantity and judges whether the total data length for the I/O data that the sequential flow is buffered in the equipment caching reaches institute
State largest buffered length；And
S420, when the quantity reach the largest buffered quantity or the total data length to reach the largest buffered long
When spending, all I/O datas being buffered in the sequential flow in the equipment caching perform refresh operation.
Embodiment five
Fig. 8 is referred to, the present embodiment on the basis of previous embodiment, can provide the strategy of the third flush buffers：
S510 according to described address skip list and the fixed length array, judges that the sequential flow is buffered in the equipment caching
In I/O data storage address whether cross over storage device memory block；And
S520, if so, be buffered in the sequential flow in the equipment caching and storage address is across memory block
All I/O datas before boundary address perform refresh operation.
In a specific example, the storage address for the I/O data that a certain sequential flow is buffered in the equipment caching is：
[4,90], since the disk of write-in is usually to carry out piecemeal by 64, if an IO spans the storage of 64 or 64 integral multiple
Boundary address can be then split, then for sequential flow of the storage address for the current cache of [4,90], in the sequential flow
The I/O data of storage address [4,64] can be performed refresh operation, be flushed in disk from caching.
Embodiment six
Fig. 9 is referred to, the present embodiment on the basis of previous embodiment, can provide the strategy of the 4th kind of flush buffers：
S610, when the I/O data being buffered in the sequential flow in the equipment caching starts to perform refresh operation, root
The renewal time of the fixed length array is recorded according to the sequential flow, records the refresh time of the sequential flow；And according to institute
Fixed length array is stated, judges whether the offset address for performing first I/O data of the refresh operation is recorded in the sequential flow
Whether the offset address in the fixed length array is consistent；
S620, if so, the initial value for setting the refreshing quantity of the sequential flow is one；
S630, if it is not, the refreshing quantity to the sequential flow adds one；
S640 at the end of the refresh operation, subtracts one to the refreshing quantity of the sequential flow；
S650, judges whether the numerical value for refreshing quantity is zero and judges the refresh time in period threshold value
Inside whether there is update；
S660, if no, all I/O datas being buffered in the sequential flow in the equipment caching perform refresh operation.
In the present embodiment, for some sequential flow, I/O data can be buffered in by block in equipment caching successively,
And in equipment caching be also that one piece one piece of data block is flushed in disk or interface, thus same sequential flow is come
It says, the initial value for refreshing quantity is one, if start to perform a refresh operation of the sequential flow, the refresh operation
The offset address of first I/O data is consistent with the offset address that the sequential flow is recorded in the fixed length array, then explanation should
Refresh operation is first refresh operation of the sequential flow, and the initial value for setting the refreshing quantity of the sequential flow is one；If no
Unanimously, then illustrate the refresh operation be not first refresh operation of the sequential flow but other refresh operation, then at this time
To the refreshing quantity of the sequential flow carry out plus one.And often performed a refresh operation of aforementioned sequence stream, i.e., it is suitable to this
The refreshing quantity of sequence stream subtracts one, so as to start when being zero by the numerical value for the refreshing quantity for determining the sequential flow and often to hold
When the renewal time that the refresh operation of one sequential flow of row is recorded also does not change, illustrate current order stream probably
Through terminate to cache or equipment caching in current order stream I/O data quantity or data length do not meet and perform brush
The condition newly operated, but also come in without subsequent I/O data caching.Performing refresh operation at this time can be to avoid a upper sequence
A larger delay is generated between stream and the refresh operation of next sequential flow, and the time of the delay can be by previous embodiment
Timing threshold value determine.
The refresh process provided by above-described embodiment can effectively realize the merging and alignment for realizing IO sequential flows
Boundary address between disk block is written.
Embodiment seven
Referring to Fig. 10, the embodiment of the present invention provides a kind of sequential stream detection and the device of caching, including：
Access request receiving module 10 for receiving the access request of I/O data, and records the address letter of the I/O data
It ceases in the skip list of address；
Fixed length array update module 20, for according to described address skip list and the I/O data, IO numbers described in more new record
According to the attribute information of affiliated sequential flow in fixed length array；Wherein, the attribute information includes the sequential flow being currently received
I/O data IO quantity；
Queue depth's judgment module 30, for judging whether the IO quantity meets preset quantity threshold value or the present count
Measure the integral multiple of threshold value；And
Sequential flow cache module 40, for if so, being recorded in the fixed length according to described address skip list and the sequential flow
Attribute information in array, to the I/O data of the sequential flow that is currently received, piecemeal delays on the basis of IO queue depths
It deposits.
Further, described address information includes the offset address of the I/O data, and the attribute information further includes described
The offset address of renewal time and the sequential flow of the attribute information of sequential flow in the fixed length array；And
The fixed length array update module includes：
Data correlated judgment unit, for whether judging the history I/O data of the I/O data and the record of described address skip list
It is related；
First attribute updating unit, for if so, being extracted and the relevant history of the I/O data from the fixed length array
I/O data has the attribute information of identical offset address and renewal time nearest sequential flow, and according to the attribute information of extraction
With the attribute information of the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；And
Second attribute updating unit, for if it is not, according to the address information of the I/O data, creating belonging to the I/O data
Sequential flow attribute information in the fixed length array.
The function of described device by hardware can also be performed corresponding software and be realized by hardware realization.It is described
Hardware or software include the one or more and corresponding module of above-mentioned function.
In a possible design, sequential stream detection and the structure of caching include processor and memory, described to deposit
Reservoir supports the device of sequential stream detection and caching to perform sequential stream detection and the side of caching in above-mentioned first aspect for storing
The program of method, the processor are configurable for performing the program stored in the memory.The sequential stream detection is with delaying
The device deposited can also include communication interface, lead to for sequential stream detection with the device cached with other equipment or communication network
Letter.
Embodiment eight
The embodiment of the present invention also provides a kind of terminal device, and as shown in figure 11, which includes：Memory 21 and processor
22,21 memory of memory contains the computer program that can be run on processor 22.Processor 22 performs the computer program
Sequential stream detection and the method for caching in Shi Shixian above-described embodiments.The quantity of memory 21 and processor 22 can be one
It is or multiple.
The equipment further includes：
Communication interface 23, for the communication between processor 22 and external equipment.
Memory 21 may include high-speed RAM memory, it is also possible to further include nonvolatile memory (non-volatile
Memory), a for example, at least magnetic disk storage.
If memory 21, processor 22 and the independent realization of communication interface 23, memory 21, processor 22 and communication connect
Mouth 23 can be connected with each other by bus and complete mutual communication.The bus can be industry standard architecture
(ISA, Industry Standard Architecture) bus, external equipment interconnection (PCI, Peripheral
Component) bus or extended industry-standard architecture (EISA, Extended Industry Standard
Component) bus etc..The bus can be divided into address bus, data/address bus, controlling bus etc..For ease of representing, Figure 11
In only represented with a thick line, it is not intended that an only bus or a type of bus.
Optionally, in specific implementation, if memory 21, processor 22 and communication interface 23 are integrated in chip piece
On, then memory 21, processor 22 and communication interface 23 can complete mutual communication by internal interface.
In the description of this specification, reference term " one embodiment ", " example ", " is specifically shown " some embodiments "
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment of the present invention or example.Moreover, particular features, structures, materials, or characteristics described
It may be combined in any suitable manner in any one or more of the embodiments or examples.In addition, without conflicting with each other, this
The technical staff in field can be by the different embodiments or examples described in this specification and the spy of different embodiments or examples
Sign is combined.
In addition, term " first ", " second " are only used for description purpose, and it is not intended that instruction or hint relative importance
Or the implicit quantity for indicating indicated technical characteristic." first " is defined as a result, the feature of " second " can be expressed or hidden
Include at least one this feature containing ground.In the description of the present invention, " multiple " are meant that two or more, unless otherwise
It is clearly specific to limit.
Any process described otherwise above or method description are construed as in flow chart or herein, represent to include
Module, segment or the portion of the code of the executable instruction of one or more the step of being used to implement specific logical function or process
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discuss suitable
Sequence, including according to involved function by it is basic simultaneously in the way of or in the opposite order, to perform function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) it uses or combines these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress used with reference to these instruction execution systems, device or equipment
It puts.
Computer-readable medium described in the embodiment of the present invention can be that computer-readable signal media or computer can
Storage medium either the two is read arbitrarily to combine.The more specific example of computer readable storage medium is at least (non-poor
Property list to the greatest extent) including following：Electrical connection section (electronic device) with one or more wiring, portable computer diskette box (magnetic
Device), random access memory (RAM), read-only memory (ROM), erasable edit read-only storage (EPROM or flash
Memory), fiber device and portable read-only memory (CDROM).In addition, computer readable storage medium even can be with
It is the paper or other suitable media that can print described program on it, because can be for example by being carried out to paper or other media
Optical scanner is then handled described electronically to obtain into edlin, interpretation or when necessary with other suitable methods
Program is then stored in computer storage.
In embodiments of the present invention, computer-readable signal media can be included in a base band or as a carrier wave part
The data-signal of propagation, wherein carrying computer-readable program code.The data-signal of this propagation may be used a variety of
Form, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media is also
Can be any computer-readable medium other than computer readable storage medium, which can send, pass
Either transmission is broadcast for instruction execution system, input method or device use or program in connection.Computer can
Reading the program code included on medium can be transmitted with any appropriate medium, including but not limited to：Wirelessly, electric wire, optical cable, penetrate
Frequently (Radio Frequency, RF) etc. or above-mentioned any appropriate combination.
It should be appreciated that each section of the present invention can be realized with hardware, software, firmware or combination thereof.Above-mentioned
In embodiment, software that multiple steps or method can in memory and by suitable instruction execution system be performed with storage
Or firmware is realized.If for example, with hardware come realize in another embodiment, can be under well known in the art
Any one of row technology or their combination are realized：With for the logic gates to data-signal realization logic function
Discrete logic, have suitable combinational logic gate circuit application-specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are appreciated that realize all or part of step that above-described embodiment method carries
Suddenly it is that relevant hardware can be instructed to complete by program, the program can be stored in a kind of computer-readable storage medium
In matter, the program when being executed, one or a combination set of the step of including embodiment of the method.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing module, it can also
That each unit is individually physically present, can also two or more units be integrated in a module.Above-mentioned integrated mould
The form that hardware had both may be used in block is realized, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized in the form of software function module and is independent product sale or in use, can also be stored in a computer
In readable storage medium storing program for executing.The storage medium can be read-only memory, disk or CD etc..
The above description is merely a specific embodiment, but protection scope of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in its various change or replacement,
These should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with the guarantor of the claim
It protects subject to range.
Claims (17)
1. a kind of sequential stream detection and the method for caching, which is characterized in that including：
The access request of I/O data is received, and records the address information of the I/O data in the skip list of address；
According to described address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length
Array；Wherein, the attribute information includes the IO quantity of the I/O data of the sequential flow being currently received；
Judge whether the IO quantity meets the integral multiple of preset quantity threshold value or the preset quantity threshold value；And
If so, the attribute information in the fixed length array is recorded according to described address skip list and the sequential flow, to currently connecing
The I/O data of the sequential flow received piecemeal on the basis of IO queue depths is cached.
2. sequential stream detection as described in claim 1 and the method for caching, which is characterized in that described address information includes described
The offset address of I/O data, the attribute information further include update of the attribute information of the sequential flow in the fixed length array
Time and the offset address of the sequential flow；And
It is described according to described address skip list and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record in
Fixed length array, including：
Judge whether the I/O data and the history I/O data that described address skip list is recorded are related；
If so, from the fixed length array extraction and the relevant history I/O data of the I/O data with identical offset address and
The attribute information of renewal time nearest sequential flow, and the attribute information according to extraction and the I/O data, described in more new record
The attribute information of the affiliated sequential flow of I/O data is in fixed length array；And
If it is not, according to the address information of the I/O data, the attribute information of the sequential flow belonging to the newly-built I/O data is in described
Fixed length array.
3. sequential stream detection as claimed in claim 2 and the method for caching, which is characterized in that described address information includes described
The data length of I/O data；And it is described judge history I/O data that the I/O data records with described address skip list whether phase
It closes, including：
According to the storage address section of the I/O data, the storage address for the history I/O data recorded with described address skip list is judged
Section is with the presence or absence of overlapping or continuous relationship；Wherein, the storage address section by the I/O data offset address sum number
The section obtained according to length computation；
If so, judge that the I/O data is related to the history I/O data that described address skip list is recorded；And
If it is not, judge that the I/O data and the history I/O data that described address skip list is recorded are uncorrelated.
4. sequential stream detection as claimed in claim 2 and the method for caching, which is characterized in that the attribute information further includes number
According to length；And
The attribute information according to extraction and the I/O data, the attribute information of the affiliated sequential flow of I/O data described in more new record
In fixed length array, including：
Add one to the IO quantity in the attribute information of extraction, and the numerical value after one to be added to update the affiliated sequential flow of I/O data
IO quantity；
With belonging to the sum of the data length of the I/O data and the data length in the attribute information of extraction the update I/O data
The data length of sequential flow；
The renewal time of the affiliated sequential flow of I/O data is updated with current time；And
The attribute information of the updated affiliated sequential flow of the I/O data is recorded in the fixed length array.
5. sequential stream detection as claimed in claim 4 and the method for caching, which is characterized in that described according to the I/O data
Address information creates the attribute information of sequential flow belonging to the I/O data in the fixed length array, including：
Using numerical value one as the IO quantity of the affiliated sequential flow of the I/O data；
Using the data length of the I/O data as the data length of the affiliated sequential flow of the I/O data；
Using current time as the renewal time of the affiliated sequential flow of the I/O data；
Using the offset address of the I/O data as the offset address of the affiliated sequential flow of the I/O data；And
The attribute information of the affiliated sequential flow of the I/O data of generation is recorded in the fixed length array.
6. sequential stream detection and the method for caching as described in claim 4 or 5, which is characterized in that the fixed length array includes
Leader is known；And the attribute information of the affiliated sequential flow of the I/O data is recorded in the fixed length array, including：
The address that the attribute information of the affiliated sequential flow of the I/O data is known in the leader is recorded to be directed toward in the fixed length array
Position；And
The cyclic address change known to the leader.
7. sequential stream detection as claimed in claim 2 and the method for caching, which is characterized in that further include：
When the I/O data and the uncorrelated history I/O data that described address skip list is recorded, monitor the I/O data whether by
It is buffered in equipment caching and starts to be performed refresh operation in equipment caching；The refresh operation is included in described
The I/O data and the operation by I/O data write-in disk are removed in equipment caching；
If so, the execution state for recording the sequential flow belonging to the I/O data is unfinished, and record the concurrent of the sequential flow
The initial value of quantity is one；And
When the refresh operation is completed in the I/O data executed, record the execution state of the sequential flow to complete and
The IO queue depths are updated according to the numerical value of the concurrent quantity.
8. sequential stream detection as claimed in claim 7 and the method for caching, which is characterized in that further include：
When the I/O data is related to the history I/O data that described address skip list is recorded, the sequence belonging to the I/O data is judged
Whether the execution state of stream is unfinished；If so, one is added to the concurrent quantity.
9. sequential stream detection as claimed in claim 7 and the method for caching, which is characterized in that described according to the concurrent quantity
Numerical value update the IO queue depths, including：
Judge whether the numerical value of the concurrent quantity is more than the numerical value of current IO queue depths；And
If so, using the numerical value of the concurrent quantity as the numerical value of the updated IO queue depths.
10. sequential stream detection as claimed in claim 2 and the method for caching, which is characterized in that the method further includes：
When the I/O data of the sequential flow is cached in equipment caching, inquired in the equipment caching according to described address skip list
With the presence or absence of with the relevant I/O data of the sequential flow；
If nothing, timing is started to the sequential flow；
It is clear to the clocking value when the I/O data being buffered in the sequential flow in the equipment caching performs refresh operation
Zero；
Judge whether the corresponding clocking value of the sequential flow reaches timing threshold value；And
When the clocking value reaches the timing threshold value, all IO numbers for being buffered in the sequential flow in equipment caching
It is clear according to all information for performing refresh operation and the sequential flow being recorded in described address skip list and the fixed length array
It removes.
11. sequential stream detection as claimed in claim 7 and the method for caching, which is characterized in that the largest buffered length is
The mean data length of I/O data and the product of the IO queue depths, the largest buffered quantity of the equipment caching are not more than institute
State IO queue depths；And the method further includes：
Judge whether the quantity for the I/O data that the sequential flow is buffered in the equipment caching reaches the largest buffered quantity,
And judge whether the total data length for the I/O data that the sequential flow is buffered in the equipment caching reaches described maximum slow
Deposit length；And
When the quantity reaches the largest buffered quantity or the total data length reaches the largest buffered length, to institute
It states all I/O datas that sequential flow is buffered in the equipment caching and performs refresh operation.
12. sequential stream detection as claimed in claim 2 and the method for caching, which is characterized in that the method further includes：
According to described address skip list and the fixed length array, the I/O data that the sequential flow is buffered in the equipment caching is judged
Storage address whether cross over storage device memory block；And
If so, be buffered in the sequential flow in equipment caching and storage address be across memory block boundary address it
Preceding all I/O datas perform refresh operation.
13. such as claim 10~12 any one of them sequential stream detection and the method for caching, which is characterized in that the method
It further includes：
When the I/O data being buffered in the sequential flow in the equipment caching starts to perform refresh operation, according to the sequence
Stream is recorded in the renewal time of the fixed length array, records the refresh time of the sequential flow；And according to the fixed length array,
Whether the offset address for judging to perform first I/O data of the refresh operation with the sequential flow is recorded in the fixed length number
Whether the offset address in group is consistent；
If so, the initial value for setting the refreshing quantity of the sequential flow is one；
If it is not, the refreshing quantity to the sequential flow adds one；
At the end of the refresh operation, subtract one to the refreshing quantity of the sequential flow；
Judge whether the numerical value for refreshing quantity is zero and judges whether the refresh time has more in period threshold value
Newly；
If no, all I/O datas being buffered in the sequential flow in the equipment caching perform refresh operation.
14. a kind of sequential stream detection and the device of caching, which is characterized in that including：
Access request receiving module for receiving the access request of I/O data, and records the address information of the I/O data in ground
In the skip list of location；
Fixed length array update module, for according to described address skip list and the I/O data, belonging to I/O data described in more new record
The attribute information of sequential flow is in fixed length array；Wherein, the attribute information includes the IO numbers of the sequential flow being currently received
According to IO quantity；
Queue depth's judgment module, for judging whether the IO quantity meets preset quantity threshold value or the preset quantity threshold value
Integral multiple；And
Sequential flow cache module, for if so, being recorded in the fixed length array according to described address skip list and the sequential flow
Attribute information, to the I/O data of the sequential flow that is currently received, piecemeal caches on the basis of IO queue depths.
15. sequential stream detection as claimed in claim 14 and the device of caching, which is characterized in that described address information includes institute
State the offset address of I/O data, the attribute information further includes the attribute information of the sequential flow in the fixed length array more
New time and the offset address of the sequential flow；And
The fixed length array update module includes：
Data correlated judgment unit, for judge history I/O data that the I/O data records with described address skip list whether phase
It closes；
First attribute updating unit, for if so, being extracted and the relevant history IO numbers of the I/O data from the fixed length array
According to the attribute information with identical offset address and renewal time nearest sequential flow, and the attribute information according to extraction and institute
I/O data is stated, the attribute information of the affiliated sequential flow of I/O data described in more new record is in fixed length array；And
Second attribute updating unit, for if it is not, according to the address information of the I/O data, creating suitable belonging to the I/O data
The attribute information of sequence stream is in the fixed length array.
16. a kind of realize sequential stream detection and the terminal device of caching, which is characterized in that the terminal device includes：
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are performed by one or more of processors so that one or more of processors
Realize the sequential stream detection and the method for caching as described in any in claim 1-13.
17. a kind of computer readable storage medium, is stored with computer program, which is characterized in that the program is held by processor
The sequential stream detection and the method for caching as described in any in claim 1-6 are realized during row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810068469.7A CN108182151B (en) | 2018-01-24 | 2018-01-24 | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810068469.7A CN108182151B (en) | 2018-01-24 | 2018-01-24 | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108182151A true CN108182151A (en) | 2018-06-19 |
| CN108182151B CN108182151B (en) | 2019-02-01 |
Family
ID=62551376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810068469.7A Active CN108182151B (en) | 2018-01-24 | 2018-01-24 | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108182151B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274550A (en) * | 2018-09-07 | 2019-01-25 | 电信科学技术第五研究所有限公司 | A kind of adaptive IO queue depth matching process of iSCSI |
| CN113923506A (en) * | 2020-07-10 | 2022-01-11 | 成都鼎桥通信技术有限公司 | Video data processing method and device, edge computing gateway and storage medium |
| KR20220106622A (en) * | 2021-01-22 | 2022-07-29 | 성균관대학교산학협력단 | Zipper compaction method and apparatus for compacting the plural of skiplists |
| CN115543937A (en) * | 2022-03-22 | 2022-12-30 | 荣耀终端有限公司 | File defragmentation method and electronic device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023931A (en) * | 2010-12-17 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Self-adaption cache pre-fetching method |
| CN102298508A (en) * | 2011-09-07 | 2011-12-28 | 记忆科技(深圳)有限公司 | Stream-based method and device for prereading solid state disk |
| CN103927136A (en) * | 2014-04-21 | 2014-07-16 | 华为技术有限公司 | Identification method and device for input and output IO types |
| CN106294211A (en) * | 2016-08-08 | 2017-01-04 | 浪潮(北京)电子信息产业有限公司 | The detection method of a kind of multichannel sequential flow and device |
| CN106557431A (en) * | 2016-11-25 | 2017-04-05 | 郑州云海信息技术有限公司 | A kind of pre-head method and device for multichannel sequential flow |
| CN107589911A (en) * | 2017-09-05 | 2018-01-16 | 郑州云海信息技术有限公司 | A kind of I O process method and device of SSD cachings |
-
2018
- 2018-01-24 CN CN201810068469.7A patent/CN108182151B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023931A (en) * | 2010-12-17 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Self-adaption cache pre-fetching method |
| CN102298508A (en) * | 2011-09-07 | 2011-12-28 | 记忆科技(深圳)有限公司 | Stream-based method and device for prereading solid state disk |
| CN103927136A (en) * | 2014-04-21 | 2014-07-16 | 华为技术有限公司 | Identification method and device for input and output IO types |
| CN106294211A (en) * | 2016-08-08 | 2017-01-04 | 浪潮(北京)电子信息产业有限公司 | The detection method of a kind of multichannel sequential flow and device |
| CN106557431A (en) * | 2016-11-25 | 2017-04-05 | 郑州云海信息技术有限公司 | A kind of pre-head method and device for multichannel sequential flow |
| CN107589911A (en) * | 2017-09-05 | 2018-01-16 | 郑州云海信息技术有限公司 | A kind of I O process method and device of SSD cachings |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274550A (en) * | 2018-09-07 | 2019-01-25 | 电信科学技术第五研究所有限公司 | A kind of adaptive IO queue depth matching process of iSCSI |
| CN113923506A (en) * | 2020-07-10 | 2022-01-11 | 成都鼎桥通信技术有限公司 | Video data processing method and device, edge computing gateway and storage medium |
| KR20220106622A (en) * | 2021-01-22 | 2022-07-29 | 성균관대학교산학협력단 | Zipper compaction method and apparatus for compacting the plural of skiplists |
| KR102568662B1 (en) | 2021-01-22 | 2023-08-22 | 성균관대학교산학협력단 | Zipper compaction method and apparatus for compacting the plural of skiplists |
| CN115543937A (en) * | 2022-03-22 | 2022-12-30 | 荣耀终端有限公司 | File defragmentation method and electronic device |
| CN115543937B (en) * | 2022-03-22 | 2023-07-11 | 荣耀终端有限公司 | File defragmentation method and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108182151B (en) | 2019-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108182151B (en) | Method, apparatus, storage medium and the terminal device of sequential flow caching and detection | |
| CN104516825B (en) | Cache memory system and its operating method | |
| US20040024971A1 (en) | Method and apparatus for write cache flush and fill mechanisms | |
| CN110275841A (en) | Access request processing method, device, computer equipment and storage medium | |
| CN104699422B (en) | Data cached determination method and device | |
| WO2011031355A1 (en) | Cache prefill on thread migration | |
| US20070282573A1 (en) | Method and System for Changing a Description for a State Transition Function of a State Machine Engine | |
| US20130205089A1 (en) | Cache Device and Methods Thereof | |
| CN108009303A (en) | Searching method, device, electronic equipment and storage medium based on speech recognition | |
| CN111949578A (en) | DDR3 controller based on DFI standard | |
| CN116893991B (en) | Storage module conversion interface under AXI protocol and conversion method thereof | |
| CN109614049B (en) | Flash memory control method, flash memory controller and flash memory system | |
| CN117217977B (en) | GPU data access processing method, device and storage medium | |
| CN102053913A (en) | Memory device and data access method of memory device | |
| JP2008234059A (en) | Data transfer apparatus and information processing system | |
| CN1329809C (en) | Disk Array Controller and Its Working Method | |
| CN101566941A (en) | FIFO storage unit based on static random access memory (SRAM) | |
| CN112767978B (en) | DDR command scheduling method, device, equipment and medium | |
| CN112395011B (en) | Method for returning command response information, return control device and electronic equipment | |
| CN115712388A (en) | Data storage method, device and equipment of solid-state disk and storage medium | |
| CN110688238B (en) | Method and device for realizing queue of separated storage | |
| CN108399146B (en) | Flash controller, instruction fetching method and computer readable storage medium | |
| US20160004441A1 (en) | Access control method for accessing data in storage and disk drive | |
| CN102156628A (en) | Microprocessor and method for prefetching data into cache hierarchy of microprocessor | |
| CN106326146B (en) | Check the method whether cache hits |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |