CN109343943B - An I/O management method based on multiple external memory devices and multiple queues - Google Patents
An I/O management method based on multiple external memory devices and multiple queues Download PDFInfo
- Publication number
- CN109343943B CN109343943B CN201811040593.9A CN201811040593A CN109343943B CN 109343943 B CN109343943 B CN 109343943B CN 201811040593 A CN201811040593 A CN 201811040593A CN 109343943 B CN109343943 B CN 109343943B
- Authority
- CN
- China
- Prior art keywords
- request
- new
- offset
- original
- prefetch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title claims abstract description 25
- 238000013507 mapping Methods 0.000 claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 13
- 238000004590 computer program Methods 0.000 claims description 4
- 238000005192 partition Methods 0.000 claims description 4
- 230000002093 peripheral effect Effects 0.000 abstract description 2
- 238000000354 decomposition reaction Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000000638 solvent extraction Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明公开了一种基于多外存设备多队列的I/O管理方法，包括：将边块文件划分成大小相等的条带单元，条带单元以递增的顺序循环地条带化到条带化文件中；采用上述条带化方式，对原始I/O请求进行地址映射；判断原始I/O请求是否需要分解，如果是，将其分解成对齐条带单元边界的多个新I/O请求；否则，直接将其映射成等长的新I/O请求；对新I/O请求进行预取合并；将新I/O请求分派到对应外存设备的相应的I/O任务队列。本发明通过基于多外设多队列的I/O管理，确保每个I/O请求只由一个外存设备服务，每个应用I/O线程专用于一个外存设备，通过对新I/O请求的预取合并，将多个小的请求合并成一个大的I/O请求，有效地减少了对外存设备的访问次数，使得多外存I/O性能充分发挥。
The invention discloses an I/O management method based on multiple external storage devices and multiple queues. Using the above striping method, perform address mapping on the original I/O request; determine whether the original I/O request needs to be decomposed, and if so, decompose it into multiple new I/Os aligned to the stripe unit boundary request; otherwise, it is directly mapped into a new I/O request of equal length; the new I/O request is prefetched and merged; the new I/O request is dispatched to the corresponding I/O task queue of the corresponding external memory device. Through the I/O management based on multiple peripherals and multiple queues, the present invention ensures that each I/O request is only served by one external storage device, and each application I/O thread is dedicated to one external storage device. The request prefetch merge combines multiple small requests into one large I/O request, which effectively reduces the number of accesses to external memory devices and enables the multi-memory I/O performance to be fully utilized.
Description
Technical Field
The invention belongs to the technical field of external memory graph processing, and particularly relates to an I/O management method based on multiple external memory devices and multiple queues.
Background
With the continuous increase of the graph data size, a graph processing system using external memory processing has become a popular scheme for replacing a distributed memory mode graph processing system due to the advantages of low cost, easy expansion and the like, but external memory I/O becomes the performance bottleneck of the system. The use of multiple peripheral devices is a natural and common solution when processing large-scale graph data sets requiring I/O throughput. However, the multiple external memory I/O management method adopted by the current graph processing system using the external memory processing has the problems of insufficient sequential bandwidth utilization, limited parallel I/O capability, high management overhead and the like, so that the multiple external memory I/O performance is not fully exerted, and the realization of the optimal performance of the system is limited.
In the prior art, when a graph processing system uses an operating system management I/O method based on a striped volume, in order to maximally saturate an aggregate bandwidth of multiple external memory devices, a thread pool model of a single I/O task queue is adopted to concurrently initiate I/O requests. FIG. 1 is a diagram of a single queue I/O management system with multiple external storage devices in the prior art. As shown in fig. 1, the application main thread does not perform any I/O management task for the multi-external memory environment, but only puts the original I/O request task into the shared single I/O task queue, then takes the task out of the shared queue by the multi-application I/O thread, and transparently initiates the I/O request to the striped volume composed of multiple external memory devices, so that the situation that the data of one I/O request initiated by one application I/O thread spans multiple external memory devices cannot be avoided; data for I/O requests initiated by multiple application I/O threads may be on the same external memory device, resulting in contention overhead for the external memory device and file lock. Furthermore, in the context of an application I/O thread, I/O service tasks are performed in a sequential polling manner, thus limiting the exploitation of the parallel capability of multiple external memory devices. After I/O request decomposition is performed by a volume manager of an operating system, a Plug-list corresponding to an application I/O thread contains small I/O requests for different external memory devices. Operating systems performing both sorting and merging in such a Plug-list are inefficient, and at fine-grained stripe depths, concurrent I/O threads cause out-of-order arrivals that may not be predictable for sequential small I/O requests on each external memory device, rendering the merging ineffective. Such a thread pool model also results in contention of multiple application I/O threads in the application layer for locking on a single shared I/O task queue.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to solve the technical problems that the data of an I/O request spans a plurality of external memory devices, the competition of multithreading on the devices and a file lock, the parallel capability is limited and the multi-external memory I/O performance is not fully exerted in the multi-external memory mode single-queue I/O management method in the prior art.
To achieve the above object, in a first aspect, an embodiment of the present invention provides a multi-external memory device multi-queue based I/O management method, where the method includes:
s1, dividing all edge block files after image data partitioning into strip units with equal sizes according to an updating sequence, and circularly striping the strip units into a plurality of striping files according to an increasing sequence;
s2, carrying out address mapping on the original I/O request by adopting a striping mode consistent with the step S1;
s3, judging whether the original I/O request needs to be decomposed or not, and if so, decomposing the original I/O request into a plurality of new I/O requests which are aligned with the boundaries of the stripe units; otherwise, the original I/O request is not decomposed and is directly mapped into a new I/O request with equal length;
s4, prefetching and merging the new I/O request;
and S5, distributing the new I/O request to a corresponding I/O task queue of the corresponding external storage device.
Specifically, step S2 is as follows:
and mapping the address of the original I/O request by adopting a striping mode consistent with the step S1 to obtain the initial striping file number and the initial offset address in the initial striping file of the original I/O request.
Specifically, step S3 is as follows:
if the initial offset address% stripe unit boundary + data length > stripe unit boundary within the initial striped file of the original I/O request, then according to the stripe unit boundary, decomposing the original I/O request into a plurality of new I/O requests aligned to the stripe unit boundary;
if the initial offset address% stripe unit boundary + data length in the initial striped file of the original I/O request is less than or equal to the stripe unit boundary, the original I/O request is not decomposed and is directly mapped into a new I/O request with equal length, and the% is a remainder operation.
Specifically, step S4 is as follows:
1) for a new I/O request directly mapped by the original I/O request, judging whether the size of the new I/O request is equal to the size of an I/O buffer area, if so, not considering prefetch merging, and entering step S5, otherwise, judging whether the new I/O request is prefetched and merged before, if so, skipping the dispatching of the new I/O request, otherwise, entering step 2); for the first new I/O request aligned to the right of the stripe unit after the original I/O request is decomposed and the next several new I/O requests equal to the size of the stripe unit, judging whether the new I/O requests are prefetched and merged before, if so, skipping the dispatch of the new I/O request, otherwise, not considering the prefetching and merging, and entering the step S5; for a new I/O request which is subjected to decomposition, is aligned to the left of a stripe unit at the last and has the length smaller than the size of the stripe unit, judging whether the new I/O request is prefetched and merged before, if so, skipping the dispatching of the new I/O request, and otherwise, entering the step 2);
2) determining a maximum prefetch offset for a current new I/O request in a current active block;
3) judging whether the maximum prefetching offset is smaller than the termination offset of the current active block, if so, updating the prefetching offset in the original large linear address space to be the maximum prefetching offset, updating the length of the current new I/O request to be the difference value between the prefetching offset and the initial offset of the current new I/O request in the original linear address space, and entering the step 5); otherwise, updating the prefetch offset to be the termination offset of the current active edge block, and entering the step 4);
4) judging whether the adjacent edge block is active or not, if not, terminating prefetching, updating the length of the current new I/O request to be the difference value of the prefetching offset and the initial offset of the current new I/O request in the original linear address space, and entering the step 5); if the current prefetch offset is less than the termination offset of the adjacent edge block, the prefetch offset is updated to be the maximum prefetch offset; otherwise, updating the prefetch offset to the edge block termination offset, and repeating the step 4);
5) the new I/O request is merged.
Specifically, the determination of whether the new I/O request was prefetched before and merged specifically is as follows:
cur _ offset _ new < prefetch _ offset, indicating that the current new I/O request has been prefetched and merged; cur _ offset _ new ≧ prefetch _ offset, which is the starting offset in the original large linear address space of the currently processed new I/O request, indicates that the current new I/O request has not been prefetched and merged, and prefetch _ offset is the prefetch offset indicating the address of the prefetch location in the original large linear address space.
To achieve the above object, in a second aspect, the embodiments of the present invention provide a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the I/O management method as described above.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
(1) the invention maintains a special I/O task queue for each external memory device through I/O management based on multiple queues of multiple external memory devices, realizes the work of mapping, decomposing, prefetching, merging and dispatching of I/O requests in an application layer, ensures that each I/O request is only served by one external memory device, and avoids the situation that the data of one I/O request spans multiple external memory devices;
(2) according to the invention, through I/O management based on multiple external memory devices and multiple queues, each application I/O thread is dedicated to one external memory device, so that competition of multithreading on devices and file locks is reduced, and the problem of limited parallel service capability of the multiple external memory devices caused by the fact that each application I/O thread initiates I/O requests aiming at different external memory devices is avoided;
(3) the invention combines a plurality of small new I/O requests into a large I/O request by prefetching and combining the new I/O requests, thereby effectively reducing the access times to the external memory equipment and leading the multi-external memory I/O performance to be fully exerted.
Drawings
FIG. 1 is a diagram illustrating a single queue I/O management system with multiple external storage devices according to the prior art;
FIG. 2 is a diagram of a multi-queue I/O management system for multiple external memory devices according to the present invention;
FIG. 3 is a flowchart of a method for managing I/O based on multiple queues of multiple external memory devices according to the present invention;
FIG. 4 is a schematic diagram illustrating an address mapping and splitting process of an original I/O request provided by the present invention, where FIG. 4(a) shows that data of the original I/O request is in a single stripe unit, and FIG. 4(b) shows that the original I/O request spans multiple stripe units;
FIG. 5 is a diagram illustrating a distribution of new I/O request data after address mapping and decomposition according to the present invention, where FIG. 5(a) is an example one and FIG. 5(b) is an example two.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The patent provides an I/O management method based on multiple external memory devices and multiple queues, which is realized by adopting multiple external memory devices and multiple queue thread pools. FIG. 2 is a diagram of a multi-queue I/O management system for multiple external memory devices according to the present invention. As shown in fig. 2, the work of splitting, prefetching, merging, and dispatching I/O requests is performed at the application layer, a dedicated I/O task queue is maintained for each external memory device, a split new I/O request that does not span multiple external memory devices is added to the corresponding I/O task queue according to the number of the striped file where the request data is located, and application I/O threads and the I/O task queue are bound, so that each application I/O thread is dedicated to a specific external memory device.
Fig. 3 is a flowchart of an I/O management method based on multiple queues of multiple external memory devices according to the present invention. As shown in fig. 3, the method comprises the steps of:
s1, dividing all edge block files after image data partitioning into strip units with equal sizes according to an updating sequence, and circularly striping the strip units into a plurality of striping files according to an increasing sequence;
s2, carrying out address mapping on the original I/O request by adopting a striping mode consistent with the step S1;
s3, judging whether the original I/O request needs to be decomposed or not, and if so, decomposing the original I/O request into a plurality of new I/O requests which are aligned with the boundaries of the stripe units; otherwise, the original I/O request is not decomposed and is directly mapped into a new I/O request with equal length;
s4, prefetching and merging the new I/O request;
and S5, distributing the new I/O request to a corresponding I/O task queue of the corresponding external storage device.
S1, dividing all edge block files after image data partitioning into strip units with equal sizes according to an updating sequence, and circularly striping the strip units into a plurality of striping files according to an increasing sequence.
The method comprises the steps of carrying out two-dimensional partition on original graph data to obtain a plurality of edge block files, carrying out striping on the edge block files after graph partition on an application layer, dividing all the edge block files after the original graph data partition into coarse-grained stripe units with equal size according to an updating sequence, and circularly and uniformly distributing the coarse-grained stripe units in N continuous striped files in an increasing sequence. The striped files are numbered consecutively from 0 to N-1 and stored on the N external storage devices one-to-one.
And S2, carrying out address mapping on the original I/O request in a striping mode consistent with the step S1.
And mapping the address of the original I/O request by adopting a striping mode consistent with the step S1 to obtain the initial striping file number and the initial offset address in the initial striping file of the original I/O request. And dividing the original I/O request by taking the boundary of the edge block as a main boundary and the size of the I/O buffer area as a secondary boundary, wherein the size of the original I/O request is the size of the edge block or the size of the I/O buffer area.
S3, judging whether the original I/O request needs to be decomposed or not, and if so, decomposing the original I/O request into a plurality of new I/O requests which are aligned with the boundaries of the stripe units; otherwise, the original I/O request is not decomposed and is directly mapped into a new I/O request with equal length.
Judging whether the original I/O request needs to be decomposed or not according to the initial offset address, the data length and the strip unit boundary in the initial striping file of the original I/O request, and if so, decomposing the original I/O request into a plurality of new I/O requests which are aligned with the strip unit boundary; otherwise, the original I/O request is not decomposed and is directly mapped into a new I/O request with equal length.
If the starting offset address% stripe unit boundary + data length > stripe unit boundary within the starting striped file of the original I/O request indicates that the data of the original I/O request spans stripe units of multiple striped files, the original I/O request is decomposed into multiple new I/O requests that align stripe unit boundaries according to the stripe unit boundaries. The new I/O request corresponds to the corresponding striping file, and the striping files correspond to the external memory devices one by one, so that the decomposed new I/O request is only in one external memory device and does not span multiple external memory devices.
If the starting offset address% stripe unit boundary + the data length in the starting striped file of the original I/O request is less than or equal to the stripe unit boundary, which indicates that the data of the original I/O request is completely in one stripe unit of the starting striped file, the original I/O request is not decomposed and is directly mapped into a new I/O request with equal length.
FIG. 4 is a schematic diagram of the mapping and decomposition process of the original I/O request address provided by the present invention, where FIG. 4(a) shows that the original I/O request data is in a single stripe unit, and FIG. 4(b) shows that the original I/O request spans multiple stripe units. For convenience of the subsequent description, the following symbols are defined: blockiThe file is the ith edge block file accessed according to the updating sequence; IO (input/output)jIs the jth original I/O request; SIOkMapping the address and decomposing the kth new I/O request; SUmThe mth stripe unit in the jth striped file is striped; stripe _ filenStriping the file for the nth; MAX _ IOSIZE is the I/O buffer size, i.e., the maximum original I/O request size; s is the size of a strip unit; wherein i, j, k, m, n is 0,1, … ….
As shown in FIG. 4(a), when the original I/O request data is in a single stripe unit, the original I ≧The structure of the O request information comprises the starting offset address O and the I/O request data length thereof in the original large linear space
Obtaining the initial striping file number SI of the new I/O request after the address mapping0Start offset address SO in start striping file0And resolved I/O request data length
I/O manager transfers SI0Corresponding file descriptor fin _ profiles [ SI ]0]、SO0And
new I/O request information is assembled. The new I/O request is dispatched to the corresponding I/O task queue Multi _ task _ queues SI0]。
As shown in FIG. 4(b), when the original I/O request spans multiple stripe units, the structure of the original I/O request information includes its offset address O and I/O request data length starting in the original large linear space
Obtaining the initial striping file number SI of the new I/O request after address mapping and decompositioniStart offset address SO in start striping fileiAnd resolved I/O request data length
I/O manager transfers SIiCorresponding file descriptor fin _ profiles [ SI ]i]、SOiAnd
new I/O request information is assembled.
And S4, performing prefetching and merging on the new I/O request.
FIG. 5 is a diagram illustrating a distribution of new I/O request data after address mapping and decomposition according to the present invention, where FIG. 5(a) is an example one and FIG. 5(b) is an example two. In order to solve the problem that an original I/O request spans multiple external memory devices, the invention maps and decomposes the original I/O request to obtain a plurality of new I/O requests, so that each new I/O request is in a stripe unit of one external memory device. However, since it cannot be guaranteed that the boundary of each edge block after partitioning is aligned with the boundary of the stripe unit, the following situations may exist after mapping decomposition:
1) an original I/O request that is not larger than the stripe unit size S may fall into a stripe unit of a striped file after mapping, and not aligned with the left and right boundaries of the stripe unit, such as IO in FIG. 5(a)j+2And IOj+3Address mapping conditions of (1); or mapping, decomposing into two new I/O requests, the first aligned to the right of the previous stripe unit and the second aligned to the left of the next stripe unit, such as IO in FIG. 5(a)j+1Address mapping conditions.
2) An original I/O request larger than the stripe unit size S is mapped and decomposed into at least two new I/O requests, wherein the first new I/O request is aligned to the right of the first stripe unit and is not larger than S, and the last new I/O request is aligned to the left of the last stripe unit and is not larger than S, as shown in FIG. 5(a) for IOjMapping decomposed SIOkIn the middle are several new I/O requests equal to the stripe unit size.
Therefore, there must be a case where two or more small new I/O request data are contained in one stripe unit. FIG. 5 is a drawing showing
And the data of the new I/O request after mapping and decomposition of the original I/O request can be distributed. In FIG. 5(a), SIOkAnd SIOk+1Is directed to BlockiOriginal I/O request IO requesting datajAnd IOj+1Obtained after mapping decomposition, and the requested data falls into SUmIf BlockiIs an active edge block, we are handling IOjIt can pre-process IOj+1Mapping and decomposition of (1), SIOkAnd SIOk+1Merging the two requests into a new I/O request to realize the prefetching of the graph data; SIOk+2Is directed to the present activityBlock jumpiLast original I/O request IOj+1Mapping decomposed with SUm+1Left aligned new I/O request, if Blocki+1Is also an active Block, for Blocki+1IO (input/output)j+2Mapping the resulting SIOk+3Can be prefetched and matched with the SIOk+2Merge, in the same way, by judging Blocki+2、Blocki+3Determines whether to prefetch merge, and if both are active, the request data falls into the SUm+1The 4 new I/O requests within can be merged into one large, continuous new I/O request. In FIG. 5(b), if we are processing for the active edge Block BlockiIO (input/output)jDecomposition to obtain SIOkFor the same for BlockiAnd requests the SIO that the data falls into the same stripe unitk+1Directly prefetching merging, considering prefetching merging according to activity of subsequent adjacent edge blocks, and if all the prefetching merging are active, the request data falls into SUmThe 5 new I/O requests within can be merged into one new I/O request.
Prefetch merge is specifically as follows:
1) for a new I/O request directly mapped from an original I/O request, determining whether the new I/O request size is equal to the I/O buffer size MAX _ IOSIZE, if so, disregarding the prefetch merge, and proceeding to step S5, otherwise, determining whether the new I/O request was previously prefetched and merged, if so, skipping dispatch of the new I/O request, otherwise, proceeding to step 2); for the first new I/O request aligned to the right of the stripe unit after the original I/O request is decomposed and the next several new I/O requests equal to the size of the stripe unit, judging whether the new I/O requests are prefetched and merged before, if so, skipping the dispatch of the new I/O request, otherwise, not considering the prefetching and merging, and entering the step S5; for a new I/O request which is subjected to decomposition, is aligned to the left of a stripe unit at the last and has the length smaller than the size of the stripe unit, judging whether the new I/O request is prefetched and merged before, if so, skipping the dispatching of the new I/O request, and otherwise, entering the step 2);
to implement such a prefetch merge function, the multi-queue based I/O manager needs to maintain two important pointers, one is the starting offset cur _ offset _ new of the currently processed new I/O request in the original large linear address space, and one is the prefetch offset prefetch _ offset indicating the address of the prefetch location in the original large linear address space, and updates cur _ offset _ new every time a new I/O request is processed, and updates prefetch _ offset once the prefetch merge is completed. By determining the magnitude relationship between cur _ offset _ new and prefetch _ offset, it is determined whether to skip the prefetch merge process and dispatch of the current new I/O request:
(1) cur _ offset _ new < prefetch _ offset, which indicates that the current new I/O request has been prefetched and merged and dispatched to the corresponding I/O task queue, then processing is skipped;
(2) cur _ offset _ new ≧ prefetch _ offset, which indicates that the current new I/O request is not processed, then its prefetch merge processing is considered, and the large new I/O request after prefetch merge is dispatched to the corresponding I/O task queue.
2) Determining a maximum prefetch offset for a current new I/O request in a current active block;
if the starting offset of the current new I/O request in the original large linear address space plus MAX _ IOSIZE is greater than the offset of the current stripe unit right boundary in the original large linear address space, then the maximum prefetch offset is the offset of the current stripe unit right boundary in the original large linear address space; if not, then the maximum prefetch offset is the starting offset of the current new I/O request in the original large linear address space plus MAX _ IOSIZE.
3) Judging whether the maximum prefetching offset is smaller than the termination offset of the current active block, if so, updating the prefetching offset in the original large linear address space to be the maximum prefetching offset, updating the length of the current new I/O request to be the difference value of the prefetching offset and the initial offset of the current new I/O request in the original linear address space, and turning to 5); otherwise, updating the prefetch offset to be the current active edge block termination offset, and turning to 4);
4) judging the activity degree of the next adjacent edge block, if not, terminating prefetching, updating the length of the current new I/O request to be the difference value of the prefetching offset and the initial offset of the current new I/O request in the original linear address space, and turning to 5); if the current prefetch offset is not less than the termination offset of the adjacent edge block, the prefetch offset is updated to be the maximum prefetch offset; otherwise, updating the prefetch offset to the edge block termination offset, and repeating the step 4);
5) the new I/O request is merged.
The new I/O requests are merged based on the updated prefetch offset in the original large linear address space and the length of the current new I/O request.
And S5, distributing the new I/O request to a corresponding I/O task queue of the corresponding external storage device.
Each new I/O request is assigned to a corresponding I/O task queue, multi _ task _ requests SIi]。
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (5)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811040593.9A CN109343943B (en) | 2018-09-07 | 2018-09-07 | An I/O management method based on multiple external memory devices and multiple queues |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811040593.9A CN109343943B (en) | 2018-09-07 | 2018-09-07 | An I/O management method based on multiple external memory devices and multiple queues |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109343943A CN109343943A (en) | 2019-02-15 |
| CN109343943B true CN109343943B (en) | 2021-08-03 |
Family
ID=65304925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811040593.9A Active CN109343943B (en) | 2018-09-07 | 2018-09-07 | An I/O management method based on multiple external memory devices and multiple queues |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109343943B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977047A (en) * | 2019-04-02 | 2019-07-05 | 成都思鸿维科技有限责任公司 | External device access method, apparatus and processor |
| CN111290711A (en) * | 2020-01-21 | 2020-06-16 | 苏州浪潮智能科技有限公司 | IO processing method and related device of RAID system |
| CN111708742B (en) * | 2020-05-24 | 2022-11-29 | 苏州浪潮智能科技有限公司 | Input and output read-ahead method and device for a distributed file system |
| CN113849421B (en) * | 2021-09-16 | 2023-11-17 | 苏州浪潮智能科技有限公司 | Hierarchical aggregation method and device for data in full flash memory |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100495384C (en) * | 2007-04-13 | 2009-06-03 | 北京时代民芯科技有限公司 | Data inputting/outputting construction in coarse-grain re-arrangable computing structure |
| US20100257329A1 (en) * | 2009-04-02 | 2010-10-07 | Brucek Khailany | Apparatus and method for loading and storing multi-dimensional arrays of data in a parallel processing unit |
| US8904115B2 (en) * | 2010-09-28 | 2014-12-02 | Texas Instruments Incorporated | Cache with multiple access pipelines |
| CN108009006B (en) * | 2016-11-02 | 2022-02-18 | 华为技术有限公司 | Scheduling method and device of I/O (input/output) request |
-
2018
- 2018-09-07 CN CN201811040593.9A patent/CN109343943B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109343943A (en) | 2019-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109343943B (en) | An I/O management method based on multiple external memory devices and multiple queues | |
| US8874807B2 (en) | Storage-side storage request management | |
| US8671134B2 (en) | Method and system for data distribution in high performance computing cluster | |
| CN101799773B (en) | Memory access method of parallel computing | |
| US9378047B1 (en) | Efficient communication of interrupts from kernel space to user space using event queues | |
| EP2293191A2 (en) | Task and data management in a multiprocessor system | |
| US20090165003A1 (en) | System and method for allocating communications to processors and rescheduling processes in a multiprocessor system | |
| EP1349065A2 (en) | Symmetrical multiprocessing in multiprocessor systems | |
| US20110055839A1 (en) | Multi-Core/Thread Work-Group Computation Scheduler | |
| US9086916B2 (en) | Architecture for efficient computation of heterogeneous workloads | |
| US20050081208A1 (en) | Framework for pluggable schedulers | |
| WO2013082809A1 (en) | Acceleration method, device and system for co-processing | |
| CA2556083A1 (en) | Memory allocation | |
| US9507633B2 (en) | Scheduling method and system | |
| JP3832341B2 (en) | Memory pool management method | |
| US7685601B2 (en) | Methods and apparatus for segmented stack management in a processor system | |
| Awatramani et al. | Increasing gpu throughput using kernel interleaved thread block scheduling | |
| US20090300634A1 (en) | Method and System for Register Management | |
| CN102184124B (en) | Task scheduling method and system | |
| CN106537321B (en) | Method, device and storage system for accessing files | |
| US11392388B2 (en) | System and method for dynamic determination of a number of parallel threads for a request | |
| CN111143058A (en) | Memory management method based on backup list | |
| CN116302391A (en) | Multithreading task processing method and related device | |
| CN115878910A (en) | Line query method, device and storage medium | |
| US20230393782A1 (en) | Io request pipeline processing device, method and system, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |