JPH10133946A - Data reading method and its device - Google Patents

Abstract

(57) [Summary] To increase the speed of reading data from a low-speed secondary storage device. SOLUTION: When a read request for data 1 is first made, another data 2 associated as data having a high probability of being read next according to a predetermined rule is stored at high speed from a low-speed storage device MO12. By pre-expanding the data in the cache memory 11 as a device, even if the subsequent read request for data 2 is actually the first request, the read request for data 1 earlier than that is required. MO12
, The data 2 expanded in the cache memory 11 can be read from the cache memory 11 at high speed, whereby the speed of reading data from the MO 12 can be substantially increased.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for reading data, and more particularly to a method for reading large data such as image data from a low-speed secondary storage device having a low access speed. is there.

[0002]

2. Description of the Related Art Conventionally, many apparatuses have been used in which digital image data is stored in a memory or the like and can be read out and displayed. The image data to be stored is input using an image input device such as an image scanner or a video camera using a CCD, for example, and is often configured to be able to store a plurality of image data. .

[0003] Basically, each image data is stored as an image file. When storing an image file having a large data amount, for example, a large-capacity magneto-optical disk (MO) or hard disk (HDD) is used as a file storage device. When a magneto-optical disk is used, image data for one file is usually recorded on one magneto-optical disk. However, if the amount of image data constituting one image file exceeds the capacity of one magneto-optical disk, it is divided and stored on a plurality of magneto-optical disks.

Here, when a large-scale image file device is constructed using the above-mentioned magneto-optical disk, for example, a disk library device such as a jukebox which can exchange a plurality of magneto-optical disks by an automatic change mechanism is used. Is done. In this disk library device,
The disk is exchanged in response to an instruction from the user, and the corresponding image file is read.

[0005]

However, in this type of apparatus, a great deal of processing time is required for disc replacement, and the time from when an image reading instruction is issued to when an image is actually displayed becomes long. There was a problem. In particular, if the disk is frequently exchanged in response to a random access request, the processing time until image display becomes very long.

[0006] Such a problem exists not only when the above-described disk library device is used but also exists. Also,
The same applies when data to be read is data other than image data. That is, access to a large-capacity external storage device having a low access speed (for example, a secondary storage device such as a magnetic tape device, a magnetic disk device, or a floppy disk), and read data stored therein. In this case, the reading process takes a long time.

In order to store a large amount of data, the data must be stored in a large-capacity secondary storage device. Even in such a case, the data can be read out in as short a time as possible. desired. Such an increase in the reading speed is desired not only for applications to be created in the future but also for any applications already on the market.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and aims to increase the speed of reading data from a low-speed secondary storage device (particularly, a disk library device requiring disk replacement). With the goal. A further object of the present invention is to make it possible to apply such means for increasing the reading speed in a general-purpose manner regardless of the application.

[0009]

According to the data reading method of the present invention, when a data reading request is issued, the data is associated as data having a higher probability of being read next to the data by a predetermined rule. Is developed in advance from a low-speed storage device to a high-speed storage device.

Another feature of the present invention is that when a data read request is issued, it is determined whether the data is in the high-speed storage device. In addition to reading from the storage device, other data associated with data having a high probability of being read next to the data according to a predetermined rule is previously expanded from the low-speed storage device to the high-speed storage device. That is.

Another feature of the present invention is that when reading data from the low-speed storage device, the read data is copied to the high-speed storage device.

Another feature of the present invention is that the data reading method according to claim 1 or 2 is performed by an operating system applied to a computer by interpreting the predetermined rule. It is.

Another feature of the present invention resides in that a file name of data to be developed in advance is determined based on a file name of data requested to be read from the operating system, and is determined based on the file name of the operating system. This is to supply it to the system.

Another feature of the present invention is that the predetermined rule includes a rule for associating different data files or parts in the same data file with each other. is there.

A data reading device according to the present invention has a rule storing means for storing a rule defining a predetermined reading rule, and, when a data read request is made, based on the rule stored in the rule storing means. Read control means for expanding in advance from the low-speed storage device to the high-speed storage device other data associated as data having a high probability of being read next to the data.

Another feature of the present invention is that a rule storage means for storing a rule defining a predetermined read rule, and whether or not the data is in a high-speed storage device when a data read request is issued. Is read from the high-speed storage device when there is a certain time, and is read out from the low-speed storage device when there is no other, and the other data associated with the data having a higher probability of being read next to the data by the rule stored in the rule storage means. Read control means for expanding data from the low-speed storage device to the high-speed storage device in advance.

According to another feature of the present invention, the rule stored in the rule storage means includes a process for expanding data from the low-speed storage device to the high-speed storage device in advance by an operating system applied to a computer. It is characterized by a rule defined to be performed by the system.

Another feature of the present invention is that the low-speed storage device is a disk library device having a plurality of disks for storing data.

Since the present invention comprises the above technical means, when a data read request is made, even if the read request is actually the first request, other data read performed earlier than that request may be performed. If data corresponding to the high-speed storage device has already been expanded from the low-speed storage device at the time of the request, the corresponding data is read from the high-speed storage device instead of from the low-speed storage device. It is possible to substantially increase the speed of reading data from the memory. Moreover, since the processing of expanding the data in advance in this way is performed based on the rule associated with the data having a high probability of being read next,
In many cases, a high-speed storage device can be accessed and read even for the first read request, and the data read processing time can be shortened as a whole.

According to another feature of the present invention, a process of pre-loading data from a low-speed storage device to a high-speed storage device is performed at an operating system level applied to a computer, and is executed without an application. Therefore, it is not necessary to modify an application to incorporate such a function, and it is possible to easily incorporate any application later.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram for explaining a data reading method of the present embodiment, FIG. 2 is a diagram showing an example of rules used in the data reading method of the present embodiment, and FIG. FIG. 4 is a conceptual diagram showing a mechanism of the present embodiment for realizing the data read method, FIG. 4 is a block diagram showing a functional configuration of the prefetch driver shown in FIG. 3, and FIG. 5 describes the data read method of the present embodiment in more detail. FIG.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 to 5. Prior to that, a read cache which has been conventionally generally used to speed up data read processing will be described. Will be described with reference to FIGS. 7 and 8. Note that the downward arrow shown in FIG. 7 indicates the elapse of time t.

In FIG. 7, reference numeral 71 denotes a cache memory;
Reference numeral 72 denotes a magneto-optical disk (MO). The cache memory 71 may be configured using a semiconductor memory such as a DRAM or an SRAM. However, when the amount of data to be stored is large, for example, a part of a large-capacity, high-speed accessible hard disk storage area is used. Is done.

The MO 72 is one of a plurality of disks provided in the disk library apparatus, and stores, for example, an image file having a large data amount.
Then, an appropriate MO 72 is selected by the automatic change mechanism, and the image file is read therefrom. Since the disk exchange is performed in this way,
Reading data takes time.

As shown in FIG. 8, in a computer such as a personal computer, an application usually runs on an operating system (OS), and reading of image files is also performed through the OS. That is, when reading the image file, the application 8
1 accesses the OS, and reads the image file by driving the driver of the storage device by the file system 82 of the OS.

The drivers for the above-mentioned storage devices include a high-speed secondary storage driver 83 and a low-speed secondary storage driver (or tertiary storage driver) 84. The former high-speed secondary storage driver 83 includes a hard disk 71 and a RAID (Redundant Array of I
nexpensive Disks: a disk array device using a plurality of small hard disks) 85 for reading and writing data, etc. The latter low-speed secondary storage driver 84 is an MO72 or tape which cannot be accessed at high speed. It is for reading and writing data to 86 and the like.

The latter low-speed secondary storage driver 84 uses M
At the same time as directly accessing the O72 or the tape 86 to read the image file and returning it to the application 81 via the OS file system 82, the read image file is stored in the cache memory (for example, the storage (A part of the area is used).

Thus, the OS file system 82
Is checked to see if it is in the cache memory, and if so, it is retrieved and passed to the application 81. On the other hand, if the requested image file is not in cache memory,
The image file is read from the MO 72 and passed to the application 81, and the read image file is copied to the cache memory.

Therefore, as shown in FIG. 7, when there is a read request for reading a certain image file 1 first, since the image file 1 is not yet held in the cache memory 71, the MO 72 And reads it, copies it to the cache memory 71, and passes it to an application (not shown) to complete the reading process of the image file 1.

Next, it is assumed that there is a read request for reading an image file 2 different from the first image file 1 read. Also in this case, since the image file 2 is not yet held in the cache memory 71, the MO 72 is accessed and read, and is copied to the cache memory 71, and is passed to an application (not shown) to read the image file 2. Complete.

Thereafter, it is assumed that there is a read request for reading the same image file 1 as the image file read first. At this stage, since the image file 1 is held in the cache memory 71, the cache memory 71 is accessed and read, and the read image file is passed to an application (not shown) to complete the reading process. As a result, the image file 1 can be read from the cache memory 71 at a higher speed than when the MO 72 is directly accessed.

However, in such a normal read cache operation, the image file can be read from the cache memory 71 at high speed only at the time of the second or subsequent read request for the same image file. Therefore, for the first read request, MO72
Had to be accessed directly, and still required a long read processing time.

On the other hand, in the case of this embodiment, when a read request is made for any of the image files,
In addition to the read image file, an image file having a high probability of being read next is copied to the cache memory in advance. Such an operation is performed based on a predetermined rule as a predetermined reading rule. Hereinafter, such an operation unique to the present embodiment is referred to as “rule-based prefetch”.

According to this, as shown in FIG. 1, when a read request for reading a certain image file 1 is first received, the MO 12 is directly accessed and read, and the read is copied to the cache memory 11. At the same time, the image file 1 is passed to an application (not shown) to complete the reading process of the image file 1. At this time, the image file 2 predicted from the image file 1 is also prefetched into the cache memory 11 based on a predetermined rule.

Next, it is assumed that there is a read request for reading an image file 2 different from the first read image file 1. This read request is made for the first time, but since the image file 2 was already copied to the cache memory 11 when the previous image file 1 was read, the image file 2 can be read from the cache memory 11. Since the image file 1 is held in the cache memory 11 according to a normal cache operation, the image file 1 can be accessed and read even when a read request for the image file 1 is made subsequently.

As described above, according to this embodiment, an image file having a high probability of being read next to a certain image file is copied to the cache memory in advance before an actual read request. The cache memory can be accessed and read even for the first read request, and the image file read processing time can be substantially reduced.

By the way, when performing prefetching, the meaning of the relationship between the image file requested to be read and the image file to be prefetched correspondingly is very important. Therefore, when considering the provision of the above-described prefetch function, it is usual to carry out the application at the application level. That is, when an application is created, a prefetch function required for the application is generally incorporated into the application itself.

However, in this case, it is impossible to incorporate the prefetch function into an existing application, or even if it is possible, considerable labor is required. Therefore, in the present embodiment, by introducing a mechanism for performing prefetching at the operating system level, which is the basis of the operation of the application, the prefetch function as an additional function can be easily incorporated later.

FIG. 2 shows an example of a rule for performing prefetching at the OS level. Of the four examples shown in FIG. 2, the first to third examples are rules that specify replacement of a file name, and the fourth example is a rule that specifies replacement of a part of a file. is there. Each of these rules is defined based on the file name handled by the OS, and the prefetch operation is performed at the OS level.

That is, in the first example, the extension is gi
The rule indicates that if a read request is issued for the image file f, an image file having the same file name and an extension tif is prefetched. The second example represents a rule of prefetching image files having the same file name and the same extension stored in different directories.

Further, the third embodiment represents a rule that prefetching image files of the same extension in the file name A _n relationship of the file name A _{n + 1.} Further, in the fourth example, if a read request is made for the xth to yth bytes of data in an image file with an extension of dat0, ax + b to ax + b in an image file with the same file name and an extension of dat1 This represents a rule of prefetching data of the cy + d-th byte (ad is a predetermined coefficient).

The image file to be prefetched is 1
The number of image files is not limited to two, and two or more image files may be prefetched simultaneously. That is, all the image files satisfying the above-described rules may be prefetched at the same time, or a rule that prefetches two or more image files for one image file may be defined. .

Further, it is not always necessary to define the rule based on the file name. If prefetching can be performed at the OS level, the rule may be defined based on something else.

The following can be realized using such rules. For example, when there is a read request for an image file in which a file search icon is stored,
An image in which reduced image data is stored for prefetching an image file in which an original image is stored as a file having a high probability of being read next, or reducing and displaying the entire image on a part of a display screen (for example, a window). When a file read request is issued, it is possible to prefetch an image file storing image data to be enlarged and displayed on the entire display screen.

FIG. 3 is a diagram showing a mechanism for realizing the prefetch at the OS level as described above. That is, in the present embodiment, the prefetch driver 33 is provided separately from the application 31 in order to prefetch the image file at the OS level.

As shown in FIG.
Operates on the OS 32, and when reading an image file, the application 31 accesses the OS 32, and the OS 32 transmits the requested file name to the file sharing system 34 via the prefetch driver 33. The prefetch driver 33 determines a file name to be prefetched through the OS 32 according to a predetermined rule based on the requested file name,
Notify the file sharing system 34.

The file sharing system 34 drives the low-speed media driver 35 to read the requested image file from a magneto-optical disk (MO) (not shown),
2 to the application 31. In addition, processing for pre-loading an image file to be prefetched in a cache memory (not shown) (for example, a partial area of a hard disk) is also performed.

The prefetch driver 33 shown in FIG.
Has a functional configuration as shown in FIG. That is, in FIG. 4, reference numeral 41 denotes a rule storage unit, which stores rules defining predetermined read rules for performing prefetch, for example, various rules illustrated in FIG.

Reference numeral 42 denotes a read control unit which, when a read request for a certain image file is made, reads out the image file next to the certain image file based on the rules stored in the rule storage unit 41. Other image files associated with files with a high probability of being
Control is performed so that the data is expanded from the secondary storage device 44 to the high-speed secondary storage device 45 in advance.

For example, as described above, the read control unit 42 uses the file name of the image file requested to be read from the operating system based on the rules stored in the rule storage unit 41. Determine the file name of the data to be expanded in advance,
A process is transmitted to the data reading unit 43 together with the requested file name.

The data read unit 43 includes the file sharing system 34 and the low speed media driver 35 shown in FIG. 3, and is controlled from the low speed secondary storage device 44 to the high speed secondary storage device under the control of the read control unit 42 as described above. 45 to prefetch image files,
The image file is read from the secondary storage device 44 or the high-speed secondary storage device 45. When an image file is read from the low-speed secondary storage device 44, a process of copying the read image file to the high-speed secondary storage device 45 is also performed, as in a normal cache operation.

That is, the data reading section 43 operates as follows. When there is a read request for a certain image file, it is determined whether or not the image file is in a high-speed secondary storage device (for example, a cache memory configured in a partial area of a hard disk) 45. The corresponding image file is read from the next storage device 45.

On the other hand, if the requested image file is
When the image file does not exist in the next storage device 45, the image file is read from the low-speed secondary storage device (for example, a jukebox composed of a plurality of MOs) 44, and is associated with the rules stored in the rule storage unit 41. Another image file or a part of the image file is developed from the low-speed secondary storage device 44 to the high-speed secondary storage device 45 before an actual read request.

The operation of this embodiment configured to perform prefetching at the OS level as described above will be described in comparison with the case of performing prefetching at the application level. As shown in FIG. 5, in the case of prefetching by an application, when there is a read request for a certain data 1 at the very beginning, the data 1 is read by accessing a low-speed medium 53 such as an MO, and the data 1 is cached. The data 1 is copied to the memory 52 and passed to the application 51 to complete the data 1 reading process.

Next, the application 51 reads out the data 2 predicted from the data 1 from the low-speed medium 53 and prefetches it into the cache memory 52 based on a predetermined rule. By doing so, when a read request for data 2 is actually made thereafter, it is possible to access and read the cache memory 52 instead of the low-speed medium 53. As is clear from this, the prefetch operation is performed by the application 51 itself.

On the other hand, in the case of the present embodiment, when there is a read request for a certain data 1 first, the data 1 is read from the low-speed medium 53 and is copied to the cache memory 52. The processing to be passed to the application 51 is the same as described above, but the prefetch of the data 2 is performed by the OS or the prefetch driver 54 that has received the read request for the data 1. Therefore, according to the present embodiment, the application 51 itself does not perform prefetch, and the prefetch function can be easily externally added later.

FIG. 6 is a block diagram showing an example of the configuration of a computer system to which the data reading method of the present embodiment is applied. As shown in FIG. 6, in the present embodiment, image data is treated as an example of data, and large-scale image data (here, a large number of image files and a large data amount of each image file) is used. A disk library device such as a jukebox is used as a storage device for storing.

As shown in FIG. 6, the computer system according to the present embodiment includes a system control unit 61, a communication interface 62, an image input unit 63, an image display unit 64, a write / read control unit 65, a hard disk (HDD) 6
6 and a disk library device (juke box) 70.

Here, the jukebox 70 has a storage 70 for storing a plurality of magneto-optical disks (MO) 704.
3, a plurality of optical disk drives 701, and an automatic change mechanism 702. Optical disk drive 7
01 performs writing and reading processing of image data with respect to the MO 704 stored in the storage 703.

The auto-change mechanism 702 performs an exchange process of the MO 704 under the control of the write / read control unit 65. That is, the MO 704 specified by the write / read control unit 65 is taken out of the storage 703 and loaded into the corresponding optical disk drive 701, or the MO 704 loaded into the optical disk drive 701 specified by the write / read control unit 65 is stored in the storage 703. Or to store it in

Normally, each image data is stored in one MO
An image file 704 is stored as an image file. However, if the amount of image data constituting the image file exceeds the storage capacity of one MO 704, it is divided into a plurality of MOs 704 and stored. In the jukebox 70 configured as described above, the disc is exchanged in response to an instruction from the user, and the corresponding image file is read out via the write / read control unit 65.

The system control section 61 controls the entire computer system, and a program for that is stored in the HDD 66. The communication interface 62 communicates image data and commands with an external device (not shown) via a network. Also,
The image input unit 63 is for inputting image data to be stored in the jukebox 70.
An image input device such as an image scanner or a video camera using the image input device is used.

The image display section 64 displays the jukebox 70
This is for displaying image data read out from a device such as a CRT, LCD, or plasma display. The writing / reading control unit 65 includes the prefetch driver 33 and the data reading unit 43 shown in FIG. 4, reads an image file from the jukebox 70 or the HDD 66, and reads an image file read from the jukebox 70 into the HD.
A process for storing the data in D66 is performed.

The HDD 66 includes an application and an OS program in addition to a program for controlling the entire system, and a part of its storage area is used for performing a cache operation including a prefetch. It has become. In other words, the write / read control unit 65 is a HDD / HDD corresponding to the high-speed secondary storage device 45 in FIG.
66, and operates between the jukebox 70 corresponding to the low-speed secondary storage device 44 to perform the above-described rule-based prefetch.

Next, the operation of the computer system of the present embodiment configured as described above will be described. First,
When there is a request for reading an image file, the write / read control unit 65 determines whether the image file requested to be read is stored in the HDD 66 which is a high-speed secondary storage device.

When it is determined that the image file is not stored, the corresponding image file is read from the jukebox 70, and is supplied to the communication interface 62 and the image display unit 64, and is expanded in the cache area of the HDD 66. At this time, the write / read control unit 65
The other image files associated with the read image file are also read from the jukebox 70 and prefetched to the cache area of the HDD 66 in accordance with the rules stored in the HDD 66.

On the other hand, when it is determined that the image file requested to be read is stored in the HDD 66, the image file is read from the HDD 66 and supplied to the communication interface 62 and the image display unit 64. Here, even if this read request is for the first time, if the corresponding image file has already been prefetched as described above, instead of the jukebox 70, the HDD 6
6 can be accessed and read. Therefore, it is not necessary to wait for the processing time such as disk replacement, and the reading time can be greatly reduced.

In the embodiment of FIG. 6, a stand-alone computer system has been described as an example.
The storage device of the image file and the device to be used (read request device) are different, and they can be applied to a network system connected via a network, for example, a client server system.

In the embodiment shown in FIG. 6, the jukebox 70 has been described as an example of an image file storage device, but the present invention is not limited to this. In other words, the present invention can be similarly applied to any storage device that requires a longer time to read data than a high-speed secondary storage device. The data to be handled is not limited to image data, but can be similarly applied to any data that takes a long time to read.

[0070]

As described above, according to the present invention, when a data read request is made, other data associated with data having a higher probability of being read next to the above-mentioned data by a predetermined rule is transmitted at a low speed. Since the storage device is developed in advance to the high-speed storage device, even if the read request for a certain data is actually the first request, the low-speed storage device may read the data before the other data read request. The data expanded in the high-speed storage device can be read from the high-speed storage device. As a result, the speed of reading data from the low-speed storage device can be substantially increased.

According to another feature of the present invention, a reading method for expanding data in advance as described above is provided.
Since the operation is performed at the operating system level applied to the computer, there is no need to modify an existing application to incorporate such a pre-deployment function, and the application can be easily incorporated later into any application. As a result, a versatile data reading method or apparatus of the present invention that does not depend on the application to be applied can be provided.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a data reading method according to an embodiment.

FIG. 2 is a diagram showing an example of a rule used in the data reading method of the embodiment.

FIG. 3 is a conceptual diagram showing a mechanism of OS-level prefetch for realizing the data reading method of the embodiment.

FIG. 4 is a block diagram illustrating a functional configuration of a prefetch driver illustrated in FIG. 3;

FIG. 5 is a diagram for explaining the data reading method of the embodiment in more detail.

FIG. 6 is a block diagram illustrating a configuration example of a computer system to which the data reading method according to the embodiment is applied.

FIG. 7 is a diagram for explaining a normal cache operation.

FIG. 8 is a conceptual diagram showing a typical configuration example of a computer system.

[Explanation of symbols]

 31 application 32 OS (operating system) 33 prefetch driver 34 file sharing system 35 low-speed media driver 41 rule storage unit 42 read control unit 43 data read unit 44 low-speed secondary storage device 45 high-speed secondary storage device 65 write / read control unit 66 HDD (Hard Disk) 70 Jukebox

Claims (10)

[Claims] When a data read request is issued,
Data reading characterized in that other data associated with data having a high probability of being read next to the data according to a predetermined rule is previously expanded from the low-speed storage device to the high-speed storage device. Method. 2. When a data read request is issued,
It is determined whether or not the data is in the high-speed storage device. When the data is read from the high-speed storage device, when the data is not present, the data is read from the low-speed storage device. A data reading method, wherein other associated data is pre-developed from the low-speed storage device to the high-speed storage device. 3. The data reading method according to claim 1, wherein when reading data from the low-speed storage device, the read data is copied to the high-speed storage device. 4. A data reading method according to claim 1, wherein an operating system applied to a computer interprets the predetermined rule to execute the method. 5. The method according to claim 1, wherein a file name of data to be expanded is determined in advance based on a file name of data requested to be read from said operating system, and is supplied to a file system of said operating system. 5. The data reading method according to claim 4, wherein: 6. The data reading method according to claim 5, wherein the predetermined rule includes a rule for associating different data files or parts in the same data file with each other. 7. A rule storing means for storing a rule defining a predetermined reading rule, and when a data read request is issued, the rule storing means stores the rule after the data based on the rule stored in the rule storing means. A data reading device, comprising: reading control means for expanding other data associated as data having a high probability of being read from a low-speed storage device to a high-speed storage device in advance. 8. A rule storage means for storing a rule defining a predetermined read rule, and when a data read request is made, it is determined whether or not the data is in a high-speed storage device. The data is read from the device, and when there is no data, the data is read from the low-speed storage device, and other data associated as data having a high probability of being read next to the data according to the rule stored in the rule storage means is read from the low-speed storage device. A data reading device, comprising: reading control means for developing the data in a storage device in advance. 9. The rule stored in the rule storage means is a rule that specifies that an operating system applied to a computer performs a process of expanding data from the low-speed storage device to the high-speed storage device in advance. The data reading device according to claim 7, wherein the data reading device is provided. 10. The data reading device according to claim 7, wherein the low-speed storage device is a disk library device including a plurality of disks for storing data.