JP2012185686A - File system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a file system capable of recovering a file by using a simple configuration.SOLUTION: A file system according to the present invention is configured as below. The file system includes: a cache memory to temporarily store actual data and meta data of a file; a writing processing part to write the actual data and the meta data of the file on the cache memory onto an auxiliary storage; a file accessing part to access to the actual data of the file on the basis on the meta data; and a startup identification information acquisition part to acquire startup identification information that is issued from an information processing apparatus itself in which the file system is structured and different at every startup of the information processing apparatus. The writing processing part writes the actual data and the meta data of the file onto the auxiliary storage, adds the startup identification information that has been issued from the information processing apparatus itself and a writing completion information to identify an actual data part of the file that is completely written onto the auxiliary storage to the meta data of the file, and stores the meta data in the auxiliary storage.

Description

  The present invention relates to a file system, and more particularly to a file system having a data recovery function.

  When writing to the file system, in addition to the actual data, the metadata is also updated so that the position of the data on the disk can be determined. If a system failure occurs during the updating of metadata, there is a possibility of inconsistency in the metadata information. Therefore, the metadata update log is generally recorded in an area called a journal and then reflected on the disk. Confirm (commit). As a result, no contradiction occurs in the metadata, but the data is not guaranteed.

  Therefore, it is necessary to have a mechanism that guarantees the data that was being written on the application side. For example, in the NFS (Network File System) protocol, the NFS client needs to keep the transmitted data until the commit is completed. If a failure occurs on the NFS server, the NFS client It is specified to resend data.

JP-T-2003-513357

  However, if the processing resume function (resume function) is not implemented in the application, the user needs to create the resume function independently. Considering the effort of making it, it is practically difficult for the user to realize the resume function with awareness of the protocol. For this reason, for example, when backing up a file via CIFS (Common Internet File System), if a failure occurs on the backup server side, the backup is generally started again from the beginning of the file. In this case, however, the time loss increases when the file size becomes very large.

  Here, an example of a technique related to file restoration is disclosed in Patent Document 1. In the system disclosed in Patent Document 1, a data block is associated with an identifier for each transaction, and which identifier was activated when a system failure occurred is checked from the activated identifier list.

  However, when the above-described technique is used, it is necessary to explicitly start and execute a recovery process for all data using the identifier list. As a result, the recovery process itself becomes complicated, and problems such as the inability to access the file during the recovery process and an increase in the load on the file system occur, making it difficult to stably operate the file system.

  Therefore, an object of the present invention is to provide a file system that can restore a file with a simple configuration and can operate stably, which is the above-described problem.

A file system according to one aspect of the present invention is:
A cache memory for temporarily storing the actual data of the file and metadata including storage location information of the actual data of the file in order to perform processing on the file;
A write processing unit for writing real data and metadata of a file on the cache memory to the auxiliary storage device;
A file access unit for accessing the actual data of the file based on the metadata;
An activation identification information acquisition unit which is issued from the information processing apparatus itself in which the file system is constructed and acquires different activation identification information for each activation of the information processing apparatus,
The write processing unit writes the actual data and metadata of the file to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the file, and the auxiliary storage device Including a write completion information specifying the actual data portion of the file that has been written to, and storing the auxiliary storage device in the auxiliary storage device,
Take the configuration.

Moreover, the program which is the other form of this invention is:
In order to perform processing on a file, an information processing apparatus that includes a cache memory that temporarily stores real data of the file and metadata including storage location information of the real data of the file, and that configures the file system ,
A write processing unit for writing real data and metadata of a file on the cache memory to the auxiliary storage device;
A file access unit for accessing the actual data of the file based on the metadata;
An activation identification information acquisition unit which is issued from the information processing apparatus itself in which the file system is constructed and acquires different activation identification information for each activation of the information processing apparatus;
And realize
The write processing unit writes the actual data and metadata of the file to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the file, and the auxiliary storage device Storing in the auxiliary storage device including the write completion information for specifying the actual data portion of the file that has been written to
It is a program for realizing this.

In addition, a file processing method according to another aspect of the present invention includes:
In order to perform processing on the file, in the file system having a cache memory that temporarily stores the actual data of the file and the metadata including the storage location information of the actual data of the file,
Write the actual data and metadata of the file on the cache memory to the auxiliary storage device,
Access the actual file data based on the metadata,
When writing the actual data and metadata of a file to the auxiliary storage device, the file metadata is activated differently for each activation of the information processing device currently issued from the information processing device itself in which the file system is constructed Storing in the auxiliary storage device including identification information and write completion information specifying the actual data portion of the file that has been written to the auxiliary storage device,
Take the configuration.

  By adopting the above-described configuration, the present invention can provide a file system that can restore a file with a simple configuration and can operate stably.

It is a block diagram which shows the structure of the file system in Embodiment 1 of this invention. It is a figure which shows an example of the data structure of general metadata and the metadata in this embodiment. 4 is a flowchart illustrating file system operations in the first embodiment. FIG. 6 is a sequence diagram illustrating a file system operation in the first embodiment. FIG. 6 is a sequence diagram illustrating a file system operation in the first embodiment. FIG. 6 is a sequence diagram illustrating a file system operation in the first embodiment. FIG. 6 is a sequence diagram illustrating a file system operation in the first embodiment. It is a figure which shows the mode of the file before and after the rollback in Embodiment 1. FIG. It is a block diagram which shows the structure of the file system in Embodiment 2 of this invention. It is a figure which shows an example of the data structure of the file processed with the file system in Embodiment 2. It is a figure which shows the mode of the file before and after the rollback in Embodiment 2. FIG. It is a block diagram which shows the whole structure of the file system in Embodiment 2.

  A first embodiment of the present invention will be described with reference to FIGS. FIGS. 1 and 2 are diagrams for explaining the configuration of the file system, and FIGS. 3 to 8 are diagrams for explaining the operation of the file system.

[Constitution]
As shown in FIG. 1, the information processing system according to the first embodiment includes an application G1, a file system G2, and an OS (Operating System) G4. Note that the file system G2 and the OSG4 are constructed by incorporating a program into an information processing apparatus constituting the information processing system. The information processing system also includes a disk G3 as an auxiliary storage device.

  The application G1 operates in the same information processing apparatus in which the file system G2 is constructed. However, the application G1 may operate from another device via a network, such as NFS (Network File System) or CIFS (Common Internet File System). The application G1 does not need to hold a write log, but holds original data. Therefore, it is conceivable that the application G1 in the present embodiment has a function of backing up data via CIFS, for example. However, the application G1 is not limited to the one having the above-described function, and may have a function of accessing the disk G3 so as to write, read, and update a file as will be described later. .

  The OSG 4 is an activation ID payout processing unit that outputs an activation ID (activation identification information) that is identification information uniquely determined for each activation of the information processing apparatus in units of information processing apparatuses or file systems in which a file system is constructed. Has B0. Specifically, the activation ID payout processing unit B0 pays out the updated new activation ID when the information processing apparatus is activated and restarted. The activation ID may be, for example, a system activation time, a certain file system mount time, and the like, but is not limited thereto.

  As shown in FIG. 1, the file system G1 includes an interface unit A0, an inode copy processing unit C0, a cache management unit D0, and a disk access unit E0. Further, it has a metadata storage area F1 and a data cache area F2 formed in a cache memory for temporarily storing data.

  The interface unit A0 includes an activation ID acquisition processing unit A1 (activation identification information acquisition unit) that acquires the activation ID paid out from the activation ID payout processing unit B0. In addition, the interface unit A0 includes an open processing unit A2, a reading processing unit A3, a writing processing unit A4, and a closing processing unit A5 as a configuration that functions as a file access unit that exchanges for the application G1 to access a file. . Each of the processing units A2 to A5 first accesses inode information that is metadata of a file, and accesses actual data of the file based on the inode information. The functions of each of the processing units A2 to A5 will be described in detail when the operation is described later.

  The inode copy processing unit C0 functions as a file access unit in cooperation with the above-described open processing unit A2, read processing unit A3, write processing unit A4, and close processing unit A5. When accessing, that is, when inode information of a file such as an inode is read from the disk G3, the inode information is rewritten and stored in the metadata storage area F1. As shown in FIG. 1, the inode copy processing unit C0 includes a restart determination processing unit C1, a commit information check processing unit C2, and an inode rewrite processing unit C3. The inode processing unit C0 Details of the inode rewriting process will be described in detail when the operation is described later.

  The cache management unit D0 is a process for preventing a cache from being accumulated in the data cache area F2 and writing to be delayed. When a certain amount of time has elapsed or when a certain amount of data has accumulated in the cache area, the application G1 When a commit request is issued, the disk access unit E0 is requested to reflect the actual data of the file in the data cache area F2 to the disk G3. In the following, the process of reflecting the data in the data cache area F2 on the disk G3 is referred to as “commit”. Accordingly, the cache management unit D0 functions as a write processing unit that writes the actual data and inode information of the file to the disk together with the write processing unit A4.

  The disk access unit E0 performs processing for writing data to the disk G3 and reading data in the disk G3, and functions as an interface between the file system G2 and the disk G3.

  The data cache area F2 is used to improve the performance of writing from the application G1 to the disk G3, and in particular, is an area of a cache memory that temporarily stores the actual data of the file.

  The metadata storage area F2 is a temporary cache memory area for storing file inode information. Here, an example of inode information will be described with reference to FIG. As shown in FIG. 2B, the inode information in this embodiment includes “file size” information indicating the size of the entire file being processed in the data cache area F2, that is, the actual data size of the file, and corresponding file assistance. And “data storage location” information indicating a storage location for the storage device. The inode information includes “owner ID”, “group ID”, “inode number”, “permission”, information regarding the corresponding file, “last file update time, last inode update time, last reference time”, etc. Time stamp ".

  Furthermore, the inode information in the present embodiment includes the “activation ID” acquired by the activation ID acquisition processing unit A1 described above, as shown in FIG. The inode information in the present embodiment includes “committed size” indicating the size of the actual data portion of the file that has been written to the disk G3. These “activation ID” and “committed size” are information that is not included in the inode information of the file used in the general file system shown in FIG. The “activation ID” and the “committed size” are updated when the actual data of the file is written to the data cache area F2 or the disk G3. Here, the “committed size” is not necessarily limited to the information indicating the size of the data. If the information specifies the portion of the actual data of the file that has been written to the disk G3, other information It may be information.

[Operation]
Next, the operation of the information processing system described above will be described. First, referring to FIG. 3, the operation when a file is already stored in the disk G3 and the file is referred to or updated from the disk G3 will be described.

  First, when the application G1 tries to refer to or update a file, the inode information of the file is referred to associate the path name of the file with the position information stored on the disk. At this time, if the inode information necessary for access is in the metadata storage area F1, it is used (Yes in step S1 and step S2). If not (No in step S2), the inode information from the disk G3 is used. Read information.

  Subsequently, in order to read inode information from the disk G3, the interface unit A0 executes the inode copy processing unit C0 (step S3). The inode copy processing unit C0 requests the disk access unit E0 to read necessary inode information from the disk G3 into the metadata storage area F1 (step S4).

  Subsequently, when the inode information can be read into the metadata storage area F1, the “activation ID” in the inode information and the “activation ID” of the current information processing apparatus or file system acquired by the activation ID acquisition processing unit A1. Are compared with each other (step S5). This is a comparison for determining whether the previous writing was performed before or after the system was restarted. Accordingly, if the “startup IDs” match (Yes in step S6), it can be said that the previous writing is after the restart of the system, and that no system failure has occurred after writing the data. I can say that. The activation ID acquisition unit A1 may acquire the “activation ID” of the system once after the system is activated. For example, when the inode copy process C0 is first executed after the system is activated, Read from the system startup script.

  If the “startup ID” acquired by the system does not match the “startup ID” on the inode (No in step S6), a system failure occurred during the previous data writing. there is a possibility. Therefore, the “file size” in the inode information is compared with the “committed size” (step S7), and after the commit, which is the process of writing the actual data of the file to the disk G3, is completed, the data writing is completed. Check if it has occurred.

  If the “boot ID” of the system does not match the “boot ID” on the inode, and the “file size” does not match the “committed size” (No in step S8), the file roll In order to execute the back process, the value of “file size” in the inode information is overwritten with the value of “committed size” (step S9).

  Subsequently, when the value of “file size” in the inode information is overwritten with the value of “committed size”, the file size becomes small, and the data block that has been used until now may not be used. Therefore, in accordance with the change of the file size, the data storage location described in the inode is corrected, and the information of the metadata data block bitmap is changed to unused (step S10).

  Thereafter, the changed inode information and other metadata values are stored in the metadata storage area F1 (step S11). Then, the inode copy processing unit C0 notifies the interface unit A0 that the processing has been completed, and the open processing unit A2, the reading processing unit A3, and the like can access the file that has been rolled back to the point of commit. (Step S12).

  The state of the file by performing the above-described processing will be described with reference to FIG. First, FIG. 8A is a diagram showing the relationship between inode information and actual file data in a general file system. As shown in this figure, in the general file system, since the actual data of the file is accessed according to the “file size”, an uncommitted part remains and access is impossible. On the other hand, in the file system according to the present embodiment, as shown in the upper part of FIG. 8B, before the processing by the inode copy processing unit C0 described above, the “file size” reaches the “data (uncommitted)” part. Although it is included, since the “committed size” is overwritten with the “file size” after the processing by the inode copy processing unit C0, the file itself is committed as shown in the lower part of FIG. 8B. Up to the data part that has been completed. In other words, the dotted line portion shown in the lower part of FIG. 8B remains on the disk G3 but does not exist. Therefore, rollback to the point where the commit is completed and access the file. Is possible.

  Next, operations of “file open”, “file write (asynchronous)”, “file write (synchronous)”, “file read”, and “file close” by the file system will be described.

(Open file)
The operation of the “file open” process will be described with reference to FIG.
1. The application G1 requests the interface unit A0 to open a file.
2. The interface unit A0 passes the processing to the open processing unit A2. The open processing unit A2 confirms whether the requested inode of the corresponding file exists in the metadata storage area F1. If it exists, the procedure proceeds to the following procedure 4. If not, the procedure proceeds to the procedure 3.

3. The open processing unit A2 executes processing by the inode copy processing unit C0. The processing by the inode copy processing unit C0 performs the same processing as described above, and notifies the open processing unit A2 that the processing has been completed.
4). The open processing unit A2 performs a file open process based on the inode information in the metadata storage area F1, and notifies the application G1 that the process has been completed when the process is completed.

(File writing (asynchronous))
The operation of the “file write (asynchronous)” process will be described with reference to FIG.
1. The application G1 transmits data to be written to the interface unit A0.
2. The interface unit A0 passes the processing to the write processing unit A4. The write processing unit A4 confirms whether the inode of the requested file exists in the metadata storage area F1. If it exists, the procedure proceeds to the following procedure 4. If not, the procedure proceeds to the procedure 3. Even if the file can be opened, inode information necessary for releasing the memory may have been cleared from the metadata storage area F1, so the presence check process of inode is performed.

3. The write processing unit A4 executes processing by the inode copy processing unit C0. The process by the inode copy processing unit C0 performs the same process as described above, and notifies the write processing unit A4 that the process has been completed.
4). The write processing unit A4 writes the received write data (actual data of the file) in the data cache area F2 based on the inode information in the metadata storage area F1. At this time, it is necessary to change the inode information in accordance with the data to be written. At this time, the current “activation ID” information held in the interface unit A0 is reflected in the inode information.
5. When the update of the inode in the metadata storage area F1 and the writing of the actual data of the file to the data cache area F2 are completed, the writing processing unit A4 notifies the application G1 of the completion of writing.

(Write file (synchronized))
The operation of the “file writing (synchronization)” process will be described with reference to FIG.
1. The application G1 transmits data to be written, which is actual file data, to the interface unit A0.
2. The interface unit A0 passes the processing to the write processing unit A4. The write processing unit A4 confirms whether the inode of the requested file exists in the metadata storage area F1. If it exists, the procedure proceeds to the following procedure 4. If not, the procedure proceeds to the procedure 3. Even if the file can be opened, inode information necessary for releasing the memory may have been cleared from the metadata storage area F1, so the presence check process of inode is performed.

3. The write processing unit A4 executes processing by the inode copy processing unit C0. The inode copy processing unit C0 performs the same processing as described above, and notifies the write processing unit A4 that the processing has been completed.
4). Based on the inode information in the metadata storage area F1, the write processing unit A4 writes the actual data of the file that is the received write data to the data cache area F2. At this time, it is necessary to change the inode information in accordance with the data to be written. At that time, the information of the “activation ID” held in the interface unit A0 is reflected in the inode.

5. When the update of the inode information in the metadata storage area F1 and the writing of data to the data cache area F2 are completed, the write processing unit A4 requests the disk access unit E0 to reflect on the disk.
6). When the reflection to the disk G3 is completed, the disk access unit E0 notifies the write processing unit A4.
7). The write processing unit A4 overwrites the value of “committed size” in the inode in the metadata storage area F1 with the value of “file size”.
8). The write processing unit A4 requests the disk access unit E0 to reflect the inode in the metadata storage area F1 on the disk G3.
9. When the reflection to the disk G3 is completed, the disk access unit E0 notifies the write processing unit A4.
10. The write processing unit A4 notifies the application G1 of the completion of writing.

(File reading)
With reference to FIG. 7, the operation of the “file read” process will be described.
1. The application G1 requests the interface unit A0 to read a file.
2. The interface unit A0 passes the processing to the reading processing unit A3. The read processing unit A3 confirms whether the requested inode of the corresponding file exists in the metadata storage area F1. If it exists, the procedure proceeds to the following procedure 4. If not, the procedure proceeds to the procedure 3.

3. The read processing unit A3 executes the process of the inode copy processing unit C0. The inode copy processing unit C0 performs the same processing as described above, and notifies the reading processing unit A3 that the processing has been completed.
4). The read processing unit A3 requests the disk access unit E0 to read data based on the inode information in the metadata storage area F1.
5. When the disk access unit E0 reads data from the disk G3, the disk access unit E0 returns the data to the read processing unit A3. Normally, the read data is generally mounted using a cache in the same manner as at the time of writing. However, here, an explanation is omitted, and an example in which the data is directly transferred to the read processing unit A3 is described.
6). The read processing unit A3 notifies the application G1 of the completion of writing.

(File close)
Next, the operation of the “file close” process will be described.
1. The application G1 requests the interface unit A0 to close the file.
2. The interface unit A0 passes the process to the close processing unit A5. The close processing unit A5 confirms whether the requested inode of the corresponding file exists in the metadata storage area F1. If it exists, the procedure proceeds to the following procedure 4. If not, the procedure proceeds to the procedure 3.

3. The close processing unit A5 executes processing by the metadata copy processing unit C0. The processing by the inode copy processing unit C0 performs the same processing as described above, and notifies the close processing unit A5 that the processing has been completed.
4). The close processing unit A5 requests the disk access unit E0 to reflect the data in the data cache area F2 on the disk G3 based on the inode information in the metadata storage area F1.

5. When the reflection to the disk G3 is completed, the disk access unit E0 notifies the close processing unit A5.
6). The close processing unit A5 overwrites the value of “committed size” in the inode in the metadata storage area F1 with the value of “file size”.
7). The close processing unit A5 requests the disk access unit E0 to reflect the inode in the metadata storage area F1 on the disk G3.
8). When the reflection to the disk G3 is completed, the disk access unit E0 notifies the close processing unit A5.
9. The close processing unit A5 notifies the application G1 of the completion of the close.

(Asynchronous reflection of data)
When the asynchronous writing described above is performed, a process for reflecting the data in the data cache area F2 to the disk G3 is separately required. Hereinafter, the processing will be described.
1. When a certain amount of time has elapsed or when a certain amount of data has accumulated in the data cache area F2, a request is made from the cache management unit D0 to the disk access unit E0 to write data in the data cache area F2 to the disk G3. To do.
2. The disk access unit E0 reads the caches in the metadata storage area F1 and the data cache area F2 and reflects them in the disk G3.
3. After reflecting the data and metadata such as inode, only the “committed size” of the inode is updated, and the inode is reflected on the disk G3 again.

(Resume processing)
Next, processing when the application G1 resumes write data will be described.
1. The application G1 requests the interface unit A0 to open a file. At this point, it is assumed that no failure has occurred in the system.
2. The interface unit A0 passes processing to the open processing unit A2, and executes processing by the inode copy processing unit C0 from the open processing unit A2.

3. Since no failure has occurred at this point, the inode information in the disk G3 is copied as it is to the metadata storage area F1.
4). The inode copy processing unit C0 notifies the completion to the open processing unit A2, and the open processing unit A2 notifies the open completion to the application G1.
5. The application G1 transmits data to the interface unit A0.
6). The interface unit A0 passes the processing to the write processing unit A4.
7). The write processing unit A4 writes the received write data in the data cache area F2 based on the inode information in the metadata storage area F1. At this time, the inode information is also changed according to the data to be written. In addition, in the inode update, the “startup ID” information is reflected in the inode.

8). Thereafter, when a system failure occurs, the system is restarted. At this time, the application G1 also ends abnormally.
9. After restarting the system, the file size is checked to resume the data written by the application G1. In order to confirm the file size, the application G1 requests opening of the file.
10. The interface unit A0 passes the processing to the open processing unit A2. The open processing unit A2 confirms whether the inode of the corresponding file exists in the metadata storage area F1. However, at this time, it is assumed that no inode exists in the metadata storage area F1.

11. The open processing unit A2 executes processing by the inode copy processing unit C0.
12 The inode copy processing unit C0 requests the disk access unit E0 to read necessary inode information from the disk G3.
13. When the inode can be read, the system "boot ID" is compared with the "boot ID" on the inode. Since the inode has been updated before the system is restarted, the “boot ID” of the system and the “boot ID” on the inode have different values.
14 Next, “file size” and “committed size” in the inode information are compared. Since a failure occurred during writing, “file size” and “committed size” have different values. Although a failure may occur immediately after the commit processing is completed, it is not considered in this example because the timing is generally extremely rare.
15. The system “boot ID” and the “boot ID” on the inode do not match, and the “file size” and “committed size” do not match. Overwrite the value of "completed size".

16. If the inode “file size” value is overwritten with the “committed size” value, the file size will be reduced, so the data storage location information described in the inode will be updated as the file size changes. Modify and change the metadata data block bitmap information to unused.
17. Information on the inodes and other metadata changed in steps 15 and 16 are stored in the metadata storage area F1.
18. The inode copy processing unit C0 notifies the open processing unit A2 that the processing has been completed.

19. The open processing unit A2 executes file open processing. When the open process is completed, the application G1 is notified that the open process has been completed.
20. The application G1 requests a file size reference.
21. The interface unit A0 notifies the file size from the inode information in the metadata storage area F1.
22. In order to resume data writing, the application G1 recognizes the received file size value as a file, and starts writing the previous data from the file size value.

  As described above, according to the present embodiment, unguaranteed data is deleted at the time of accessing such data, so the only data that can be finally referred to is guaranteed (committed) data. . Then, when copying the metadata from the disk to the memory at the time of accessing the data, the rollback process is performed by rewriting the metadata on the memory, so that the data rollback process does not take time.

  Furthermore, according to the present embodiment, since it appears that only guaranteed data is stored in the disk G3 from the application G1, if the file size is confirmed and only the subsequent data is written, the application is resumed. Even if the write log is not prepared in G1, the data write can be resumed easily.

<Embodiment 2>
Next, a second embodiment of the present invention will be described with reference to FIGS. 9 and 12 are block diagrams showing the configuration of the present invention, and FIGS. 10 and 11 are diagrams showing the data structure of files in the present invention.

[Constitution]
The file system according to the present embodiment has a deduplication function that divides a file before storing it and eliminates that the same data is stored redundantly for the divided data. In a file system having such a deduplication function, a rollback function described below is introduced to prevent a reduction in the deduplication rate.

  As shown in FIG. 9, the information processing system according to the present embodiment includes an OSQ2, a file system Q3, a cluster function Q5, a disk Q4, and an application Q1. As shown in FIG. 12, the information processing system according to the present embodiment has a configuration in which a plurality of information processing apparatuses each constituting a file system are connected via a network.

  The cluster function Q5 is used to switch access from the outside to another device for load distribution or the like. Switching is realized by executing the file system termination script R2 on the device before switching and executing the file system activation script R1 on the device after switching.

  As in the first embodiment, the OSQ 2 includes the activation ID payout processing unit P0, and issues an “activation ID” that is updated every time the system is activated.

  The file system Q3 includes an interface unit L0, a partition management unit N1, a disk access unit N2, a metadata storage area M1, a data cache area M2, and a reserved data cache area M3. The interface unit L0 includes an activation ID acquisition processing unit L1, a close processing unit L2, an open processing unit L3, a read processing unit L4, a write processing unit L5, and a cache deletion processing unit L6. Hereinafter, the configuration different from the first embodiment will be mainly described in detail.

  The cache deletion processing unit L6 is called from the file system end script R2 of the cluster function Q5 described above, and executes cache deletion processing.

  The division management unit N1 divides one file into a plurality of files according to a specific rule at the time of writing and stores them in the disk Q4, and can read the data divided into the plurality of files as one original file at the time of reading. The process to do is done. For example, when writing a file, the file shown in FIG. 10A is divided into each divided file such as “a1” and “b1” as shown in FIG. Each divided file is grouped as divided data D2 by type according to the rule and stored in the disk Q4. At the same time, in correspondence with each divided file, a position information file D1 composed of each metadata including storage position information of the divided file with respect to the disk Q4 and position information with respect to the file before the division of the divided file is generated. Store in Q4. At the time of reading, each divided file corresponding to each metadata is read from the storage position information included in each metadata of the position information file D1, and the file is restored and read based on the position information with respect to the original file.

  As described above, each metadata constituting each entry of the position information file D1 includes the corresponding divided file type (actual data) in addition to the storage position information of the corresponding divided file and the position information with respect to the file before the division. / Non-data), the size of the corresponding split file, and information such as CRC.

  The division management unit N1 uses a content address unique to the data content of the divided file as storage location information of the corresponding divided file included in the metadata. Accordingly, since the divided files having the same data contents have the same content address, it is only necessary to store one divided file on the disk Q4. That is, when another divided file having the same data content is written, the duplicate storage can be eliminated by referring to the storage position that is the content address of the already stored divided file having the same data content.

  The reserved data cache area M3 is a temporary cache area that is used for holding what could not be determined by the rules for dividing the file and determining it together with the data coming later. is there.

In addition, the system in this embodiment has the following differences compared with Embodiment 1. FIG.
1. First, when a file (actual data) is stored in a file system using an application such as archive software, information such as a date (non-data) for management by the archive software may be assigned to the actual data and stored. Then, even if there is no actual data update, if non-data update is performed, it is handled as a separate file for the file system. For this reason, even if the file system has a deduplication function, deduplication may not be performed well. In order to perform deduplication efficiently even in such a situation, the present embodiment has a function of efficiently performing deduplication by dividing and storing actual data and non-data. Since there is already a technique for dividing and storing a plurality of files, a detailed description thereof will be omitted.

2. In the present embodiment, each piece of metadata including storage location information of each divided file includes “commit information” and “activation ID” indicating whether or not the corresponding divided file has been committed.
3. In the first embodiment, when the inode that is the metadata of the file is stored in the metadata storage area F1, it is stored after being rewritten by the processing by the inode copy processing unit C0. However, in this embodiment, the inode is divided. In order to make the stored data look like a single file, the division management unit N1 restores the original data to the original file state.

4). In this embodiment, the pending data cache area M3 is provided so that the data in the data cache area M2 can be divided in a meaningful manner when stored in the disk Q4.
5. In this embodiment, a cluster function Q5 and a cache deletion processing unit L6 called from the file system end script R2 of the cluster function Q5 are provided.

[Operation]
Next, an outline of the operation of the above-described system will be described. First, when the application Q1 tries to refer to or update a file, if the file is divided into a plurality of divided files and stored in the disk Q4, the divided file must be restored and processed. Therefore, in order to access metadata and data from the interface unit L0, all access is performed through the division management unit N1.
At this time, each piece of metadata, which is information related to each divided file that is the actual data of the file, includes “commit information” and “activation ID” that indicate whether or not writing of the corresponding divided data to the disk G4 has been completed. Is remembered.
It is possible to divide the file and write it to the disk by encrypting the data or performing deduplication. However, when processing the written data, the data cache area has enough data. Otherwise, information on how to divide may not be obtained. In such a case, effective division can be performed by temporarily storing data that could not be divided into the reserved data cache area M3 and combining it with the data to be written next.

  However, if there is a commit request in the state where data exists in the pending data cache area M3, it is necessary to write to the disk even if there is not enough information to divide. In this case, if the file system performs deduplication, the deduplication rate decreases. In general, if a file system can be accessed from another device for load distribution, it is necessary to stop all access on the device used before load distribution, and at this time, a commit request comes.

  As described above, since the deduplication rate is lowered simply by repeatedly performing load distribution, in this embodiment, when switching is performed by load distribution or the like, a process of discarding all cache data without performing commit is performed. Do.

  Next, the “commit information registration” process will be described. In this embodiment, when there is a commit request, after writing the divided file to the disk, the metadata corresponding to the divided file is updated, and the commit is set by setting a flag indicating that the metadata is committed. Register “commit information” indicating the completion of. For example, “commit information” is registered in the metadata as indicated by the black circles in FIG.

  Next, the “rollback” process will be described. In order to read a file from the disk Q4, it is necessary to refer to the position information file D1 including a metadata group corresponding to the file. First, the “activation ID” in the metadata in the position information file D1 is It is checked whether or not it coincides with the current “activation ID” acquired by the activation ID acquisition processing unit L1. If they do not match, there may be a case where the file changes made before the failure has not been committed. Therefore, “commit information” in each piece of metadata in the position information file D1 is subsequently searched. At this time, the commit information in each metadata is searched from the end side of the position information file D1. Note that commit information can be quickly confirmed by setting each metadata, which is each entry in the position information file, to a fixed length.

  When the position information file D1 is expanded on the metadata storage area M1, it is handled that there is no metadata after the last commit point. That is, when “commit information” is detected from the metadata in the location information file D1, metadata after the metadata located at the end of the location information file D1 among the metadata in which the “commit information” is detected is “ The divided file corresponding to the metadata that has become “0” is also changed to “0”. At this time, the divided file having no corresponding metadata is also changed to “0”. In other words, the interface unit L0 such as the open processing unit L3 and the reading processing unit L4 includes “commit information” that is a black circle as seen from the top of the position information file D1, as shown in FIG. A file restored up to the split file corresponding to the last metadata is made accessible. Specifically, in the example of FIG. 11B, it is handled that there is no data of the dotted line portion.

Next, the cache deletion process will be described with reference to FIGS.
1. The file system end script R2 of the cluster function Q5 is executed for switching load distribution.
2. The file system end script R2 calls processing by the cache deletion processing unit L6 of the interface unit L0.
3. The cache deletion processing unit L6 stops all access to the interface unit L0. This is to prevent receiving an additional write or commit request from the application Q1.
4). When all accesses are stopped, the data in the data cache area M2 and the data in the reserved data cache area M3 are all deleted. That is, no writing is performed on the disk Q4.
5. When all the cache data has been deleted, the remaining processing of the file system termination script R2 is executed. As a result, the file system is unmounted.
6). The file system activation script R1 is executed on the destination device that is switched by load distribution. As a result, the file system is mounted.

  As a result, the data in the reserved data cache area M3 is not written to the disk, and there is no need to forcibly write to the disk Q4 in a state where it cannot be divided. Therefore, it is possible to prevent the deduplication rate from decreasing. In addition, the data is guaranteed because the data rollback is executed in the switching destination device.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The configuration of the file system and the like in the present invention will be described below. However, the present invention is not limited to the following configuration.

(Appendix 1)
A cache memory for temporarily storing the actual data of the file and metadata including storage location information of the actual data of the file in order to perform processing on the file;
A write processing unit for writing real data and metadata of a file on the cache memory to the auxiliary storage device;
A file access unit for accessing the actual data of the file based on the metadata;
An activation identification information acquisition unit which is issued from the information processing apparatus itself in which the file system is constructed and acquires different activation identification information for each activation of the information processing apparatus,
The write processing unit writes the actual data and metadata of the file to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the file, and the auxiliary storage device Including a write completion information specifying the actual data portion of the file that has been written to, and storing the auxiliary storage device in the auxiliary storage device,
File system.

(Appendix 2)
The file system according to attachment 1, wherein
The file access unit reads the metadata from the auxiliary storage device into the cache memory when the metadata of the file is not stored in the cache memory when accessing the file, and the metadata in the metadata When the activation identification information is different from the activation identification information currently issued from the information processing device itself, the file of the actual data part specified by the write completion information in the metadata of the file is accessed.
File system.

(Appendix 3)
The file system according to appendix 2,
The write processing unit includes file size information indicating the size of the actual data of the file on the cache memory in the metadata of the file, and the real data portion of the file that has been written to the auxiliary storage device. Including the size as the completion of writing information, storing file metadata in the auxiliary storage device,
The file access unit is different from the activation identification information currently issued from the information processing apparatus itself in the metadata of the file read from the auxiliary storage device to the cache memory, and in the metadata When the value of the written completion information is different from the value of the file size information in the metadata, the value of the written information is overwritten on the file size.
File system.

(Appendix 4)
The file system according to appendix 3,
The file access unit accesses a file of an actual data part having a size corresponding to the file size information in the metadata of the file read from the auxiliary storage device to the cache memory;
File system.

(Appendix 5)
The file system according to appendix 2,
For each divided file obtained by dividing the actual data of the file, the writing processing unit stores the actual data of the divided file, the storage location information of the divided file with respect to the auxiliary storage device, and the file before the division of the divided file. Metadata including location information is written to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the divided file and the auxiliary storage device of the divided file Having the function of storing in the auxiliary storage device including the writing completion information indicating that writing has been completed,
File system.

(Appendix 6)
The file system according to appendix 5,
The file access unit, when the activation identification information in the metadata corresponding to the divided file of the file read from the auxiliary storage device to the cache memory is different from the activation identification information currently issued from the information processing device itself In addition, among the metadata corresponding to each divided file of the file read from the auxiliary storage device to the cache memory, the metadata corresponding to the last metadata including the written completion information as seen from the top of the file Access the file of the actual data part up to the split file to be
File system.

(Appendix 7)
In order to perform processing on a file, an information processing apparatus that includes a cache memory that temporarily stores real data of the file and metadata including storage location information of the real data of the file, and that configures the file system ,
A write processing unit for writing real data and metadata of a file on the cache memory to the auxiliary storage device;
A file access unit for accessing the actual data of the file based on the metadata;
An activation identification information acquisition unit which is issued from the information processing apparatus itself in which the file system is constructed and acquires different activation identification information for each activation of the information processing apparatus;
And realize
The write processing unit writes the actual data and metadata of the file to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the file, and the auxiliary storage device Storing in the auxiliary storage device including the write completion information for specifying the actual data portion of the file that has been written to
A program to make things happen.

(Appendix 8)
The program according to appendix 7,
The file access unit reads the metadata from the auxiliary storage device into the cache memory when the metadata of the file is not stored in the cache memory when accessing the file, and the metadata in the metadata When the activation identification information is different from the activation identification information currently issued from the information processing device itself, the file of the actual data part specified by the write completion information in the metadata of the file is accessed.
program.

(Appendix 9)
In order to perform processing on the file, in the file system having a cache memory that temporarily stores the actual data of the file and the metadata including the storage location information of the actual data of the file,
Write the actual data and metadata of the file on the cache memory to the auxiliary storage device,
Access the actual file data based on the metadata,
When writing the actual data and metadata of a file to the auxiliary storage device, the file metadata is activated differently for each activation of the information processing device currently issued from the information processing device itself in which the file system is constructed Storing in the auxiliary storage device including identification information and write completion information specifying the actual data portion of the file that has been written to the auxiliary storage device,
File processing method.

(Appendix 10)
A file processing method according to attachment 9, wherein
When accessing the file, if the metadata of the file is not stored in the cache memory, the metadata is read from the auxiliary storage device to the cache memory, and the activation identification information in the metadata is information When different from the activation identification information currently issued from the processing device itself, to access the file of the actual data portion specified by the write completion information in the metadata of the file,
File processing method.

A0 interface unit A1 activation ID acquisition processing unit A2 open processing unit A3 read processing unit A4 write processing unit A5 close processing unit B0 activation ID issue processing unit C0 inode copy processing unit C1 restart determination processing unit C2 commit information check processing unit C3 inode rewrite processing unit D0 cache management unit E0 disk access unit F1 metadata storage area F2 data cache area G1 application G2 file system G3 disk G4 OS
L0 interface unit L1 activation ID acquisition processing unit L2 close processing unit L3 open processing unit L4 read processing unit L5 write processing unit L6 cache deletion processing unit M1 metadata storage area M2 data cache area M3 reserved data cache area N1 partition management unit N2 disk Access unit P0 Startup ID issue processing unit Q1 Application Q2 OS
Q3 File system Q4 Disk Q5 Cluster function R1 File system start script R2 File system end script

Claims (10)

A cache memory for temporarily storing the actual data of the file and metadata including storage location information of the actual data of the file in order to perform processing on the file;
A write processing unit for writing real data and metadata of a file on the cache memory to the auxiliary storage device;
A file access unit for accessing the actual data of the file based on the metadata;
An activation identification information acquisition unit which is issued from the information processing apparatus itself in which the file system is constructed and acquires different activation identification information for each activation of the information processing apparatus,
The write processing unit writes the actual data and metadata of the file to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the file, and the auxiliary storage device Including a write completion information specifying the actual data portion of the file that has been written to, and storing the auxiliary storage device in the auxiliary storage device,
File system. The file system according to claim 1,
The file access unit reads the metadata from the auxiliary storage device into the cache memory when the metadata of the file is not stored in the cache memory when accessing the file, and the metadata in the metadata When the activation identification information is different from the activation identification information currently issued from the information processing device itself, the file of the actual data part specified by the write completion information in the metadata of the file is accessed.
File system. The file system according to claim 2,
The write processing unit includes file size information indicating the size of the actual data of the file on the cache memory in the metadata of the file, and the real data portion of the file that has been written to the auxiliary storage device. Including the size as the completion of writing information, storing file metadata in the auxiliary storage device,
The file access unit is different from the activation identification information currently issued from the information processing apparatus itself in the metadata of the file read from the auxiliary storage device to the cache memory, and in the metadata When the value of the written completion information is different from the value of the file size information in the metadata, the value of the written information is overwritten on the file size.
File system. The file system according to claim 3,
The file access unit accesses a file of an actual data part having a size corresponding to the file size information in the metadata of the file read from the auxiliary storage device to the cache memory;
File system. The file system according to claim 2,
For each divided file obtained by dividing the actual data of the file, the writing processing unit stores the actual data of the divided file, the storage location information of the divided file with respect to the auxiliary storage device, and the file before the division of the divided file. Metadata including location information is written to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the divided file and the auxiliary storage device of the divided file Having the function of storing in the auxiliary storage device including the writing completion information indicating that writing has been completed,
File system. The file system according to claim 5,
The file access unit, when the activation identification information in the metadata corresponding to the divided file of the file read from the auxiliary storage device to the cache memory is different from the activation identification information currently issued from the information processing device itself In addition, among the metadata corresponding to each divided file of the file read from the auxiliary storage device to the cache memory, the metadata corresponding to the last metadata including the written completion information as seen from the top of the file Access the file of the actual data part up to the split file to be
File system. In order to perform processing on a file, an information processing apparatus that includes a cache memory that temporarily stores real data of the file and metadata including storage location information of the real data of the file, and that configures the file system ,
A write processing unit for writing real data and metadata of a file on the cache memory to the auxiliary storage device;
A file access unit for accessing the actual data of the file based on the metadata;
An activation identification information acquisition unit which is issued from the information processing apparatus itself in which the file system is constructed and acquires different activation identification information for each activation of the information processing apparatus;
And realize
The write processing unit writes the actual data and metadata of the file to the auxiliary storage device, and the activation identification information currently issued from the information processing device itself to the metadata of the file, and the auxiliary storage device Storing in the auxiliary storage device including the write completion information for specifying the actual data portion of the file that has been written to
A program to make things happen. The program according to claim 7,
The file access unit reads the metadata from the auxiliary storage device into the cache memory when the metadata of the file is not stored in the cache memory when accessing the file, and the metadata in the metadata When the activation identification information is different from the activation identification information currently issued from the information processing device itself, the file of the actual data part specified by the write completion information in the metadata of the file is accessed.
program. In order to perform processing on the file, in the file system having a cache memory that temporarily stores the actual data of the file and the metadata including the storage location information of the actual data of the file,
Write the actual data and metadata of the file on the cache memory to the auxiliary storage device,
Access the actual file data based on the metadata,
When writing the actual data and metadata of a file to the auxiliary storage device, the file metadata is activated differently for each activation of the information processing device currently issued from the information processing device itself in which the file system is constructed Storing in the auxiliary storage device including identification information and write completion information specifying the actual data portion of the file that has been written to the auxiliary storage device,
File processing method. The file processing method according to claim 9, wherein
When accessing the file, if the metadata of the file is not stored in the cache memory, the metadata is read from the auxiliary storage device to the cache memory, and the activation identification information in the metadata is information When different from the activation identification information currently issued from the processing device itself, to access the file of the actual data portion specified by the write completion information in the metadata of the file,
File processing method.

Images