CN106095817A - Extensible file system based on micro-kernel and file access method - Google Patents

Abstract

The invention discloses an extensible file system based on a microkernel, including disk services for reading and writing disks; for copying file data between user processes and file service processes, and helping disk services execute privileged IO instructions in kernel mode system services; directory service process: work on a single processing core, manage global metadata, set block cache to save file attribute information inode and directory information dentry, receive all file processing requests from user processes, execute directory traversal, and process all Describe the request for operating metadata in the file processing request, and route the request for reading and writing files to the file service process; multiple file service processes: used to process requests for reading and writing files, and set a cache pool for caching data file blocks . The invention also discloses a file access method of the extensible file system based on the microkernel. The invention can avoid most lock competition overheads, and realize load balancing and elastic expansion of file service processes.

Description

Extensible file system and file access method based on microkernel

technical field

The invention relates to a file system and a file access method, in particular to a microkernel-based scalable file system and a file access method, belonging to the technical field of data storage access.

Background technique

For web servers or large multi-core workstations, file system parallelism is critical. Especially with the widespread application of distributed storage systems in the era of cloud computing, in a multi-core cloud storage system, for the chunk service process on each core, the efficiency of chunk read and write becomes a key factor, because it is usually necessary to Handle reading and writing of a large number of files in a short period of time.

The current traditional file system can work well when the number of cores is small, but due to the lock contention overhead, its performance will be greatly degraded as the number of parallel code streams increases. Its lock competition overhead mainly comes from the following two parts:

(1) Metadata. When multiple directory query code flows are in parallel, the cache data structure dcache and file attribute information inode of the directory information dentry in the metadata of the file system (including directory information dentry, file attribute information inode, super block list, and block bitmap) The access to the cache data structure icache will become a bottleneck. According to the tree structure, dcache organizes the basic information of all directories in the system and is protected by the lock object dcache_lock; icache caches all referenced file attribute information inode nodes in the system and is protected by the lock object inode_lock. When creating or opening a file, the system will parse the file path name, decompose it into multiple dentry and inode in sequence, and retrieve them in dcache and icache. This process requires acquiring global spinlocks on these data structures and mutexes on each dentry and inode.

(2) Block cache. There is also significant contention overhead for the shared block cache in the file system when reading and writing files in parallel. In addition to the global lock, each block cache entry is controlled by a state value (equivalent to a mutex) to prevent blocks from being written at the same time, which makes programs with more write operations less scalable.

At the same time, the unbalanced load between multiple file service processes or the mismatch between the amount of user requests and the service capabilities of the service processes will also cause unreasonable use of computing resources, resulting in a decline in the overall performance of the file system.

Contents of the invention

The invention provides a microkernel-based parallel file system that can be extended to a multi-core platform and a file access method based on the system. By dividing data objects and operations of the file system, most lock competition overheads are avoided. The master-slave service structure is adopted to realize the load balancing and elastic expansion of the file service process.

The technical scheme of the present invention is as follows: a microkernel-based scalable file system, comprising:

Disk service: used to load data on disk to memory or write memory data back to disk;

System task: used to copy file data between the user process and the file service process, and to help the disk service execute kernel-mode privileged IO instructions when it needs to access the disk;

Directory service process: work on a single processing core, manage global metadata, set block cache to save file attribute information inode and directory information dentry, receive all file processing requests from user processes, perform directory traversal, and process the file processing requests Requests for operating metadata in the file, and routing requests for reading and writing files to the file service process;

A plurality of file service processes: for processing requests for reading and writing files, and setting up a cache pool for caching data file blocks, and the data file blocks include data blocks and intermediate blocks.

Further, when the directory service process routes, it estimates the load value of each file service process according to the amount of read and write data, and selects the file service process with the lowest load value as the routing target. When the directory service process routes a read and write request The load value of the corresponding file service process is increased, and the file service process subtracts the corresponding load value when it obtains a queued request and is about to process it.

Further, the directory service process monitors the average load of the file service process and adjusts the number of file service processes.

The metadata includes directory information dentry, file attribute information inode, super block list and block bitmap.

The request for manipulating metadata includes opening a file, closing a file, creating a file, creating a folder, entering a folder, and changing file attributes.

A file access method based on a microkernel-based extensible file system, comprising steps: a user process sends a file access request to a directory service process, and the parameters of the file access request include a file descriptor ID, a buffer address and a file request word The number of nodes; the directory service process retrieves the inode pointer in the file descriptor table of the user process, and at the same time retrieves the target file service process in the routing table; the target file service process retrieves the file inode and analyzes the block map, and reads block by block from the cache Get the indirect blocks and data blocks related to the requested file, and the blocks not found in the hash table are loaded by the disk service; the target file service process requests the system task to copy data from the address space of the file service process to the buffer of the user process.

Further, the directory service estimates the load value of each file service process according to the amount of read and write data when retrieving the target file service process in the routing table, and selects the file service process with the lowest load value as the routing target, and the directory service process When routing a read and write request, the load value of the corresponding file service process is increased, and the file service process subtracts the corresponding load value after obtaining a queued request and is about to process it.

The advantage of the technical solution provided by the present invention is:

(1) The directory service process performs serial directory operations, which can avoid locking metadata and remove the overhead caused by the cache consistency protocol.

(2) For a read and write request of an opened file, the directory service process routes it to a specific file service process for processing, thereby ensuring that the sets of files operated by different file service processes are disjoint. The distributed block caches of multiple file service processes each store different data content, so they are independent of each other, and the data structure does not need to be locked. When multiple user processes try to access the same file block, the reading and writing process will be performed serially by a single file serving process, so the mutex on each block cache can also be removed.

(3) The directory service process, as the main service process, uses a global view to balance the load of the file service process (slave service process), and dynamically adjusts the scale of the service to match the request load of the user process.

(4) The functions of the file system are decomposed and run on specific processing cores, and at the same time, the highly competitive data structures are separated, so that the working sets of each service process are almost disjoint, which can avoid most locks of the file system.

Description of drawings

Figure 1 is a schematic diagram of the architecture of a microkernel-based scalable file system;

Fig. 2 is an interaction sequence diagram of the microkernel-based scalable file system.

detailed description

The present invention will be further described below in conjunction with the examples, but not as a limitation of the present invention.

As shown in Figure 1, the microkernel-based scalable file system involved in this embodiment includes:

Disk service: a block device driver that uses direct memory access to load data on the disk into memory or write memory data back to disk;

System tasks: run in the kernel mode, used to copy file data between the user process and the file service process, and help the disk service execute privileged IO instructions in the kernel mode when it needs to access the disk;

Directory service process and multiple file service processes: access directory or file data in the disk by interacting with system tasks and disk services, and provide file services for user processes. In the master-slave structure, the directory service process can be regarded as the master service process of the present invention, and the file service process is the slave service process.

The directory service process works on a single processing core and manages metadata in memory (including directory information dentry, file attribute information inode, super block list, block bitmap), and it provides a set of unified interfaces to users as an access entry. Requests from user processes are sent to the directory service process. Among them, requests for manipulating metadata, such as opening files, closing files, creating files, creating folders, entering folders, changing file attributes, etc., are directly processed by the directory service process, while requests for reading and writing ordinary files are routed to file services process is processed.

The routing policy operation of the directory service process is completed according to a hash routing table, which records the routing items mapped from the file attribute information inode to the file service process. When a user process requests to open a new file, the directory service process will select a file service process among multiple file service processes for it, and create a new routing item. Subsequent requests to access the file will be routed to the file serving process. When no process references the file, its routing entry will be deleted. Such a routing method makes the cached file blocks always be stored in a fixed file service process during the file opening period. During this period, it is impossible for other file service processes to operate on the file, thereby ensuring the consistency between cached data and disk data. In order to avoid unnecessary routing operations, the user process also caches routing items, so that most file requests can be directly sent to the file service process without going through the routing process.

Although some metadata with low contention is maintained by the directory service process, it also needs to be accessed by the file service process. These metadata include superblock list, block bitmap, and icache, which are stored in the directory service process and shared by each file service process, so that each file service process can access them conveniently and quickly. After each file service process is started, the system task shares these metadata through memory page mapping.

Since the directory service process and the file service process access all block data indiscriminately, both the super block list and the block bitmap need to be locked and synchronized. icache maintains the inode index nodes referenced by all current processes. Before the file service process reads a file, it needs to retrieve an inode block map, and its data structure does not need to be locked. This is because at a certain moment, each service The objects operated by the process are disjoint: (1) the file service process will only access the block map of an inode, but the directory service process will not access the block map of the inode; (2) due to the isolation between the file service processes, an inode in It will only be accessed by the only file serving process at a time.

For the load balancing problem of the microkernel-based scalable file system proposed by the present invention, if the workload of some file service processes is overloaded or idle, the overall performance will become poor because the physical cores of the processor are not properly utilized. The file service process must be able to expand elastically, that is, the service as a whole can be dynamically scaled according to the load requested by users. In the present invention, the directory service process implements a load balancing strategy when creating a static routing item for a newly opened file. For each file read and write request, use the amount of data read and written to measure the load. This is because the duration of the read and write process is approximately proportional to the amount of data. The file serving process uses a ring buffer to store outstanding requests and processes these messages sequentially, so the total load of these outstanding tasks reflects the current workload of a file serving process. When routing, the directory service process first estimates the load value of each file service process, and selects the one with the lowest load value as the routing target. The directory service process increases the corresponding load value when routing a read and write request, and the file service process subtracts the corresponding load value after obtaining a queued request and is about to process it.

The directory service process is also responsible for monitoring the average load of all file service processes and periodically adjusting the number of file service processes. When the overall load is high, it will request the microkernel to allocate additional cores to run new file-serving processes. A new file serving process cannot start working until it has initialized and updated its state. Conversely, when the overall load is too low, some file serving processes will be terminated to reclaim the physical core resources of the processors they use. Before termination, outstanding requests are sent back to the directory service process and rerouted, while its cached dirty blocks are rewritten back to disk to ensure data consistency.

Please refer to Fig. 2, the interactive process of each service process of the microkernel-based scalable file system proposed by the present invention is as follows:

1. Directory service process ←→ file service process

Before reading a file, the metadata that the file service process must obtain includes inode items and file descriptor information. The request sent by the user process includes the file descriptor ID, the buffer address and the number of bytes, and the request will be sent to the emergency buffer of the directory service process first, so that it can be routed preferentially. Then, the directory service process retrieves the inode pointer in the file descriptor table of the user process, and at the same time the target file service process is found in the routing table. Finally, the directory service process will pass the inode number and file descriptor to the file service process.

2. File service process ←→ disk service

After receiving the routing request, the file service process retrieves the file inode and analyzes the block map. Next, the indirect blocks and data blocks associated with the file are read from the cache block by block. If a block is not found in the hash table, it will be loaded by the disk service. There are two write-back strategies for dirty blocks: (1) periodic write-back; (2) when there are not enough free cache blocks, use the Least Recently Used (LRU) strategy to write back dirty blocks.

Considering that there is a synchronization problem within the multiple worker threads of the file serving process, data inconsistencies may occur during the time a thread loads an uncached block and builds a new hash table entry for it. Before the hash table is built, the thread blocks and waits for the disk service to read the block content for it. If a subsequently scheduled thread also tries to read the same block, it will request disk again and create redundant hash entries since the block cache has not been updated. The solution of the present invention is: if subsequent threads attempt to read the same block, they will be suspended. When the block data is loaded, the suspended thread is woken up and reads directly from the cache. The directory service process also uses a similar method when reading dentry and inode.

3. Disk Service ←→ System Task

The system task provides three interface functions for the disk service: interface function 1, execute a single I/O command according to a port number; interface function 2, execute a group of I/O commands according to multiple port numbers; interface function 3, open /Close interrupts and set interrupt handling strategies.

The disk service utilizes the above interface to perform direct memory access operations. When a disk interrupt occurs, the handler function registered by the system simply notifies the disk service.

4. File service process ←→ system task

Once the file serving process has successfully read the block, it will request the system task to copy the data from the address space of the file serving process to the buffer of the user process.

Claims (7)

1. an extensible file system based on micro-kernel, it is characterised in that including:

Disk service: for the data on disk being loaded onto internal memory or internal storage data being write back disk；

System task: for copied files data between consumer process and file service process, and need to visit at disk service It is helped to perform the franchise I/O instruction of kernel state when asking disk；

Directory service process: work in single process core, the metadata of the management overall situation, block is set and caches for preserving file attribute Information inode and directory information dentry, the All Files receiving consumer process processes request, performs directory traversal, processes institute State the request of operation metadata in document processing request, and the request of reading and writing of files is routed to file service process；

Multiple file service processes: for processing the request of reading and writing of files, be provided with cache pool for data cached blocks of files, institute State data file block and include data block and intermediate mass.

Extensible file system based on micro-kernel the most according to claim 1, it is characterised in that described directory service is entered Estimate the load value of each file service process during journey route according to read-write data volume, and choose the file service that load value is minimum Process is as route target, and described directory service process increases the negative of corresponding document service processes when routeing a read-write requests Load value, described file service process deducts respective load value after obtaining a request queued up and when will process.

Extensible file system based on micro-kernel the most according to claim 1, it is characterised in that described directory service is entered The average load of range monitoring file service process also adjusts file service number of processes.

Extensible file system based on micro-kernel the most according to claim 1, it is characterised in that described metadata includes Directory information dentry, file attribute information inode, superblock list and bitmap block.

Extensible file system based on micro-kernel the most according to claim 1, it is characterised in that described operation metadata Request include opening file, close closed file, create file, create file, enter file and change file attributes.

6. the file access method of an extensible file system based on micro-kernel, it is characterised in that include step: Yong Hujin Journey sends file access request to directory service process, and the parameter of described file access request includes filec descriptor ID, buffering Regional address and the byte number of file request；Directory service process retrieves inode pointer in the filec descriptor table of consumer process, Searched targets file service process the most in the routing table；File destination service processes retrieves file inode and analysis block reflects Penetrating figure, from caching, block-by-block reads the indirect block relevant to demand file and data block, and the block not found in Hash table is by magnetic Dish service loads；File destination service processes Request System task copy from the address space of file service process data to The relief area of family process.

The file access method of extensible file system based on micro-kernel the most according to claim 6, it is characterised in that Estimate that each file service is entered according to read-write data volume during described directory service searched targets file service process in the routing table The load value of journey, and choose the minimum file service process of load value as route target, described directory service process is in route Increasing the load value of corresponding document service processes during one read-write requests, described file service process is obtaining asking of a queuing Respective load value is deducted after asking and when will process.