CN111651379A - DAX device address translation cache method and system - Google Patents

Abstract

The invention provides a DAX device address translation cache method and system, including: constructing a DAX address translation cache composed of a mapping file first address register MFA, an object offset register OFS, a file number register FID and an address translation table; according to the address translation function , write the file number in the persistent address and the object offset in the persistent address into the file number register and the object offset register respectively; the fast table converts the virtual address sent by the CPU to a physical address, and the DAX address translation cache will pass the The data stored in the file number register retrieves the address conversion table, adds the first address corresponding to the retrieval result and the data in the object offset register to obtain the direct access address, and feeds the direct access address as the conversion result of the virtual address to CPU. The invention can reduce the instruction overhead of the address conversion function by half, and greatly enhance the efficiency of processing multi-map files.

Description

DAX device address translation cache method and system

technical field

The invention relates to the research field of computer architecture and non-volatile memory, in particular to a DAX device address translation cache method and system.

Background technique

Opening and mapping the file to memory, and then accessing the file through the load and store fetching instructions is a common way to access the mapped file in the current system. In such a case, the file is mapped as a huge byte array whose first address is determined by the mapping function. The application can freely access any data in this array during runtime. The problem caused by this is that if the program writes data into the mapping file and hopes that the data can be retrieved after the program restarts, the program must process the data to make it follow a certain format. Because the mapping function does not guarantee that the same file is mapped to the same address, when the program is restarted, the mapped address in the previous run is invalid in this run. Programs cannot use virtual addresses to locate data stored in the mapped file in the previous run.

With the emergence of a new generation of non-volatile memory (Non-Volatile Memory), some researchers and enterprises are developing NVM development libraries, expecting to provide friendly application interfaces for developers to use. These development libraries work on persistent devices that support DAX (Direct Access) mode, and the libraries that can interface well with the existing operating system choose to use the file system to manage the non-volatile memory NVM, and use the file system to manage the non-volatile memory NVM. way to access resources on NVM. Therefore, these development libraries also need to provide solutions so that programs can easily access data on NVM after restarting.

At present, the common development library design is that each development library maintains persistent addresses for different storage objects. In the persistent addresses, the number of the mapping file and the offset of the storage object relative to the first address of the mapping file are stored, and the address is provided. The conversion function converts persistent addresses to virtual addresses at runtime, avoiding the overhead of formatting the data. As can be seen from the foregoing, such a conversion is necessary. Because the virtual address is volatile, after each process restart and the file is remapped, it is not guaranteed that the first address of the mapped file is similar to the mapped address of the last runtime, so the development library has to maintain its own persistent address. Although this design enables the program to still access the data in the NVM normally after restarting, it also becomes the performance bottleneck of the NVM development library.

For the existing NVM development library, the overhead of its address translation function is relatively large, accounting for about 13% of the overall overhead. And such functions cannot be optimized in software, because in traditional hardware, address translation is done by hardware. However, the current development library performs address translation in software, which will consume more time. At the same time, if there are multiple files that need to be managed, the address conversion function will have to repeatedly query the first address of different files in this run, which is extremely inefficient. And because the logic of the address conversion function is very simple and the code is extremely short, optimization at the software level is very difficult.

SUMMARY OF THE INVENTION

The present invention designs a DAX address translation cache on hardware, provides hardware functions for development libraries to accelerate address translation, and provides the operating efficiency of applications using these development libraries.

Specifically, in view of the deficiencies of the prior art, the present invention proposes a DAX device address translation caching method, which includes:

Step 1, construct the DAX address translation cache consisting of mapping file first address register MFA, object offset register OFS, file number register FID and address translation table;

Step 2, according to the address conversion function, write the file number in the persistent address and the object offset in the persistent address into the file number register and the object offset register respectively;

Step 3. The fast table converts the virtual address sent by the CPU into a physical address, and the DAX address translation cache retrieves the address translation table through the data stored in the file number register, and retrieves the first address corresponding to the retrieval result and the data in the object offset register Add up to obtain the direct access address, and feed the direct access address to the CPU as the conversion result of the virtual address.

Described DAX device address translation cache method, wherein this step 3 comprises:

Step 31. If the direct access address is 0, the address conversion function fills the first address of the mapping file into the first address register of the mapping file, and writes 0 to the DAX address translation cache. After the DAX address translation cache receives the write request, it replaces The algorithm fills the data in the file number register and the first address register of the mapping file into the address conversion table.

The DAX device address translation caching method further includes:

Step 4. Send the physical address to the cache memory, and use the data corresponding to the physical address in the cache memory as the response result to determine whether the direct access address is valid. If so, the direct access address is fed back to the CPU, otherwise the response The result is fed back to the CPU.

In the described DAX device address translation caching method, the address translation table is 32 register pairs.

The present invention also provides a DAX device address translation cache system, which includes:

Module 1. Build a DAX address translation cache consisting of the first address register MFA of the mapping file, the object offset register OFS, the file number register FID and the address translation table;

Module 2. According to the address conversion function, write the file number in the persistent address and the object offset in the persistent address into the file number register and the object offset register respectively;

Module 3. The fast table converts the virtual address sent by the CPU into a physical address, and the DAX address translation cache will retrieve the address translation table through the data stored in the file number register, and retrieve the first address corresponding to the retrieval result and the data in the object offset register Add up to obtain the direct access address, and feed the direct access address to the CPU as the conversion result of the virtual address.

Described DAX equipment address translation cache system, wherein this module 3 comprises:

Module 31. If the direct access address is 0, the address conversion function fills the first address of the mapping file into the first address register of the mapping file, and writes 0 to the DAX address translation cache. After the DAX address translation cache receives the write request, it replaces The algorithm fills the data in the file number register and the first address register of the mapping file into the address conversion table.

The DAX device address translation cache system further includes:

Module 4. Send the physical address to the cache memory, and use the data corresponding to the physical address in the cache memory as the response result to determine whether the direct access address is valid. If so, the direct access address is fed back to the CPU, otherwise the response The result is fed back to the CPU.

In the described DAX device address translation cache system, the address translation table is 32 register pairs.

As can be seen from the above scheme, the advantages of the present invention are:

The invention can reduce the instruction overhead of the address conversion function by half, and greatly enhance the efficiency of processing multi-map files.

Description of drawings

Figure 1 is an address translation cache structure diagram;

Figure 2 is a connection diagram between CPU, TLB and Cache;

Fig. 3 is the structure diagram of the present invention;

Fig. 4 is the effect comparison diagram of the present invention.

Detailed ways

During the research on the efficiency of the address translation function, the inventor found that the defect in the prior art is caused by too many redundant instructions. These excessive instructions come from conditional branches, redundant address loads, safety checks, etc. The purpose of these redundant instructions is to maintain a simple cache that temporarily stores the first address of the most recently accessed mapped file.

Obviously, it is impossible to maintain a large cache in software, otherwise the search efficiency will be extremely low; and the validity check of the cache also brings more redundant instructions. Considering that the purpose of these instructions is to implement address translation, and the current computer architecture already includes a fast table for accelerating address translation, it can be considered that the process is completed by hardware. However, several requirements need to be met in the design: (1) The current computer architecture should be changed as little as possible, and the data path should not be changed too much. It is best to avoid the change of the data path (2) Avoid adding new instructions as much as possible. , otherwise the actual value of the invention will be greatly reduced, (3) it is easy to use, and developers who expect to achieve performance improvement through the device cannot rewrite too much code.

Drawing on the structure of the fast table, the present invention designs a DAX address translation cache. With this cache, the instruction count of the address translation function can be reduced by half, and the device will greatly improve the performance of the address translation function in processing multiple mapped files, because parallel lookups are extremely efficient in hardware.

The main inventive points of the present invention include:

Key point 1, considering the balance between hardware performance and power consumption, the DAX address translation cache consists of 32 pairs of registers and 3 independent registers. The register pair is called Address Translation Table, and the three independent registers are called MFA register (mapped file first address register), OFS register (object offset register) and FID register (file number register). The number of the mapping file and the first address of the mapping file are stored in the register pair, and three independent registers can perform data transmission on the register pair;

Key point 2, the address translation function needs to explicitly access the DAX address translation cache through the memory fetch instruction. Since the present invention does not want to destroy the existing data paths in the current computer architecture, the address translation function needs to explicitly access the DAX address translation cache to obtain the required first address. On the one hand, this approach can avoid reducing the efficiency of existing data at the same time and affecting the performance of the existing system when adding a DAX address translation cache; on the other hand, it can also avoid adding additional instructions or modifying existing instructions. The only thing that needs to be changed is to register four virtual addresses in the operating system and map these four virtual addresses to the DAX address translation cache. In view of the fact that there are reserved addresses of a certain size in the current architecture chips, the above operations are not complicated;

Key point 3, the DAX address translation cache should be responsible for checking the validity of the address. The file number is 0 and the offset is 0, which are illegal addresses;

The key point 4 is that the DAX address translation cache can be writable and unreadable, so as to prevent malicious programs from illegally obtaining the address of the mapping file without access rights by reading the DAX operation.

In order to make the above-mentioned features and effects of the present invention more clearly and comprehensible, embodiments are given below, and detailed descriptions are given below in conjunction with the accompanying drawings.

DAX address translation cache structure:

The address translation cache structure is shown in Figure 1. In this figure, three independent registers can unidirectionally send data to the address translation table, the OFS register and the address register in the address translation table can send data to an adder, and the result generated by the adder is the DAX address translation buffer. The translated virtual address.

DAX address translation cache location:

In modern computer architecture, the connection relationship between CPU, TLB (fast table) and Cache is shown in Figure 2. When the memory access instruction is executed, the virtual address generated by the instruction is sent to the TLB by the CPU. In the case of a hit, the TLB will perform address translation to generate a physical address. The physical address will be sent to the Cache. Once hit, the data in the cache will be transferred to the CPU to complete the memory access. If it is not hit, the generated physical address will be sent to the memory bus, further handed over to the memory controller, and finally sent to the DRAM to complete the data read. The Cache is completely transparent to the programmer, and the Cache is used to cache the DRAM. Dynamic random access memory DRAM itself is used to hold instructions and data.

The DAX address translation cache should be placed between the TLB and the Cache, and the DAX address translation cache should be addressed to the reserved address of the CPU. After the TLB completes the address translation, the generated physical address is directly sent to the DAX address translation cache and the Cache. If the physical address is an access to the DAX address translation cache, the DAX address translation cache should respond and transmit the data to the CPU. Otherwise, the Cache will transfer the data to the CPU, or an error will be reported. Therefore, an arbitration logic should be set between the DAX address translation cache and the Cache. The response of the DAX address translation cache has a higher priority, and the data sent by the DAX address translation cache should be transmitted first.

The address translation function is written by the developer, and the address translation function utilizes the DAX cache to improve the execution speed of the function. Usually this function needs to query a software-maintained cache and then decide how to do the address translation. The present invention is equivalent to moving the cache maintained by the software to the hardware. The address translation function should perform the following flow:

1. Write the FID register, and write the file number in the persistent address into this register. The file number must not be 0, other than that there are no special requirements. Therefore, each development library can freely choose how to generate file numbers. Among them, the file number in the address is determined by the upper-level developer, and the file number is just a 64-bit integer. At present, the PMDK developed by Intel uses the file number + offset in the file to manage objects. The FID here is the file number in the PMDK, and the OFS is the offset within the file.

2. Write the OFS register, and write the object offset in the persistent address into this register. Object offset must not be 0, otherwise there are no special requirements.

3. Read the address translation table in the DAX address translation cache. At this time, the DAX address translation cache will retrieve the address translation table through the data stored in the FID. If a match is found, the corresponding first address will be added to the data in the OFS register, and the result will be used as the response to the read request of the address translation function. .

4. Check whether the read data is 0, if it is not 0, the address conversion is over; if it is 0, go to the next step.

5. Write the MFA register and fill the register with the first address of the mapping file. When programming, the first step to access NVM is to perform file mapping, and at this time, you can get the first address of the mapped file.

6. Write 0 to the DAX address translation cache. After the DAX address translation cache receives the write request, it fills the FID and MFA data into the address translation table through the replacement algorithm.

7. The address conversion function ends.

Arbitration between DTLB (Direct access TLB) and Cache:

As mentioned above, when the TLB completes the address translation, the obtained physical address should be sent to the DTLB and the Cache at the same time, and the responses of the DTLB and the Cache should be arbitrated, and the response of the DTLB should be preferentially transmitted to the CPU. Figure 3 shows the hardware structure that should be used to accomplish such an arbitration.

assessment. Since it is currently impractical to add this component to the CPU, the evaluation is done in a simulated manner. The present invention selects gem5 simulator. The simulator simulates different architecture CPUs, including X86, ARM, etc. Two modes of full system simulation and system interrupt simulation are provided. Since the present invention works in the user mode, there is no need to run the operating system, so the system interrupt simulation mode is used.

In the test, the present invention uses the pmemobj_direct address conversion function in the PMDK library developed and maintained by Intel to compare with the self-written address conversion function that calls the DAX address translation cache. Under the conditions of using a single memory pool and multiple memory pools, respectively, Address translation of 8 million persistent objects. The respective time-consuming (unit: second) is shown in FIG. 4 .

Impact on existing systems:

In order to assess how the addition of the DAX address translation cache will affect the existing system, it is necessary to evaluate the performance of the various components in the existing computer system.

At present, the TLB can complete the response in 1 clock cycle, and the Cache can complete the response in 5 clock cycles. In theory, when the instruction is decoded and enters the execution stage, the data can be sent to the CPU after 6 clock cycles at the earliest. Currently

The available data shows that the hit rate of the first-level cache is as high as 95%, and with the second-level cache, a 97% hit rate can be achieved. Therefore, it can be estimated that the average latency of memory access is 9 clock cycles. If the DAX address translation cache is added between the TLB and the Cache, each memory access will increase by one clock cycle to arbitrate whether to transfer the physical address to the Cache, so that ordinary memory access instructions suffer an additional 1 clock cycle delay. Performance drops by about 20%. Therefore, this paper emphasizes in the design that the TLB should send the physical address to the Cache and the DAX address translation cache at the same time, and select the data that responds to the memory access request through the arbitration logic, instead of sending it to the DAX address translation cache in sequence, and then to the Cache. .

The following are system embodiments corresponding to the foregoing method embodiments, and this implementation manner may be implemented in cooperation with the foregoing implementation manners. The related technical details mentioned in the foregoing embodiment are still valid in this embodiment, and are not repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned embodiments.

The present invention also provides a DAX device address translation cache system, which includes:

Module 1. Build a DAX address translation cache consisting of the first address register MFA of the mapping file, the object offset register OFS, the file number register FID and the address translation table;

Module 2. According to the address conversion function, write the file number in the persistent address and the object offset in the persistent address into the file number register and the object offset register respectively;

Module 3. The fast table converts the virtual address sent by the CPU into a physical address, and the DAX address translation cache will retrieve the address translation table through the data stored in the file number register, and retrieve the first address corresponding to the retrieval result and the data in the object offset register Add up to obtain the direct access address, and feed the direct access address to the CPU as the conversion result of the virtual address.

Described DAX equipment address translation cache system, wherein this module 3 comprises:

Module 31. If the direct access address is 0, the address conversion function fills the first address of the mapping file into the first address register of the mapping file, and writes 0 to the DAX address translation cache. After the DAX address translation cache receives the write request, it replaces The algorithm fills the data in the file number register and the first address register of the mapping file into the address conversion table.

The DAX device address translation cache system further includes:

Module 4. Send the physical address to the cache memory, and use the data corresponding to the physical address in the cache memory as the response result to determine whether the direct access address is valid. If so, the direct access address is fed back to the CPU, otherwise the response The result is fed back to the CPU.

In the described DAX device address translation cache system, the address translation table is 32 register pairs.

Claims (8)

1. A DAX device address translation caching method is characterized by comprising the following steps:

step 1, constructing a DAX address translation cache consisting of a mapping file initial address register (MFA), an object offset register (OFS), a file number register (FID) and an address translation table;

step 2, writing the file number in the persistent address and the object offset in the persistent address into the file number register and the object offset register respectively according to the address conversion function;

and 3, converting the virtual address sent by the CPU into a physical address by the fast table, searching the address conversion table by data stored in the file number register by the DAX address conversion cache, adding a first address corresponding to a search result and data in the object offset register to obtain a direct access address, and feeding back the direct access address serving as the conversion result of the virtual address to the CPU.

2. The DAX device address translation caching method of claim 1, wherein the step 3 comprises:

step 31, if the direct access address is 0, the address translation function fills the first address of the mapping file into the first address register of the mapping file, and writes 0 into the DAX address translation cache, and after the DAX address translation cache receives the write request, the data in the file number register and the first address register of the mapping file is filled into the address translation table through a replacement algorithm.

3. The DAX device address translation caching method of claim 1, further comprising:

and 4, sending the physical address to a cache memory, taking data corresponding to the physical address in the cache memory as a response result, judging whether the direct access address is effective, if so, feeding the direct access address back to the CPU, and otherwise, feeding the response result back to the CPU.

4. The DAX device address translation caching method of claim 1, wherein the address translation table is 32 register pairs.

5. A DAX device address translation cache system, comprising:

the module 1 is used for constructing a DAX address translation cache consisting of a mapping file initial address register (MFA), an object offset register (OFS), a file number register (FID) and an address translation table;

the module 2 writes the file number in the persistent address and the object offset in the persistent address into the file number register and the object offset register respectively according to the address conversion function;

the module 3, the fast table converts the virtual address sent by the CPU into the physical address, the DAX address translation cache retrieves the address translation table through the data stored in the file number register, adds the first address corresponding to the retrieval result and the data in the object offset register to obtain the direct access address, and feeds back the direct access address as the translation result of the virtual address to the CPU.

6. The DAX device address translation cache system of claim 5, wherein the module 3 comprises:

if the direct access address is 0, the address translation function fills the first address of the mapping file into the first address register of the mapping file, and writes 0 into the DAX address translation cache, and after the DAX address translation cache receives the write request, the data in the file number register and the first address register of the mapping file are filled into the address translation table through a replacement algorithm.

7. The DAX device address translation cache system of claim 5, further comprising:

and the module 4 sends the physical address to a cache memory, takes data corresponding to the physical address in the cache memory as a response result, judges whether the direct access address is effective, if so, feeds the direct access address back to the CPU, and otherwise, feeds the response result back to the CPU.

8. The DAX device address translation cache system of claim 5, wherein the address translation table is 32 register pairs.

Images