CN118093499A - Data transmission method, device, equipment and storage medium for remote memory access - Google Patents

Abstract

The present invention discloses a data transmission method, device, equipment and storage medium for remote memory access, including communicating through the RDMA Sendℜ primitive after determining that the message type of a message to be transmitted is a small message; generating a corresponding work queue element and putting it into a sending queue; determining that there are remaining cache blocks or free caches in a small message buffer area, taking out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sending it; after determining that the message type of the message to be transmitted is a large message, communicating through the RDMA Sendℜ primitive, determining that the required capacity of the message to be transmitted is less than or equal to the remaining storage space, allocating capacity in the large message buffer area according to the required capacity, and dividing the message to be sent into multiple data packets for sending; when the cache area in the last-level cache module receives data, the cache area notifies the application to process the data through a shared cache mapping with the application.

Description

Data transmission method, device, equipment and storage medium for remote memory access

Technical Field

The present invention relates to the technical field of data transmission, and in particular to a data transmission method, device, equipment and storage medium for remote memory access.

Background technique

Data center applications are increasingly dependent on high-speed networks, which require network communication features with high throughput, low latency, and low CPU overhead to support large-scale network communication. In particular, for bandwidth-intensive applications such as distributed machine learning and cloud storage, more than 100Gbps of network bandwidth is required between servers, while for online services such as database online analytical processing, low latency is required to minimize query response time. At the same time, most applications want the network stack to have low CPU overhead in order to reserve as many CPU cores as possible for computing.

Remote Direct Memory Access (RDMA) has become a popular high-speed network technology, mainly due to its high throughput, low latency and low CPU overhead provided by its architectural innovations such as kernel bypass and transport offload. However, despite the high performance and low CPU overhead of RDMA technology, commercial RDMA Network Interface Card (RNIC) still faces the problem of main memory bandwidth competition, which is manifested as follows: when RNIC performs direct memory access (DMA) operations to store received messages to main memory, RNIC will compete with other CPU computing processes (such as main memory data analysis, data replication and garbage collection, etc.) for main memory bandwidth; under this competition, RNIC is difficult to obtain sufficient main memory bandwidth to store received data packets to main memory in time, causing the RNIC on-chip cache to be filled with unprocessed data packets, eventually leading to data overflow in RNIC, and it can only discard the received but unprocessed data packets; at the same time, data overflow triggers congestion control, resulting in a decrease in network throughput and a significant increase in latency.

With the development of high-speed networks and the expansion of deployment scale, the main memory bandwidth competition problem will be further aggravated. Therefore, how to avoid the impact of main memory bandwidth competition on RDMA network communication is an urgent problem to be solved in RDMA network applications.

Summary of the invention

Based on this, it is necessary to propose a data transmission method, device, equipment and storage medium for remote memory access to address the above problems.

A remote memory access data transmission method, the method comprising:

S1, a message to be transmitted that the second host needs to send to the first host;

S2. The second host determines the message type of the message to be transmitted, where the message type includes a large message and a small message;

S3. After determining that the message type of the message to be transmitted is a small message, the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes a cache block from the small message cache area in the last-level cache module and generates a corresponding work queue element and puts it into the shared receiving queue;

S4: The first host determines whether there are any remaining cache blocks or free caches in the small message cache area;

S5: If there are no remaining cache blocks and free cache in the small message cache area, the first host rejects the communication request of the second host;

S6: If there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue, and executes S11;

S7. After determining that the message type of the message to be transmitted is a large message, the second host communicates with the first host through an RDMA Read/Write primitive, and the first host determines the required capacity size of the message to be transmitted;

S8. The first host determines whether the required capacity of the message to be transmitted satisfies the remaining storage space;

S9. If the required capacity of the message to be transmitted is greater than the remaining storage space, the first host rejects the communication request of the second host;

S10, if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission, and executes S11 immediately after each data packet arrives at the large message buffer of the first host;

S11, when the cache area in the last-level cache module receives the data, the cache area notifies the application to process the data through a shared cache mapping with the application;

S12, if the residence time of the data in the last-level cache module is greater than the data residence time threshold, unloading the data of the message to the receiving buffer of the main memory, and informing the application of the relevant information, and the application subsequently takes out the data from the main memory and processes it;

S13: If the residence time of the data in the last-level cache module is less than or equal to the data residence time threshold, it is determined that the processing is completed.

In a specific embodiment, before S1, the method further includes:

The first host establishes RDMA communication with the second host and initializes communication resources, including queue pair numbers and data cache addresses of both parties;

The initialization of communication resources is completed through socket communication or RDMA connection manager.

In the method, determining the message type of the message to be transmitted specifically includes: comparing the size of the message to be transmitted with a data threshold, and determining the message type according to the comparison result;

If the size of the message to be transmitted is less than or equal to the data threshold, determining that the message type is a small message;

If the size of the message to be transmitted is greater than the data threshold, it is determined that the message type is a large message.

In a specific embodiment, the data threshold is set to 4KB; each work queue element in the shared receiving queue points to a cache block of 4KB in size in the last-level cache module.

In a specific embodiment, the cache space used for data reception under RDMA communication is allocated in the last-level cache module, and the small message cache area and the large message cache area share the cache space; the cache space size of the last-level cache module = network bandwidth * data residence time threshold * proportion of residence data that needs to be processed in the cache space.

In a specific embodiment, the first host determines the required capacity size of the message to be transmitted, specifically including: the required capacity of the message to be transmitted = the amount of data to be sent * the data residence time threshold, and the data residence time threshold is set to 200 microseconds.

In a specific embodiment, the second host divides the message to be sent into multiple data packets for sending, specifically including: dividing the message to be sent into multiple data packets no larger than 256KB for sending.

A remote communication device, comprising:

A request module, used for generating a communication request when the second host needs to send a message to be transmitted to the first host;

A message type determination module, used to determine the message type of the message to be transmitted, wherein the message type includes a large message and a small message;

A small message transmission module is used to determine that the message type of the message to be transmitted is a small message, and then the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes out a cache block from the small message cache area in the final cache module and generates a corresponding work queue element and puts it into the shared receiving queue, and determines whether there are remaining cache blocks or free caches in the small message cache area; if there are no remaining cache blocks and free caches in the small message cache area, the communication request from the second host to the first host is rejected; if there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue;

A large message transmission module is used to determine that the message type of the message to be transmitted is a large message, and then the second host communicates with the first host through the RDMA Read/Write primitive, and the first host determines the required capacity of the message to be transmitted, and judges whether the required capacity of the message to be transmitted meets the remaining storage space; if the required capacity of the message to be transmitted is greater than the remaining storage space, the communication request from the second host to the first host is rejected; if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer area according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission;

The transmission processing module is used to notify the application to process the data through the shared cache mapping with the application after the cache area in the last-level cache module receives the data; if the residence time of the data in the last-level cache module is greater than the data residence time threshold, the data of the message is unloaded to the receiving buffer of the main memory, and the relevant information is notified to the application, and the application subsequently retrieves the data from the main memory and processes it; if the residence time of the data in the last-level cache module is less than the data residence time threshold, it is determined that the processing is completed.

An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

S1, a message to be transmitted that the second host needs to send to the first host;

S2. The second host determines the message type of the message to be transmitted, where the message type includes a large message and a small message;

S3. After determining that the message type of the message to be transmitted is a small message, the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes a cache block from the small message cache area in the last-level cache module and generates a corresponding work queue element and puts it into the shared receiving queue;

S4: The first host determines whether there are any remaining cache blocks or free caches in the small message cache area;

S5: If there are no remaining cache blocks and free cache in the small message cache area, the first host rejects the communication request of the second host;

S6: If there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue, and executes S11;

S7. After determining that the message type of the message to be transmitted is a large message, the second host communicates with the first host through an RDMA Read/Write primitive, and the first host determines the required capacity size of the message to be transmitted;

S8. The first host determines whether the required capacity of the message to be transmitted satisfies the remaining storage space;

S9. If the required capacity of the message to be transmitted is greater than the remaining storage space, the first host rejects the communication request of the second host;

S10, if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission, and executes S11 immediately after each data packet arrives at the large message buffer of the first host;

S11, when the cache area in the last-level cache module receives the data, the cache area notifies the application to process the data through a shared cache mapping with the application;

S12, if the residence time of the data in the last-level cache module is greater than the data residence time threshold, unloading the data of the message to the receiving buffer of the main memory, and informing the application of the relevant information, and the application subsequently takes out the data from the main memory and processes it;

S13: If the residence time of the data in the last-level cache module is less than or equal to the data residence time threshold, it is determined that the processing is completed.

A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps:

S1, a message to be transmitted that the second host needs to send to the first host;

S2. The second host determines the message type of the message to be transmitted, where the message type includes a large message and a small message;

S3. After determining that the message type of the message to be transmitted is a small message, the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes a cache block from the small message cache area in the last-level cache module and generates a corresponding work queue element and puts it into the shared receiving queue;

S4: The first host determines whether there are any remaining cache blocks or free caches in the small message cache area;

S5: If there are no remaining cache blocks and free cache in the small message cache area, the first host rejects the communication request of the second host;

S6: If there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue, and executes S11;

S7, after determining that the message type of the message to be transmitted is a large message, the second host communicates with the first host through the RDMA Read/Write primitive, and the first host determines the required capacity size of the message to be transmitted;

S8. The first host determines whether the required capacity of the message to be transmitted satisfies the remaining storage space;

S9. If the required capacity of the message to be transmitted is greater than the remaining storage space, the first host rejects the communication request of the second host;

S10, if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission, and executes S11 immediately after each data packet arrives at the large message buffer of the first host;

S11, when the cache area in the last-level cache module receives the data, the cache area notifies the application to process the data through a shared cache mapping with the application;

S12, if the residence time of the data in the last-level cache module is greater than the data residence time threshold, unloading the data of the message to the receiving buffer of the main memory, and informing the application of the relevant information, and the application subsequently takes out the data from the main memory and processes it;

S13: If the residence time of the data in the last-level cache module is less than or equal to the data residence time threshold, it is determined that the processing is completed.

The embodiments of the present invention have the following beneficial effects:

The present invention adopts different RDMA operation and control methods for small messages and large messages respectively. For small messages, RDMA SEND/RECV is selected, and the receiving queue is aggregated by a shared receiving queue mechanism; for large messages, RDMA READ/WRITE is used, and according to the required capacity, the two share the cache space of the data receiving cache pool, and the cache resource monopoly is avoided by setting the data usage threshold of the two types of messages.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

in:

FIG1 is a flow chart of a data transmission method for remote memory access in one embodiment;

FIG2 is a flow chart of a small message receiving process in a data transmission method for remote memory access in one embodiment;

FIG3 is a flow chart of a large message receiving process in a data transmission method for remote memory access in one embodiment;

FIG. 4 is a structural block diagram of an electronic device in an embodiment.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The basic idea of the present invention to solve the main memory bandwidth competition problem is to avoid excessive use of slow main memory and directly use the CPU cache, that is, to reserve a small isolated cache pool in the last level cache (LLC). From the bandwidth point of view, the CPU cache is sufficient to meet the needs of RNIC, but only a very limited cache capacity can be used. If the data obtained by RNIC is small enough and the life cycle is short enough, then the CPU cache is sufficient. However, for data that is relatively large or has a long life cycle, a better solution is needed.

Specifically, the RNIC data receiving module moves the main memory out of the receiver's data path and reserves a small isolated cache pool in the last-level cache (LLC) to receive message data from the network card; then, in order to improve the reuse level of the reserved LLC, different RDMA operations and control methods are used for small messages and large messages respectively; small messages use RDMA SEND/RECV operations and use the shared receive queue mechanism to aggregate the receive queue; large messages use RDMA READ/WRITE operations and allocate cache according to the message size and data residence time threshold. Small and large messages share the cache pool and avoid exclusive cache resources through the user-set data residence time threshold and the proportion of residence data that needs to be processed in the cache space.

In addition, in order to improve data processing efficiency and cache reuse, the present invention uses the SLAB (Sequential Locality Allocation Buffer) algorithm to manage the cache to avoid internal fragmentation, uses multi-threaded parallel data processing operations to improve processing efficiency, and uses pipeline technology to achieve more fine-grained cache recycling. In order to maintain a high-performance network, when the application occupies the cache for more than a threshold, the data on the cache pool will be copied to the main memory, thereby quickly releasing the cache.

The present invention can significantly improve communication throughput and reduce network delay and main memory bandwidth occupancy.

Existing experimental results show that the present invention increases the communication throughput by 2.11 times, reduces the average network delay by 64.2%, and reduces the main memory bandwidth occupancy by 96.4%.

In one embodiment, a data transmission method for remote memory access is provided.

As shown in FIG1 , the data transmission method for remote memory access specifically includes the following steps:

S1, a message to be transmitted that the second host needs to send to the first host;

S2. The second host determines the message type of the message to be transmitted, where the message type includes a large message and a small message;

Specifically, comparing the size of the message to be transmitted with a data threshold, and determining the message type according to the comparison result;

If the size of the message to be transmitted is less than or equal to the data threshold, determining that the message type is a small message;

If the size of the message to be transmitted is greater than the data threshold, it is determined that the message type is a large message.

Exemplarily, the data threshold is set to 4 KB; each work queue element in the shared receiving queue points to a cache block of 4 KB in the last-level cache module.

S3. After determining that the message type of the message to be transmitted is a small message, the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes a cache block from the small message cache area in the last-level cache module and generates a corresponding work queue element and puts it into the shared receiving queue;

Specifically, the cache space used for data reception under RDMA communication is allocated in the last-level cache module, and the small message cache area and the large message cache area share the cache space; the cache space size of the last-level cache module = network bandwidth * data residence time threshold * the proportion of residence data that needs to be processed in the cache space.

The first host takes a cache block from the small message cache area in the final cache module and generates a corresponding work queue element and puts it into the shared receive queue. The second host generates a corresponding work queue element request and puts it into the send queue, where the data threshold for distinguishing message types is set to 4KB. Each work queue element in the shared receive queue points to a cache block of 4KB in the final cache module.

The cache in the last level cache module is determined according to the actual CPU structure. If the last level cache of the CPU is L3 cache, the cache is allocated in the L3 cache; if the last level cache of the CPU is L4 cache, the cache is allocated in the L4 cache.

The cache space allocated in the last-level cache module is exclusively used for data reception under RDMA communication and cannot be used for other purposes. That is, the cache space is isolated, and the small message cache area and the large message cache area share a cache space, but there is a minimum occupied space threshold between the two. When one side is lower than the threshold set by itself, the other side is refused to continue to occupy the cache space. The size of the minimum occupied space threshold is determined according to the request frequency of small and large messages in the actual business.

The cache space management of the small message cache area and the large message cache area in the last-level cache module uses the SLAB algorithm based on object management, and the object granularity of management is 4KB. The small message cache area pre-allocates a fixed number of cache blocks, and the number of pre-allocated cache blocks is related to the set space threshold and the actual small message request frequency. The SLAB (Sequential Locality Allocation Buffer) algorithm is an efficient algorithm for memory management, which is used to improve the efficiency of memory allocation and release and reduce memory fragmentation. The algorithm divides the memory into different cache blocks and selects the appropriate block for allocation according to the size of the object, thereby avoiding frequent memory allocation and release operations and improving memory utilization.

For the communication of RDMA Send/Recv primitives, before data transmission, the receiver will check whether there is a free cache block in the small message buffer. If so, it will generate a corresponding work queue element and notify the other end to send data.

In the communication of the above primitives, after each data packet arrives at the corresponding area of the cache, the cache monitor immediately notifies the application to process the data through the shared cache mapping with the application. After the application completes the processing, it notifies the cache monitor through the shared cache mapping to release and recycle the cache that has been used for further use. If the data processing time exceeds the maximum residence time, the cache monitor copies the data to the main memory and notifies the application, and releases and recycles the cache occupied by the data for further use.

When the application processes data, it uses multi-threading and pipeline technology to process the data arriving at the cache. The data received by the data receiving cache pool in LLC is managed with a granularity of 4KB using the SLAB algorithm, that is, each incoming data packet is cut into small packets no larger than 4KB and then input into a data processing pipeline for processing. The pipeline includes three stages: data storage, data processing, and data release. It uses multi-threaded parallel processing. After a granular data goes through three stages, the corresponding cache space is released immediately without waiting for the entire message to be processed by the application.

S4: The first host determines whether there are any remaining cache blocks or any free cache in the small message cache area;

S5: If there are no remaining cache blocks and free cache in the small message cache area, the first host rejects the communication request of the second host;

S6: If there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue, and executes S11;

Specifically, the data address translation and permission protection functions are provided by the main memory translation table (MTT) and the main memory protection table (MPT), respectively. At the same time, huge page technology, physical segment technology, and traffic locality technology are used to shrink the table structure and store these two tables in the network card storage medium.

S7. After determining that the message type of the message to be transmitted is a large message, the second host communicates with the first host through an RDMA Read/Write primitive, and the first host determines the required capacity size of the message to be transmitted;

Specifically, the required capacity of the message to be transmitted = the amount of data to be sent * the data residence time threshold, and the data residence time threshold is set to 200 microseconds.

For RDMA Read/Write primitive communication, before data transmission, the receiver will calculate the buffer capacity required to receive the data. If the remaining buffer space of the large message buffer is greater than or equal to the required capacity, the buffer space is allocated and the peer is notified to send the data. Before the message is sent, it will be divided into data packets according to the preset maximum transmission unit size.

In the communication of the above primitives, after each data packet arrives at the corresponding area of the cache, the cache monitor immediately notifies the application to process the data through the shared cache mapping with the application. After the application completes the processing, it notifies the cache monitor through the shared cache mapping to release and recycle the cache that has been used for further use. If the data processing time exceeds the maximum residence time, the cache monitor copies the data to the main memory and notifies the application, and releases and recycles the cache occupied by the data for further use.

When the application processes data, it uses multi-threading and pipeline technology to process the data arriving at the cache. The data received by the data receiving cache pool in LLC is managed with a granularity of 4KB using the SLAB algorithm, that is, each incoming data packet is cut into small packets no larger than 4KB and then input into a data processing pipeline for processing. The pipeline includes three stages: data storage, data processing, and data release. It uses multi-threaded parallel processing. After a granular data goes through three stages, the corresponding cache space is released immediately without waiting for the entire message to be processed by the application.

S8. The first host determines whether the required capacity of the message to be transmitted satisfies the remaining storage space;

S9. If the required capacity of the message to be transmitted is greater than the remaining storage space, the first host rejects the communication request of the second host;

S10, if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission, and executes S11 immediately after each data packet arrives at the large message buffer of the first host;

Specifically, the message to be sent is divided into several data packets no larger than 256KB and sent.

The first host allocates the required capacity from the large message buffer area, and the second host divides the message to be sent into multiple data packets for transmission, each of which is no larger than 256KB.

As soon as each data packet arrives at the large message buffer of the first host, it proceeds to the next step without waiting for the message to be completely transmitted.

S11, when the cache area in the last-level cache module receives the data, the cache area notifies the application to process the data through a shared cache mapping with the application;

Specifically, data is pipelined through multi-threading and pipeline technology.

The specific details of the multi-thread and pipeline technology for data pipeline processing are as follows: the last-level cache module divides the incoming data into 4KB granularity, and the divided data undergoes three stages: data storage, data processing, and data release, and multi-thread parallel processing is used to form a data processing pipeline. After a granularity of data has gone through the three stages, the cache monitoring module immediately releases and reclaims the corresponding cache space, without waiting for the entire message to be processed by the application before releasing and reclaiming the cache space.

S12, if the residence time of the data in the last-level cache module is greater than the data residence time threshold, unloading the data of the message to the receiving buffer of the main memory, and informing the application of the relevant information, and the application subsequently takes out the data from the main memory and processes it;

Specifically, for the unloaded data, the cache monitoring module reclaims and releases the corresponding cache space.

S13: If the residence time of the data in the last-level cache module is less than or equal to the data residence time threshold, it is determined that the processing is completed.

Specifically, if the residence time of the data in the last-level cache module is less than the data residence time threshold, the data leaves the pipeline and the processing is considered completed. Without waiting for the processing result, the cache monitoring module reclaims and releases the corresponding cache space.

Furthermore, before S1, the method further includes:

The first host establishes RDMA communication with the second host and initializes communication resources, including queue pair numbers and data cache addresses of both parties;

The initialization of communication resources is completed through socket communication or RDMA connection manager.

Specifically, the initialization communication resources include information such as the queue pair numbers of both parties, data cache addresses, etc. The data of both parties are sent from the main memory, and the last-level cache receives the data.

Exemplarily, assuming that the second host needs to transmit information to the first host, the initialization of communication resources can be completed through socket communication or RDMA connection manager.

The queue pair context is stored in the network card storage medium. The queue pair scheduler is responsible for scheduling the ready queue pairs, and the work queue element mover is responsible for taking out the work queue elements of the ready queue pairs in batches from the main memory each time.

In LLC, an isolated cache is allocated according to network bandwidth and business complexity as a cache pool for data reception. In order to improve the statistical multiplexing level of LLC, the present invention adopts different RDMA operations and control methods for small messages and large messages respectively. For small messages, RDMA SEND/RECV is selected, and the receiving queue is aggregated using a shared receiving queue mechanism; for large messages, RDMA READ/WRITE is used, and the required cache is allocated according to the product of the message size and the data residence time threshold. The two share the cache space of the data receiving cache pool, and the cache resource monopoly is avoided by setting the usage threshold of the two types of messages.

Compared with the data direct IO access (Data Direct Input Output) technology without cache isolation, the present invention increases the hit throughput by 1.6 times on average, increases the network throughput by 5% on average, and reduces the main memory bandwidth usage by 5% on average. At the same time, according to Little's law, the size of the buffer used for data reception is limited to solve the mismatch between the small size of the reserved LLC and the inefficient reservation of the RDMA queue pair, and only a small cache is needed to support high bandwidth requirements.

For example, for a bandwidth of 200 Gbps, when the data residence time threshold is 200 microseconds, the present invention only needs 5 MB of cache space to support the demand.

In order to better understand the specific details and features of receiving a small message in a remote memory access data transmission method of the present invention, a detailed description is given in conjunction with FIG. 2, including steps 1 to 6:

Step 1: Prepare resources for communication between the two parties, including constructing a queue pair, obtaining the queue pair number, data cache address and other information of the other end, and the two parties use a reliable connector to establish a connection; after the establishment is completed, the second host applies for a data sending request to the first host, informing it that a small message is about to be sent, and the second host will use the RDMA Send primitive to send data.

Step 2: The first host receives the data sending request from the second host, determines that the request is for a small message, and uses the RDMA Recv primitive to receive data. The application in the first host checks whether there are any free cache blocks or free caches in the small message buffer area in the LLC. If not, the corresponding work queue element receiving request is rejected. If there are free cache blocks or free caches, the corresponding receiving cache block is allocated, and the application generates a corresponding work queue element pointing to the cache block, and stores the work queue element in the receiving queue of the main memory, waiting for the network card program to extract it.

Step 3: After the first host is ready to receive the data buffer block and work queue element, it notifies the second host that it can send data. After receiving the data, the second host replies with an acknowledgment (ACK), and the first host polls the completion queue to confirm that the other end has received the information.

Step 4: The second host puts the corresponding sending request into the sending queue in the form of a work queue element, and the work queue element points to the data information to be sent in the main memory, including the address, length, etc. When the program on the network card obtains the work queue element from the main memory and executes it, the data to be sent is transferred to the network card receiving queue of the first host according to the information stored in the work queue element. The first host takes out the corresponding work queue element from the shared receiving queue and transfers the data from the receiving queue to the cache block allocated in advance in the small message buffer area.

Step 5: As soon as the transmitted data reaches the corresponding cache block in the last-level cache, the application processes the data through multi-threading and pipelining techniques.

Step 6: As soon as the data leaves the pipeline, the application notifies the cache monitor through the shared cache mapping to reclaim the cache block for further use.

In order to better understand the specific details and features of a remote memory access data transmission method of the present invention when receiving a large message, a detailed description is given in conjunction with FIG. 3, including RDMA Read and RDMA Write:

The RDMA Read process includes steps 1 to 5:

Step 1: Prepare the resources for communication between the two parties, including constructing a queue pair, obtaining the queue pair number, data address and other information of the other end, and the first host notifies the second host that it needs to perform an RDMA Read operation and prepares the data to be read. The second host places the data to be read in the registered area of the main memory, notifies the first host that the data to be read is ready, and informs the main memory information and permissions for storage.

Step 2: The first host checks whether the large message buffer area in the last-level cache has enough capacity to receive the data to be read. If yes, it generates a corresponding Read work queue element request and puts it into the send queue. When the program on the network card obtains the work queue element from the main memory and executes it, the first host sends a Read request to the second host, which contains the target main memory address of the data to be read, the data size, access rights and other information.

Step 3: After the second host receives the RDMA Read request sent by the first host, the network card of the second host obtains data from the corresponding area in the main memory according to the information in the request and divides it into data packets according to the set packet size and sends it to the network card receiving queue of the first host. When each data packet arrives at the network card of the first host, the first host replies with an ACK confirmation to indicate that the data has been received.

Step 4: When the network card of the first host sends the transmitted data packet from RX to the corresponding cache area of the last-level cache with a granularity of 4KB, the application immediately processes the data through multi-threading and pipeline technology as soon as the data arrives.

Step 5: As soon as the data leaves the pipeline, the application notifies the cache monitor through the shared cache map to reclaim the cache for further use.

The RDMA Write process includes steps 6 to 11:

Step 6: Prepare resources for communication between the two parties, including constructing a queue pair and obtaining the queue pair number, data address and other information of the other end.

Step 7: The second host sends a Write request, which includes relevant information about the data to be written.

Step 8: After receiving the request, the first host checks whether the large message buffer area in the last-level cache of the first host has enough capacity to receive the data to be written based on the relevant information provided. After finding enough buffer, the first host replies with an ACK to notify the second host to write the data. The reply contains information such as the cache address and access rights of the data to be written.

Step 9: The second host prepares the data to be written from the main memory and places it in the registered main memory area. The network card of the second host divides the data to be written according to the set packet size and sends it to the network card receiving queue of the first host.

Step 10: When the network card of the first host sends the transmitted data packet from the receiving queue to the corresponding cache area of the last-level cache with a granularity of 4KB, the application immediately processes the data through multi-threading and pipeline technology as soon as the data arrives.

Step 11: As soon as the data leaves the pipeline, the application notifies the cache monitor through the shared cache mapping to reclaim the cache for further use.

In one embodiment, a remote communication device is provided, the device comprising:

The request module is used to generate a communication request when the second host needs to send a message to be transmitted to the first host;

The message type determination module is used to determine the message type of the message to be transmitted, and the message type includes a large message and a small message;

The small message transmission module is used to determine that the message type of the message to be transmitted is a small message, and then the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes out a cache block from the small message cache area in the final cache module and generates a corresponding work queue element and puts it into the shared receiving queue, and determines whether there are remaining cache blocks or free caches in the small message cache area; if there are no remaining cache blocks and free caches in the small message cache area, the communication request from the second host to the first host is rejected; if there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue;

The large message transmission module is used to determine that the message type of the message to be transmitted is a large message, and then the second host communicates with the first host through the RDMA Read/Write primitive. The first host determines the required capacity of the message to be transmitted, and determines whether the required capacity of the message to be transmitted meets the remaining storage space; if the required capacity of the message to be transmitted is greater than the remaining storage space, the communication request from the second host to the first host is rejected; if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer area according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission;

The transmission processing module is used to notify the application to process the data through the shared cache mapping with the application after the cache area in the last-level cache module receives the data; if the residence time of the data in the last-level cache module is greater than the data residence time threshold, the data of the message is unloaded to the receiving buffer of the main memory, and the relevant information is notified to the application, and the application subsequently retrieves the data from the main memory and processes it; if the residence time of the data in the last-level cache module is less than the data residence time threshold, it is determined that the processing is completed.

In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

S1, a message to be transmitted that the second host needs to send to the first host;

S2. The second host determines the message type of the message to be transmitted, where the message type includes a large message and a small message;

S3. After determining that the message type of the message to be transmitted is a small message, the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes a cache block from the small message cache area in the last-level cache module and generates a corresponding work queue element and puts it into the shared receiving queue;

S4: The first host determines whether there are any remaining cache blocks or free caches in the small message cache area;

S5: If there are no remaining cache blocks and free cache in the small message cache area, the first host rejects the communication request of the second host;

S6: If there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue, and executes S11;

S7. After determining that the message type of the message to be transmitted is a large message, the second host communicates with the first host through an RDMA Read/Write primitive, and the first host determines the required capacity size of the message to be transmitted;

S8. The first host determines whether the required capacity of the message to be transmitted satisfies the remaining storage space;

S9. If the required capacity of the message to be transmitted is greater than the remaining storage space, the first host rejects the communication request of the second host;

S10, if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission, and executes S11 immediately after each data packet arrives at the large message buffer of the first host;

S11, when the cache area in the last-level cache module receives the data, the cache area notifies the application to process the data through a shared cache mapping with the application;

S12, if the residence time of the data in the last-level cache module is greater than the data residence time threshold, unloading the data of the message to the receiving buffer of the main memory, and informing the application of the relevant information, and the application subsequently takes out the data from the main memory and processes it;

S13: If the residence time of the data in the last-level cache module is less than or equal to the data residence time threshold, it is determined that the processing is completed.

As shown in FIG. 4 , the electronic device 500 includes a processor 501 , which can execute a computer program in a readable storage medium 503 or a computer program loaded from a storage unit 508 into the readable storage medium 503 to implement the functions described in the present invention.

The PCIe bus 505 in the device 500 is connected to the internal bus 504. The various components in the device 500 interact with these two types of buses. The components connected to the PCIe bus include an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as a display, a speaker, etc.; a storage unit 508, such as a disk, an optical disk, etc.; a communication unit 509, including a network card, a network card storage area, and a processing unit; the network card storage medium stores a computer program, and the processing unit on the network card executes the embodiment of the present invention by running the computer program therein. The communication unit 509 allows the device 500 to exchange information/data with other devices through a computer network such as the Internet. The communication unit 509 can be implemented in computer hardware, firmware, software, and/or a combination thereof, such as a digital electronic circuit system, an integrated circuit system, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system on a chip (SOC), a complex programmable logic device (CPLD), etc.

In one embodiment, a computer-readable storage medium is provided, storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

S1, a message to be transmitted that the second host needs to send to the first host;

S2. The second host determines the message type of the message to be transmitted, where the message type includes a large message and a small message;

S3. After determining that the message type of the message to be transmitted is a small message, the second host communicates with the first host through the RDMA Send&Recv primitive; the second host generates a corresponding work queue element and puts it into the sending queue; the first host takes a cache block from the small message cache area in the last-level cache module and generates a corresponding work queue element and puts it into the shared receiving queue;

S4: The first host determines whether there are any remaining cache blocks or free caches in the small message cache area;

S5: If there are no remaining cache blocks and free cache in the small message cache area, the first host rejects the communication request of the second host;

S6: If there are remaining cache blocks or free caches in the small message cache area, the second host takes out the message to be transmitted according to the main memory message storage address pointed to by the work queue element and sends it to the first host, and the first host transmits the message to be transmitted to the prepared cache space according to the address pointed to by the work queue element in the shared receiving queue, and executes S11;

S7. After determining that the message type of the message to be transmitted is a large message, the second host communicates with the first host through an RDMA Read/Write primitive, and the first host determines the required capacity size of the message to be transmitted;

S8. The first host determines whether the required capacity of the message to be transmitted satisfies the remaining storage space;

S9. If the required capacity of the message to be transmitted is greater than the remaining storage space, the first host rejects the communication request of the second host;

S10, if the required capacity of the message to be transmitted is less than or equal to the remaining storage space, the first host allocates capacity in the large message buffer according to the required capacity, and the second host divides the message to be sent into multiple data packets for transmission, and executes S11 immediately after each data packet arrives at the large message buffer of the first host;

S11, when the cache area in the last-level cache module receives the data, the cache area notifies the application to process the data through a shared cache mapping with the application;

S12, if the residence time of the data in the last-level cache module is greater than the data residence time threshold, unloading the data of the message to the receiving buffer of the main memory, and informing the application of the relevant information, and the application subsequently takes out the data from the main memory and processes it;

S13: If the residence time of the data in the last-level cache module is less than or equal to the data residence time threshold, it is determined that the processing is completed.

The computer readable storage medium includes a receiving storage medium, a readable storage medium or a network card storage medium. The receiving storage medium stores data sent by the network card. The readable storage medium and the network card storage medium store computer programs to implement the data communication method based on remote direct data access described in the present invention.

Specifically, the receiving storage medium may be any entity or recording medium capable of receiving data sent by the network card, including static random access memory (SRAM), embedded dynamic random access memory (eDRAM), etc. The readable storage medium may be any entity or recording medium capable of carrying the computer program instructions, including a USB flash drive, a mobile hard disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), etc. The network card storage medium may be any entity or recording medium capable of carrying the computer program instructions and combined with the network card, including a field programmable gate array (FPGA), static random access memory (SRAM), flash memory, electronically erasable programmable read-only memory (EEPROM), etc.

The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims (10)

1. A method for transmitting data for remote memory access, the method comprising:

S1, a second host needs to send a message to be transmitted to a first host;

s2, the second host determines the message type of the message to be transmitted, wherein the message type comprises a big message and a small message;

S3, after the message type of the message to be transmitted is determined to be a small message, the second host and the first host communicate through RDMA Send & Recv primitives; the second host generates corresponding work queue elements and puts the work queue elements into a sending queue; the first host takes out a buffer block from a small message buffer area in the final buffer module and generates corresponding work queue elements to be put into a shared receiving queue;

S4: the first host judges whether residual cache blocks or idle caches exist in the small message cache region;

S5: if the small message buffer area does not have the residual buffer blocks and the idle buffer, the first host refuses the communication request of the second host;

S6: if the small message buffer area has residual buffer blocks or idle buffer, the second host takes out the message to be transmitted according to the main storage message storage address pointed by the work queue element and sends the message to the first host, and the first host transmits the message to be transmitted to the prepared buffer space according to the address pointed by the work queue element in the shared receiving queue, and S11 is executed;

S7, after the message type of the message to be transmitted is determined to be a large message, the second host and the first host communicate through RDMA Read/Write primitives, and the first host determines the required capacity of the message to be transmitted;

s8, the first host judges whether the required capacity of the message to be transmitted meets the residual storage space or not;

s9, if the required capacity of the message to be transmitted is larger than the residual storage space, the first host refuses the communication request of the second host;

S10, if the required capacity of the message to be transmitted is smaller than or equal to the remaining storage space, the first host allocates capacity in a large message buffer area according to the required capacity, the second host segments the message to be transmitted into a plurality of data packets for transmission, and S11 is executed immediately after each data packet reaches the large message buffer area of the first host;

s11, after the buffer area in the final buffer module receives data, the buffer area informs the application program to process the data through the shared buffer mapping with the application program;

S12, if the residence time of the data in the final-stage buffer module is larger than the data residence time threshold value, unloading the data of the message into a main storage receiving buffer area, informing the application program of related information, and then taking out the data from the main storage and processing the data by the application program;

S13, if the residence time of the data in the final cache module is smaller than or equal to the data residence time threshold value, determining that the processing is completed.

2. The method for data transmission of remote memory access according to claim 1, wherein prior to S1, the method further comprises:

The first host establishes RDMA communication with the second host, and initializes communication resources including queue pair numbers and data cache addresses of the two parties;

the initializing communication resources is accomplished through a socket communication or RDMA connection manager.

3. The method for transmitting data for remote memory access according to claim 2, wherein the determining the message type of the message to be transmitted specifically includes: comparing the size of the message to be transmitted with a data threshold value, and determining the type of the message according to a comparison result;

If the size of the message to be transmitted is smaller than or equal to a data threshold value, determining that the message type is small message;

And if the size of the message to be transmitted is larger than the data threshold value, determining that the message type is a large message.

4. A method of data transmission for remote memory access according to claim 3, wherein the data threshold is set to 4KB; each work queue element in the shared receive queue points to a 4KB sized cache block in the last level cache block.

5. A method for transferring data in remote memory access according to claim 3, wherein the last level buffer module allocates a buffer space for receiving data in RDMA communication, and the small message buffer and the large message buffer share the buffer space; the last buffer module allocates buffer space size = network bandwidth data residence time threshold value × the proportion of residence data that needs to be processed in the buffer space.

6. The method for remote memory access data transmission according to claim 1, wherein the first host determines a required capacity of a message to be transmitted, specifically comprising: the required capacity of the message to be transmitted = amount of data to be sent =data dwell time threshold set to 200 microseconds.

7. The method for transmitting remote memory access data according to claim 1, wherein the second host splits a message to be transmitted into a plurality of data packets for transmission, and specifically comprises: and splitting the message to be transmitted into a plurality of data packets with the size not larger than 256KB for transmission.

8. A telecommunications device, the device comprising:

The request module is used for generating a communication request when the second host needs to send a message to be transmitted to the first host;

The message type determining module is used for determining the message type of the message to be transmitted, wherein the message type comprises a big message and a small message;

The small message transmission module is used for determining that the message type of the message to be transmitted is a small message, and the second host and the first host communicate through RDMA Send & Recv primitives; the second host generates corresponding work queue elements and puts the work queue elements into a sending queue; the first host computer takes out a buffer block from a small message buffer area in a final buffer module, generates corresponding work queue elements, puts the work queue elements into a shared receiving queue, and judges whether residual buffer blocks exist or idle buffer exists in the small message buffer area; if the small message buffer area does not have the residual buffer blocks and the idle buffer, rejecting the communication request from the second host to the first host; if the small message buffer area has residual buffer blocks or idle buffer, the second host takes out the message to be transmitted according to the main storage message storage address pointed by the work queue element and sends the message to the first host, and the first host transmits the message to be transmitted to the prepared buffer space according to the address pointed by the work queue element in the shared receiving queue;

The large message transmission module is used for determining that the message type of the message to be transmitted is large, the second host and the first host communicate through RDMA Read/Write primitives, the first host determines the required capacity of the message to be transmitted, and judges whether the required capacity of the message to be transmitted meets the residual storage space or not; if the required capacity of the message to be transmitted is larger than the residual storage space, rejecting the communication request from the second host to the first host; if the required capacity of the message to be transmitted is smaller than or equal to the remaining storage space, the first host allocates capacity in a large message buffer area according to the required capacity, and the second host segments the message to be transmitted into a plurality of data packets for transmission;

The transmission processing module is used for notifying the application program to process the data through the shared cache mapping with the application program after the buffer area in the final buffer module receives the data; unloading the data of the message to a main storage receiving buffer area and informing the application program of related information if the residence time of the data in the final-stage buffer module is larger than the data residence time threshold value, and then taking the data from the main storage and processing the data by the application program; if the residence time of the data in the last cache block is less than the data residence time threshold, the process is determined to be complete.

9. An electronic device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 7.

10. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method of any one of claims 1 to 7.