CN116471242A - RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method - Google Patents
RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method Download PDFInfo
- Publication number
- CN116471242A CN116471242A CN202310583845.7A CN202310583845A CN116471242A CN 116471242 A CN116471242 A CN 116471242A CN 202310583845 A CN202310583845 A CN 202310583845A CN 116471242 A CN116471242 A CN 116471242A
- Authority
- CN
- China
- Prior art keywords
- rdma
- data
- queue
- module
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 133
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000010365 information processing Effects 0.000 claims abstract description 46
- 238000005516 engineering process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/6295—Queue scheduling characterised by scheduling criteria using multiple queues, one for each individual QoS, connection, flow or priority
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9005—Buffering arrangements using dynamic buffer space allocation
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention relates to a sending end, a receiving end and a data transmission system and method based on RDMA, wherein the system comprises: a transmitting end and a receiving end; the transmitting end comprises: a first information processing module and a first RDMA transmission operation module; the first information processing module is used for generating a work request according to information processing requirements and adding the work request into a work queue; the first RDMA transmission operation module is used for acquiring a work request from a work queue, processing first data to be processed, generating first information data and then performing remote memory access; the receiving end comprises a second information processing module and a second RDMA transmission operation module; the second information processing module is used for creating a work queue and a registered memory; the second RDMA transmission operation module is used for receiving a message request sent by the information source end, processing second data to be processed, generating second information data and then performing remote memory access. The invention has low data transmission delay, high node resource utilization rate and high efficiency.
Description
Technical Field
The present invention relates to the field of data transmission technologies, and in particular, to a sending end, a receiving end, a data transmission system and a data transmission method based on RDMA.
Background
The data transmission system or the information processing system is an embedded multi-node system, the nodes execute predefined tasks, and high-bandwidth and low-delay data transmission requirements exist between the nodes.
In the process of data packet processing, the traditional TCP/IP technology is subjected to operating system software protocol stack processing and multiple data movement, and is found in high-bandwidth network technology (100 Gbps) application, the traditional TCP/IP technology has higher communication delay, processor resources and memory bus bandwidth are occupied by large-scale data transmission tasks, the resource utilization rate is low, the network delay effect is serious, and the real-time requirement cannot be met.
For the traditional RDMA technology, sending Send/receiving Receive is a bilateral operation, and as the remote application is required to sense participation to complete the sending and receiving, the participation process of both message sending and receiving parties is complex, and the message transmission mode is mostly used for transmitting control messages, so that the transmission efficiency is low.
Disclosure of Invention
The invention aims to provide an RDMA-based transmitting end, an RDMA-based receiving end, a RDMA-based data transmission system and a RDMA-based data transmission method, wherein data transmission delay is low, node resource utilization rate is high, and efficiency is high.
In order to solve the technical problems, the technical scheme of the invention is as follows:
in a first aspect, an RDMA-based sender is provided, where a sender RDMA device is provided in the sender, including: a first information processing module and a first RDMA transmission operation module;
the first information processing module is used for generating a work request according to information processing requirements and adding the work request into a work queue; the work request carries a command word;
the first RDMA transmission operation module is used for acquiring a work request from a work queue, processing first data to be processed based on the work request, generating first information data and then performing remote memory access, wherein the remote memory access comprises: a message request is generated based on the first information data and sent to the destination remote.
Further, the first RDMA transfer operation module is integrated in the embedded system, and the first RDMA transfer operation module includes: a first information operation unit and a first RDMA transmission unit;
the first information operation unit is used for acquiring first data to be processed from the memory, processing the first data to be processed according to a command word carried by the work request so as to generate first information data, and generating a message request based on the first information data;
a first RDMA transfer unit configured to obtain a work request from a work queue and send a message request to a destination remote.
Further, a first RDMA transfer unit to queue management logic, descriptors, and message completion processing operations;
the first RDMA transmission unit comprises a DMA processing module, a queue management module, a transmission scheduling module and a MAC+PHY module;
the DMA processing module comprises DMA+interface control, wherein the DMA responds to a read request to transmit data in a memory of the transmitting end to the RDMA equipment, the interface control is used for associating multiple queues with the interfaces, each interface corresponds to independent transmission scheduling control logic, and the transmission of data packets is controlled based on the interfaces;
the queue management module is used for maintaining the queue states of all the multiple queues, including sending, sending completion, receiving completion and event queues;
the queue management module is used for managing the descriptor queue and the message completion queue; the descriptor queue is used for forming a data transmission channel with the processor and carrying information of storage positions of the data packets in a system memory of the processor; the completion queue is used for forming a control channel with the processor and comprises information of completed operations and metadata;
the queue management is used for realizing a ring buffer area of the processor, and the ring buffer area is accessible by DMA;
queue management is used to maintain queue state information, including pointers to ring buffer DMA addresses, ring buffer size, producer and consumer pointers, and references to associated completion queues;
queue management realizes the management of the states of at least thousands of queues, realizes multi-queue parallel reading operation and multi-queue parallel dequeue/enqueue request operation;
the transmission scheduling module is used for designating queues to transmit, generating commands for the transmission engine, starting the transmission operation of the corresponding transmission queues, and transmitting the queue states of all the queues; scheduling operations on the associated data paths; for achieving load balancing of message flows between ports; the transmission scheduling module comprises a sending engine, a receiving engine and a cache RAM;
the cache RAM is used to temporarily store message packets during DMA operations;
the sending engine is used for scheduling the sending operation of the data packet, sending the message request of the appointed queue, and after the data packet is transmitted to the network through the Ethernet Media Access Controller (MAC) and the physical interface layer (PHY), the sending engine starts to establish a completion record and transmits the completion record to the completion sending queue;
the receiving engine is used for scheduling the receiving operation of the data packet, the data packet transmitted in the network is processed through a physical interface layer PHY and an Ethernet media access controller MAC and then transmitted to the receiving engine after being processed through a hash algorithm, the receiving engine sends a writing operation request, namely the data packet data is written into a processor memory after the writing operation is finished, and a finishing record is generated;
the MAC+PHY module comprises an Ethernet media access controller MAC and a physical interface layer PHY, and is used for determining a target receiving queue of an incoming data packet through a hash algorithm when receiving a direction, generating a command for a receiving engine and scheduling the operation of a relevant receiving data path.
In a second aspect, an RDMA-based receiver is provided, where a receiver RDMA device is provided in the receiver, including: the second information processing module and the second RDMA transmission operation module;
the second information processing module is used for creating a work queue and a registered memory;
the second RDMA transmission operation module is used for receiving the message request sent by the information source terminal, processing second data to be processed based on the command word carried by the message request, generating second information data, and then performing remote memory access, wherein the remote memory access comprises the steps of generating a received message request and sending the data request to a cache area of the local node.
Further, the second RDMA transfer operation module is integrated in the embedded system, the second RDMA transfer operation module comprising: a second information operation unit and a second RDMA transmission unit;
the second information operation unit is used for acquiring second data to be processed from the memory, processing the second data to be processed according to the command word carried by the message request so as to generate second information data, and generating a message receiving request based on the second information data;
and the second RDMA transmission unit is used for acquiring the message request sent by the receiving information source end and sending the received message request to the local node cache area.
In a third aspect, there is provided an RDMA-based data transfer system comprising: a transmitting end and at least one receiving end; wherein,,
the sending end is the sending end based on RDMA;
the receiving end is the RDMA-based receiving end.
Further, the sending end and the receiving end are connected through an Ethernet switching module network, and the Ethernet switching module supports the data center to bridge the DCB and control the PFC with priority.
In a fourth aspect, an RDMA-based data transmission method is provided, and the steps of using the RDMA-based data transmission system include:
step S100: the first information processing module of the transmitting end distributes annular transmission multi-queues, creates corresponding completion multi-queues, registers a memory area distribution buffer area to the first RDMA transmission operation module, and data in the memory buffer area is transmitted to the receiving end, namely a target remote memory;
the second information processing module of the receiving end distributes annular receiving multi-queues, creates corresponding finishing multi-queues, distributes buffer areas to the registration area of the second RDMA transmission operation module and is used for storing data sent by the sending end, namely the information source end;
step S200: the first information processing module of the transmitting end acquires information processing requirements, forms a work queue element WQE according to the information processing requirements, generates a work request, and adds the work request into a work queue; the pointer in the work queue element WQE points to a section of memory buffer, and the data in the buffer is transmitted to the memory at the destination remote end;
the first RDMA transmission operation module of the sending end obtains a work request from a work queue, consumes a work queue element WQE in the work request, converts first data to be processed in a memory buffer area pointed by a work queue element WQE pointer into first information data, generates a message request based on the first information data, and sends the message request to the second RDMA transmission operation module of the receiving end through a network data transmission channel;
step S300: the second RDMA transmission operation module of the receiving end receives the message request sent by the sending end and writes the data into the corresponding memory buffer area;
step S400: when a first pending data transfer is completed, the first RDMA transfer operation module creates a completion queue element CQE, which is placed into the completion queue CQ, indicating that the data transfer has completed.
Further, in step S400, each time the first RDMA transfer operation module consumes a work queue element WQE, a completion queue element CQE is generated;
in parallel, the first RDMA transmission operation module consumes all work queue elements WQEs, generates all corresponding completion queue elements CQEs, that is, the sending operation of all work queue elements WQEs is completed, and the message sending request is completed.
Further, in step S400, if an error occurs, the first RDMA transfer operation module creates a completion queue element CQE, where the current completion queue element CQE includes a field for recording the status of the error transfer.
The invention has the following beneficial effects:
1. RDMA (Remote Direct Memory Access), remote direct memory Access, is a technique that bypasses the operating system of the remote host to access the data in its memory. Because the operation system is not passed, a large amount of CPU resources are saved, the data transmission delay is reduced, the problems of high transmission delay and low node resource utilization rate in the prior art are solved;
2. the RDMA-based low-delay data transmission method provided by the invention simplifies the message interaction process of sending Send/receiving Receive operation, so that the transmission mechanism of the message multi-queue is more in line with the characteristic of data stream operation, and the processing efficiency of the local node is obviously improved; the nodes are connected through lossless Ethernet, and Data Center Bridging (DCB) and Priority Flow Control (PFC) flow control technologies are supported through an Ethernet switching module integrated into the system, so that reliable transmission is ensured at a physical layer, network congestion is solved, and the transmission performance of RDMA is improved;
3. the invention executes remote access in hardware, improves the utilization rate of the information processing node, reduces data transmission delay, improves the utilization rate of the information processing node, improves the overall information processing capability of the system, and improves the instantaneity.
Drawings
FIG. 1 is a schematic diagram of a data transmission system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a first RDMA transmission operation module in the present embodiment;
FIG. 3 is a flow chart showing the steps of a data transmission method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a low-delay data transmission process in practical application of the embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, the present invention is an RDMA-based data transmission system, comprising: a transmitting end and at least one receiving end; the transmitting end is an information source end for transmitting data, and the receiving end is a target far end for receiving data;
a transmitting end, comprising: a first information processing module and a first RDMA transmission operation module;
the first information processing module comprises a first information processing engine and is used for generating a work request according to information processing requirements and adding the work request into a work queue; the working request carries a command word, and the command word corresponds to a first information operation unit supported by a first RDMA transmission operation module;
referring to fig. 2, a first RDMA transmission operation module is integrated in an embedded system, and is configured to obtain a work request from a work queue, process first data to be processed based on the work request, generate first information data, and then perform remote memory access;
the first RDMA transmission operation module includes: a first information operation unit and a first RDMA transmission unit;
the first information operation unit is integrated in the first RDMA transmission operation module, and the first information operation unit preprocesses an incoming data packet before the incoming data packet is transmitted to a far end and then performs remote memory access; preprocessing is to acquire first data to be processed from a memory, process the first data to be processed according to a command word carried by a work request so as to generate first information data, and generate a message request based on the first information data; performing remote memory access includes: a message request is generated based on the first information data.
A first RDMA transmission unit, configured to obtain a work request from a work queue, and send a message request to a destination remote end; queue management logic, descriptors, and message completion processing operations are also included.
The first RDMA transmission unit comprises a DMA processing module, a queue management module, a transmission scheduling module and a MAC+PHY module;
the DMA processing module comprises DMA+interface control, wherein the DMA responds to a read request to transmit data in a memory of the transmitting end to the RDMA equipment, the interface control is used for associating multiple queues with the interfaces, each interface corresponds to independent transmission scheduling control logic, and the transmission of data packets is controlled based on the interfaces;
the queue management module is used for maintaining the queue states of all the multiple queues, including sending, sending completion, receiving completion and event queues;
the queue management module is used for managing the descriptor queue and the message completion queue; the descriptor queue is used for forming a data transmission channel with the processor and carrying information of storage positions of the data packets in a system memory of the processor; the completion queue is used for forming a control channel with the processor and comprises information of completed operations and metadata;
the queue management is used for realizing a ring buffer area of the processor, and the ring buffer area is accessible by DMA;
queue management is used to maintain queue state information, including pointers to ring buffer DMA addresses, ring buffer size, producer and consumer pointers, and references to associated completion queues;
queue management realizes the management of the states of at least thousands of queues, realizes multi-queue parallel reading operation and multi-queue parallel dequeue/enqueue request operation;
the transmission scheduling module is used for designating queues to transmit, generating commands for the transmission engine, starting the transmission operation of the corresponding transmission queues, and transmitting the queue states of all the queues; scheduling operations on the associated data paths; for achieving load balancing of message flows between ports;
the transmission scheduling module comprises a sending engine, a receiving engine and a cache RAM;
the cache RAM is used to temporarily store message packets during DMA operations;
the sending engine is used for scheduling the sending operation of the data packet, sending the message request of the appointed queue, and after the data packet is transmitted to the network through the Ethernet Media Access Controller (MAC) and the physical interface layer (PHY), the sending engine starts to establish the completion record and transmits the completion record to the completion sending queue;
the receiving engine is used for scheduling the receiving operation of the data packet, the data packet transmitted in the network is transmitted to the receiving engine after being processed by a hash algorithm through a physical interface layer (PHY) and an Ethernet Media Access Controller (MAC), and the receiving engine sends a writing operation request, namely the data packet data is written into a processor memory after the writing operation is finished, and a finishing record is generated;
the MAC+PHY module comprises an Ethernet Media Access Controller (MAC) and a physical interface layer (PHY), when the MAC+PHY module is used for receiving direction, an incoming data packet determines a target receiving queue through a flow hash algorithm, generates a command for a receiving engine, and schedules operation of a relevant receiving data path.
A receiving end, comprising: the second information processing module and the second RDMA transmission operation module;
the second information processing module comprises a second information processing engine and is used for creating a work queue and a registration memory;
the second RDMA transmission operation module is integrated in the embedded system and is used for acquiring a data receiving request from the work queue, namely receiving a message request sent by the information source end, processing second data to be processed based on a command word carried by the message request, generating second information data, and then performing remote memory access, wherein the remote memory access comprises the steps of generating a received message request and sending the data request to a cache area of the local node.
The second RDMA transmission operation module is integrated in the embedded system, and the second RDMA transmission operation module comprises: a second information operation unit and a second RDMA transmission unit;
the second information operation unit is used for acquiring second data to be processed from the memory, processing the second data to be processed according to the command word carried by the message request so as to generate second information data, and generating a message receiving request based on the second information data;
and the second RDMA transmission unit is used for acquiring the message request sent by the receiving information source end and sending the received message request to the local node cache area.
The real-time performance and the resource utilization rate of the embedded system are important, and in an information processing system or a data transmission system, a predefined data stream pattern is generally adopted, wherein small stream data is control information, large stream data is information data with large bandwidth, and the small stream data and the large stream data have high delay requirements; in the invention, the embedded RDMA transmission operation module is integrated in the embedded information processing system, RDMA technology is executed through the RDMA transmission unit, information data transmission delay is reduced, acceleration operation processing operation is provided through the information operation unit, and load of the information processing node is unloaded, so that the overall information processing capability of the system is improved.
In the embodiment of the invention, the system nodes are connected through a lossless Ethernet network, and the Ethernet switching module integrated into the system supports the flow control technologies such as Data Center Bridging (DCB), priority Flow Control (PFC) and the like, so that reliable transmission is ensured at a physical layer, network congestion is solved, and the transmission performance of RDMA is improved.
With reference to fig. 3 and fig. 4, the present invention further provides an RDMA-based data transmission method, which adopts the RDMA-based data transmission system, and the method includes:
step S100: the first information processing module of the transmitting end distributes ring-shaped transmission multi-queue RINGSQ for the first RDMA transmission operation module, creates corresponding completion multi-queue RINGSCQ, registers a memory area distribution buffer area for the first RDMA transmission operation module, and data in the memory buffer area is transmitted to a receiving end, namely a target far-end memory; the annular sending multi-queue is identified as a multi-section buffer, and the data of the multi-section buffer is carried to a remote node memory;
meanwhile, a second information processing module of the receiving end distributes ring-shaped receiving multi-queue RINGRQ, creates a corresponding finishing multi-queue, distributes a plurality of empty buffer areas to a registration area of a second RDMA transmission operation module and is used for storing data sent from a sending end, namely an information source end;
step S200: a first information processing module of a transmitting end acquires information processing requirements, forms a work queue element WQE according to the information processing requirements, generates a work request, and adds the work request with the work queue element WQE into a ring-shaped transmitting multi-queue RINGSQ; the pointer in the work queue element WQE points to a section of memory buffer, and the data in the buffer is transmitted to the memory at the destination remote end; in this embodiment, the second information processing module need not create a work queue element for the received data;
the first RDMA transmission operation module of the transmitting end acquires a work request from a work queue, judges whether a work element WQE exists in the transmission queue, consumes the work queue element WQE in the work request if the work element WQE exists, converts first data to be processed in a memory buffer area pointed by a work queue element WQE pointer into first information data, generates a data stream of a message request based on the first information data, and transmits the message request to the second RDMA transmission operation module of the receiving end through a network data transmission channel;
step S300: when the data flow reaches the second RDMA transmission operation module of the receiving end, the second RDMA transmission operation module of the receiving end receives the message request sent by the sending end, and writes the data into the corresponding target memory buffer area pointed by the empty annular receiving multi-queue RINGRQ.
Step S400: when a first pending data transfer is completed, the first RDMA transfer operation module creates a completion queue element CQE, which is placed into the completion queue CQ, indicating that the data transfer has completed.
In step S400, each time the first RDMA transfer module consumes one work element WQE, one completion queue element CQE is generated, and in parallel, the first RDMA transfer operation module consumes all work queue elements WQE, generates all corresponding completion queue elements CQE, that is, the sending operation of all work queue elements WQE has been completed, the message sending request is completed. Obviously, a Send operation realizes low-delay and high-performance remote data transmission by creating a plurality of WQEs simultaneously and transmitting the stored working mechanism in parallel.
In step S400, if an error occurs, the first RDMA transfer operation module creates a completion queue element CQE, where the current completion queue element CQE contains a field for recording the status of the error transfer.
In the data transmission method, the sending/receiving operation is different from the prior technical scheme, the interaction process of the message receiving operation is simplified, and the transmission mechanism of the designed message multi-queue is more in line with the characteristic of processing based on data flow, so that the transmission efficiency of the local node is obviously improved.
The invention is not related in part to the same or implemented in part by the prior art.
The foregoing is a further detailed description of the invention in connection with specific embodiments, and it is not intended that the invention be limited to such description. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (10)
1. RDMA-based transmitting end, be equipped with transmitting end RDMA equipment in the transmitting end, its characterized in that: comprising the following steps: a first information processing module and a first RDMA transmission operation module;
the first information processing module is used for generating a work request according to information processing requirements and adding the work request into a work queue; the work request carries a command word;
the first RDMA transmission operation module is used for acquiring a work request from a work queue, processing first data to be processed based on the work request, generating first information data and then performing remote memory access, wherein the remote memory access comprises: a message request is generated based on the first information data and sent to the destination remote.
2. The RDMA-based sender of claim 1, wherein: the first RDMA transmission operation module is integrated in the embedded system, and comprises: a first information operation unit and a first RDMA transmission unit;
the first information operation unit is used for acquiring first data to be processed from the memory, processing the first data to be processed according to a command word carried by the work request so as to generate first information data, and generating a message request based on the first information data;
a first RDMA transfer unit configured to obtain a work request from a work queue and send a message request to a destination remote.
3. The RDMA-based sender of claim 2, wherein: a first RDMA transfer unit for queue management logic, descriptors, and message completion handling operations;
the first RDMA transmission unit comprises a DMA processing module, a queue management module, a transmission scheduling module and a MAC+PHY module;
the DMA processing module comprises DMA+interface control, wherein the DMA responds to a read request to transmit data in a memory of the transmitting end to the RDMA equipment, the interface control is used for associating multiple queues with the interfaces, each interface corresponds to independent transmission scheduling control logic, and the transmission of data packets is controlled based on the interfaces;
the queue management module is used for maintaining the queue states of all the multiple queues, including sending, sending completion, receiving completion and event queues;
the queue management module is used for managing the descriptor queue and the message completion queue; the descriptor queue is used for forming a data transmission channel with the processor and carrying information of storage positions of the data packets in a system memory of the processor; the completion queue is used for forming a control channel with the processor and comprises information of completed operations and metadata;
the queue management is used for realizing a ring buffer area of the processor, and the ring buffer area is accessible by DMA;
queue management is used to maintain queue state information, including pointers to ring buffer DMA addresses, ring buffer size, producer and consumer pointers, and references to associated completion queues;
queue management realizes the management of the states of at least thousands of queues, realizes multi-queue parallel reading operation and multi-queue parallel dequeue/enqueue request operation;
the transmission scheduling module is used for designating queues to transmit, generating commands for the transmission engine, starting the transmission operation of the corresponding transmission queues, and transmitting the queue states of all the queues; scheduling operations on the associated data paths; for achieving load balancing of message flows between ports; the transmission scheduling module comprises a sending engine, a receiving engine and a cache RAM;
the cache RAM is used to temporarily store message packets during DMA operations;
the sending engine is used for scheduling the sending operation of the data packet, sending the message request of the appointed queue, and after the data packet is transmitted to the network through the Ethernet Media Access Controller (MAC) and the physical interface layer (PHY), the sending engine starts to establish a completion record and transmits the completion record to the completion sending queue;
the receiving engine is used for scheduling the receiving operation of the data packet, the data packet transmitted in the network is processed through a physical interface layer PHY and an Ethernet media access controller MAC and then transmitted to the receiving engine after being processed through a hash algorithm, the receiving engine sends a writing operation request, namely the data packet data is written into a processor memory after the writing operation is finished, and a finishing record is generated;
the MAC+PHY module comprises an Ethernet media access controller MAC and a physical interface layer PHY, and is used for determining a target receiving queue of an incoming data packet through a hash algorithm when receiving a direction, generating a command for a receiving engine and scheduling the operation of a relevant receiving data path.
4. An RDMA-based receiver, in which a receiver RDMA device is disposed, comprising: the second information processing module and the second RDMA transmission operation module;
the second information processing module is used for creating a work queue and a registered memory;
the second RDMA transmission operation module is used for receiving the message request sent by the information source terminal, processing second data to be processed based on the command word carried by the message request, generating second information data, and then performing remote memory access, wherein the remote memory access comprises the steps of generating a received message request and sending the data request to a cache area of the local node.
5. The RDMA-based receiver of claim 4, wherein: the second RDMA transmission operation module is integrated in the embedded system, and the second RDMA transmission operation module comprises: a second information operation unit and a second RDMA transmission unit;
the second information operation unit is used for acquiring second data to be processed from the memory, processing the second data to be processed according to the command word carried by the message request so as to generate second information data, and generating a message receiving request based on the second information data;
and the second RDMA transmission unit is used for acquiring the message request sent by the receiving information source end and sending the received message request to the local node cache area.
6. An RDMA-based data transfer system, comprising: a transmitting end and at least one receiving end; wherein,,
the sender is an RDMA-based sender according to any of claims 1-3;
the receiver is an RDMA-based receiver as claimed in any of claims 4-5.
7. The RDMA-based data transfer system as recited in claim 6, wherein: the sending end and the receiving end are connected through an Ethernet switching module network, and the Ethernet switching module supports the data center to bridge the DCB and control the PFC with priority.
8. An RDMA-based data transfer method employing the RDMA-based data transfer system of any of claims 6-7, the steps comprising:
step S100: the first information processing module of the transmitting end distributes annular transmission multi-queues, creates corresponding completion multi-queues, registers a memory area distribution buffer area to the first RDMA transmission operation module, and data in the memory buffer area is transmitted to the receiving end, namely a target remote memory;
the second information processing module of the receiving end distributes annular receiving multi-queues, creates corresponding finishing multi-queues, distributes buffer areas to the registration area of the second RDMA transmission operation module and is used for storing data sent by the sending end, namely the information source end;
step S200: the first information processing module of the transmitting end acquires information processing requirements, forms a work queue element WQE according to the information processing requirements, generates a work request, and adds the work request into a work queue; the pointer in the work queue element WQE points to a section of memory buffer, and the data in the buffer is transmitted to the memory at the destination remote end;
the first RDMA transmission operation module of the sending end obtains a work request from a work queue, consumes a work queue element WQE in the work request, converts first data to be processed in a memory buffer area pointed by a work queue element WQE pointer into first information data, generates a message request based on the first information data, and sends the message request to the second RDMA transmission operation module of the receiving end through a network data transmission channel;
step S300: the second RDMA transmission operation module of the receiving end receives the message request sent by the sending end and writes the data into the corresponding memory buffer area;
step S400: when a first pending data transfer is completed, the first RDMA transfer operation module creates a completion queue element CQE, which is placed into the completion queue CQ, indicating that the data transfer has completed.
9. The RDMA-based data transfer method as recited in claim 8, wherein: in step S400, the first RDMA transfer operation module generates a completion queue element CQE every time it consumes a work queue element WQE;
in parallel, the first RDMA transmission operation module consumes all work queue elements WQEs, generates all corresponding completion queue elements CQEs, that is, the sending operation of all work queue elements WQEs is completed, and the message sending request is completed.
10. The RDMA-based data transfer method as recited in claim 8, wherein: in step S400, if an error occurs, the first RDMA transfer operation module creates a completion queue element CQE, where the current completion queue element CQE contains a field for recording the status of the error transfer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310583845.7A CN116471242A (en) | 2023-05-23 | 2023-05-23 | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310583845.7A CN116471242A (en) | 2023-05-23 | 2023-05-23 | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116471242A true CN116471242A (en) | 2023-07-21 |
Family
ID=87175571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310583845.7A Pending CN116471242A (en) | 2023-05-23 | 2023-05-23 | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116471242A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117312202A (en) * | 2023-11-30 | 2023-12-29 | 浙江国利信安科技有限公司 | System on chip and data transmission method for system on chip |
| CN118093499A (en) * | 2024-02-06 | 2024-05-28 | 贝格迈思(深圳)技术有限公司 | Data transmission method, device, equipment and storage medium for remote memory access |
| CN118869477A (en) * | 2024-06-26 | 2024-10-29 | 浙江大学 | Low latency transmission system for RDMA network cards |
-
2023
- 2023-05-23 CN CN202310583845.7A patent/CN116471242A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117312202A (en) * | 2023-11-30 | 2023-12-29 | 浙江国利信安科技有限公司 | System on chip and data transmission method for system on chip |
| CN117312202B (en) * | 2023-11-30 | 2024-03-01 | 浙江国利信安科技有限公司 | System on chip and data transmission method for system on chip |
| CN118093499A (en) * | 2024-02-06 | 2024-05-28 | 贝格迈思(深圳)技术有限公司 | Data transmission method, device, equipment and storage medium for remote memory access |
| CN118093499B (en) * | 2024-02-06 | 2024-11-19 | 贝格迈思(深圳)技术有限公司 | Data transmission method, device, equipment and storage medium for remote memory access |
| CN118869477A (en) * | 2024-06-26 | 2024-10-29 | 浙江大学 | Low latency transmission system for RDMA network cards |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7263103B2 (en) | Receive queue descriptor pool | |
| EP1581875B1 (en) | Using direct memory access for performing database operations between two or more machines | |
| US9344490B2 (en) | Cross-channel network operation offloading for collective operations | |
| US8019902B2 (en) | Network adapter with shared database for message context information | |
| US12119958B2 (en) | Cross network bridging | |
| US8719456B2 (en) | Shared memory message switch and cache | |
| KR101502808B1 (en) | A method and system for improved multi-cell support on a single modem board | |
| CN116471242A (en) | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method | |
| US20090106470A1 (en) | Host bus adapter with multiple hosts | |
| US20160065659A1 (en) | Network operation offloading for collective operations | |
| CN102185833A (en) | Fiber channel (FC) input/output (I/O) parallel processing method based on field programmable gate array (FPGA) | |
| CN112953967A (en) | Network protocol unloading device and data transmission system | |
| CN117015764A (en) | Message communication between integrated computing devices | |
| CN118200253A (en) | RDMA UD transmission-oriented reliable communication method, electronic equipment and readable medium | |
| Yuanpeng et al. | Research on One-way Data Transmission Technology Based on RDMA | |
| CN118869775A (en) | Method and device for message transmission | |
| CN119003437A (en) | RDMA network data transmission method and device | |
| AU2003300885B2 (en) | Using direct memory access for performing database operations between two or more machines | |
| CN118093225A (en) | Subscription message transmission method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |