CN118606079B - A communication method and system based on socket interface - Google Patents

Abstract

The application provides a communication method and a system based on a socket interface, wherein the communication method comprises the following steps: under the condition that a local node and a remote node support a shared memory communication SMC protocol and a target link is reachable, registering a local memory address applied by a user state into a kernel to obtain local address registration information, wherein the target link corresponds to the type of bottom hardware equipment of the SMC; transmitting the local address registration information to a remote node, and acquiring the remote address registration information of the remote node; after the user mode writes the target data into the space of the local memory address, the target data is read from the user mode by a direct memory access mode and is transmitted to the space of the remote memory address corresponding to the remote address registration information, so that the technical problem that communication performance is reduced because a CPU is required to participate in memory copying and semantic conversion between the kernel mode and the user mode based on SMC protocol communication in the related art is solved.

Description

A communication method and system based on socket interface

Technical Field

The present application relates to the field of communication technology, and in particular to a communication method and system based on a socket interface.

Background Art

Large-scale data analysis and transaction processing, high-frequency trading systems, real-time data synchronization and backup, large-scale parallel computing tasks and artificial intelligence model training, as well as cloud computing and data center internal network optimization scenarios have strict requirements on the communication performance between hosts.

User-mode remote direct memory access (RDMA) verbs are the first choice in the above high-performance network communication scenario. They allow user-mode processes to bypass the kernel and the central processing unit (CPU) to directly operate data on the network hardware, which can significantly reduce latency and CPU load, and provide higher data throughput and lower data transmission latency. However, developing RDMA applications based on verbs requires developers and operation and maintenance teams to not only have professional network and hardware knowledge, but also to continuously invest time and resources in technology updates and system optimization to ensure efficient operation and stability of the system.

Shared Memory Communication (SMC) is a high-performance network protocol proposed by IBM. It can transparently replace TCP communication with RDMA communication without any modification, solving the problem of overly complex programming based on verbs interface. At the same time, it limits the maintenance cost within the kernel, greatly reducing the cost of developers and maintenance teams. However, in order to be fully compatible with standard TCP applications, SMC introduces instruction overhead and copy overhead for switching between user mode and kernel in its implementation. For this technical problem, relevant technologies have not yet proposed an effective solution.

Summary of the invention

The embodiments of the present application provide a communication method and system based on a socket interface to solve one or more of the above-mentioned technical problems.

In a first aspect, an embodiment of the present application provides a communication method based on a socket interface, comprising:

In the case where the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, registering the local memory address applied for in the user state into the kernel to obtain local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC;

Sending the local address registration information to the remote node, and receiving the remote address registration information sent by the remote node;

After the user state writes the target data into the space of the local memory address, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information.

In a second aspect, an embodiment of the present application provides a socket interface, including:

The first sub-interface is used to register the local memory address applied by the user state into the kernel to obtain local address registration information when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, wherein the target link corresponds to the underlying hardware device type of the SMC, send the local address registration information to the remote node, and receive the remote address registration information sent by the remote node;

The second sub-interface is used to read the target data from the user state through direct memory access and transfer the target data to the space of the remote memory address corresponding to the remote address registration information after the user state writes the target data into the space of the local memory address.

In a third aspect, this embodiment provides a communication system based on a socket interface, including:

A local node is used to register the local memory address applied by the user state into the kernel to obtain local address registration information when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, wherein the target link corresponds to the underlying hardware device type of the SMC; send the local address registration information to the remote node, and receive the remote address registration information sent by the remote node; after the user state writes the target data into the space of the local memory address, read the target data from the user state through direct memory access and transfer the target data to the space of the remote memory address corresponding to the remote address registration information;

The remote node is used to determine the remote address registration information, send the remote address registration information to the local node, and after receiving the target data, write the target data into the space of the remote memory address corresponding to the remote address registration information.

In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

In a sixth aspect, an embodiment of the present application provides a computer program product, including computer instructions, which implement any of the methods described above when executed by a processor.

Compared with the related art, this application has the following advantages:

The embodiment of the present application extends the socket interface. When the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, the extended socket interface registers the local memory address applied by the user state into the kernel to obtain the local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC; sends the local address registration information to the remote node, and obtains the remote address registration information of the remote node; after the user state writes the target data into the space of the local memory address, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information. In other words, the extended socket interface of the embodiment of the present application allows the application to use the user state memory area to directly communicate data without the CPU performing any operation on the communication data path, which solves the technical problem in the related technology that the CPU needs to participate in the memory copy and semantic conversion between the kernel state and the user state when communicating based on the SMC protocol, resulting in reduced communication performance, thereby achieving the technical effect of improving communication performance.

The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

FIG1 shows a flowchart of a communication method based on a socket interface provided in an embodiment of the present application;

FIG2 shows a schematic diagram of a communication architecture based on a socket interface provided in an embodiment of the present application;

FIG3 shows a flowchart of another communication method based on a socket interface provided in an embodiment of the present application;

FIG4 shows a block diagram of a socket interface structure provided in an embodiment of the present application;

FIG5 shows a structural block diagram of a communication device based on a socket interface provided in an embodiment of the present application;

FIG6 shows a structural block diagram of a communication system based on a socket interface provided in an embodiment of the present application;

FIG. 7 shows a block diagram of an electronic device for implementing an embodiment of the present application.

DETAILED DESCRIPTION

In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

Explanation of terms

TCP, Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol defined by IETF RFC 793. In the simplified computer network OSI model, it completes the functions specified by the fourth layer, the transport layer.

Socket, a socket is a software structure inside a computer network node, acting as an endpoint for sending and receiving data through the network, or a software endpoint for process communication within a node.

RDMA, remote direct memory access (RDMA) is a technology that bypasses the kernel of the remote host operating system to access data in its memory. Since it does not go through the operating system, it not only saves a lot of CPU resources, but also improves system throughput and reduces the system's network communication latency. It is widely used in large-scale parallel computer clusters.

ISM, internal shared memory (ISM) is a virtual device that provides a shared memory space for multiple logical partitions (LPARs) or virtual machines (VMs) running on the same physical server. It enables each partition or virtual machine to efficiently exchange data by directly accessing this shared memory area without going through the network protocol stack, thereby reducing latency and improving communication performance.

SMC/SMC-D/SMC-R, Shared Memory Communication (SMC) is a high-performance network protocol proposed by IBM. The SMC protocol transparently transmits the stream data of TCP socket in the form of shared memory access, providing a high-throughput, low-latency, and low-overhead network. When the shared memory operation in SMC is implemented based on IBM ISM devices, it is called Shared Memory Communication over DMA (SMC-D); when the shared memory operation in SMC is implemented based on RDMA devices, it is called Shared Memory Communication over RDMA (SMC-R).

verbs, verbs is a type of application programming interface (API) or command set, a programming abstraction used to perform specific operations. In RDMA, ibverbs is the name of the user-mode programming library that operates InfiniBand (IB) hardware (IB is a high-performance computer network communication standard, mainly used for high-speed data transmission, especially in data centers, high-performance computing and enterprise-level storage. IB is designed to support very high data transmission speed requirements and ensure data transmission reliability and low latency. IB hardware refers to various physical devices and components related to this communication standard). It defines a series of operations to manage RDMA resources, such as protection domains, memory regions, queue pairs, etc. These operations usually include creating, modifying and destroying resources, and starting data transmission operations.

Developing RDMA applications based on verbs faces many challenges: 1) Developers need to have a deep understanding of RDMA technical details, especially the learning threshold and programming complexity brought about by the abstraction of various network card hardware resources; 2) There is a lack of monitoring and operation and maintenance management capabilities provided by a unified operating system (OS), which requires applications using RDMA to implement them themselves; 3) The significant cost of maintaining a huge RDMA software stack. All of these require developers and operation and maintenance teams to not only have professional network and hardware knowledge, but also to continuously invest time and resources in technology updates and system optimization to ensure efficient operation and stability of the system.

In a related technology prior to this application, RDMA Communication Manager (RDMA-CM) is a set of APIs for managing RDMA (Remote Direct Memory Access) device connections, providing a mechanism to establish, monitor, disconnect, and destroy RDMA connections. RDMA-CM is built on the InfiniBand Trade Association (IBTA) specification and RDMA over Converged Ethernet (RoCE) technology. It aims to simplify RDMA programming and enable applications to more easily use RDMA technology for high-performance, low-latency network communications. However, its main disadvantages include: 1) It cannot reduce the significant cost of maintaining RDMA software stack dependencies, and developers and maintenance teams still need to have professional network and hardware knowledge, and continue to invest time and resources in technology updates and system optimization; 2) The development cost based on RDMA-CM is still relatively high compared to the socket interface, and developers may need to implement different code logic for different scenarios.

In another related technology before this application, an RDMA-based memory system designed on a distributed computing platform for large-scale memory-intensive computing scenarios exposes the memory of machines in the cluster as a shared address space. Applications can use transactions to allocate, read, write, and release objects in the address space. One of its important goals is to simplify programming, so that applications can more easily use RDMA technology for high-performance, low-latency network communications. Its main disadvantages include: 1) There is still the cost of user-mode maintenance of RDMA software stack dependencies, which requires developers and maintenance teams to have certain professional network and hardware knowledge; 2) The development and learning costs are high, which is still high compared to the socket interface. On the other hand, as a distributed memory system based on RDMA, the coding flexibility in general scenarios is weak, and developers may also need to implement additional code for different scenarios. For example, in the scenario where RDMA fails, developers may need to switch back to using the socket model coding.

Later, related technologies proposed a new kernel network protocol stack SMC-R, which can transparently replace TCP communication with RDMA communication without any modification, solving the problem of overly complex programming based on verbs interfaces, while limiting maintenance costs within the kernel, greatly reducing the cost of developers and maintenance teams. However, in order to be fully compatible with standard TCP applications, SMC introduces instruction overhead and copy overhead for switching between user mode and kernel in its implementation. At the same time, compared with the thin data path of user-mode verbs, the data path of SMC, which emphasizes universality, is more complex, which leads to a large performance gap between SMC-based communication and user-mode verbs-based communication.

In view of this, the embodiment of the present application provides a communication method based on a socket interface to fully or partially solve the above technical problems. The application scenarios of the embodiment of the present application include but are not limited to: latency-sensitive data query and processing (such as high-performance data query and processing scenarios such as memory database Redis, distributed memory object caching system Memcached, relational database PostgreSQL, etc.), high-throughput data transmission.

As shown in FIG1 , the above-mentioned communication method based on the socket interface includes:

S102, when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, register the local memory address applied for in the user state into the kernel to obtain local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC.

It should be noted that the above-mentioned local node and remote node can be a local host and a remote host, and the operating systems of the local host and the remote host include user state, kernel state, and underlying hardware devices. Before the local host and the remote host establish SMC communication, the local host protocol stack can first establish a TCP connection with the remote host in the kernel, use a special TCP option during the handshake process to indicate that it supports SMC, and confirm that the remote host also supports SMC. In addition, the local host and the remote host SMC-R protocol stack will create new or reuse existing RDMA resources to establish an available RDMA RC link, so that network transmission will be completed based on the RDMA network, that is, the target link is reachable. The above-mentioned underlying hardware devices may include: internal shared memory ISM devices, remote direct memory access RDMA devices. Correspondingly, the above-mentioned target link may include: a link implemented based on RDMA technology, and a link implemented based on direct memory access DMA technology. Correspondingly, the kernel module may be an SMC-R module or an SMC-D module. The above-mentioned local address registration information may include a local address registration ID.

S104: Send the local address registration information to the remote node, and receive the remote address registration information sent by the remote node.

Optionally, in an embodiment of the present application, the method of sending the local address registration information to the remote node includes but is not limited to: Method 1, copying the local address registration information to the kernel buffer, reading the local address registration information from the kernel buffer by direct memory access, and sending the local address registration information to the remote node. That is, the local address registration information is sent using the SMC general data path. Method 2, establishing a TCP connection to send the local address registration information. Accordingly, after receiving the local address registration information sent by the local node, the remote node sends the remote address registration information of the remote node to the local node in the same manner as the sending end.

S106, after the user state writes the target data into the space of the local memory address, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information. It should be noted that the above target data may include: application data, user state custom control data, etc.

As can be seen from the above, the embodiment of the present application extends the socket interface. When the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, the local memory address applied by the user state is registered in the kernel to obtain the local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC; the local address registration information is sent to the remote node, and the remote address registration information of the remote node is obtained; after the user state writes the target data into the space of the local memory address, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information. In other words, the extended socket interface of the embodiment of the present application can allow the application to use the user state memory area to directly communicate data without the CPU performing any operation on the communication data path, which solves the technical problem in the related technology that the CPU needs to participate in the memory copy and semantic conversion between the kernel state and the user state when communicating based on the SMC protocol, resulting in reduced communication performance, thereby achieving the technical effect of improving communication performance.

In a possible implementation, reading the target data from the user state by direct memory access and transmitting the target data to the space of the remote memory address corresponding to the remote address registration information includes: S11, mapping the target data to the kernel to obtain a data mapping relationship; S12, through the data mapping relationship, enabling the underlying hardware device to directly read the target data in the user state, and transmitting the target data to the space of the remote memory address corresponding to the remote address registration information through the network corresponding to the underlying hardware device. Optionally, in an embodiment of the present application, the implementation of S11 can be based on mmap memory mapping to map the target data to the kernel. Since the local memory address applied by the user state is registered in the kernel to obtain the local address registration information, based on the local address registration information, the target data can be mapped to the space corresponding to the local address registration information to obtain the mapping relationship between the target data and the space corresponding to the local address registration information. Optionally, the mapping relationship can include: the starting address of the data mapping, the data mapping length, etc. In SMC, the underlying hardware reads data from the kernel buffer through DMA technology, and the embodiment of the present application determines the above mapping relationship, so that the underlying hardware can directly access the target data in the user state, bypassing the CPU. Through the above steps S11~S12, while being compatible with the SMC protocol, the application is allowed to use the user state memory area to directly communicate data, avoiding memory copying and semantic conversion between kernel state and user state, and realizing that the CPU does not need to perform any operation on the data on the communication data path, greatly improving communication performance.

Optionally, the method of sending the local address registration information to the remote node may include: S21, copying the local address registration information to the kernel buffer; S22, reading the local address registration information from the kernel buffer by direct memory access, and sending the local address registration information to the remote node. That is, in an embodiment of the present application, the SMC general data path can be used to transmit local address registration information and remote address registration information. It should be noted that the SMC general data path can be a local node application (APP) that copies the data to be sent to the ring buffer allocated by the SMC protocol stack on this side for data transmission through the socket interface, and the SMC protocol stack directly and efficiently writes it to the ring buffer of the remote node through the RDMA write operation.

In addition to being able to directly exchange application data on user-state memory, the embodiments of the present application can also implement information notification. Specifically, the above-mentioned target data may include: application data, user-state custom control data, wherein the user-state custom control data includes at least one of the following: remote address registration information, write offset information, write length information, read offset information, and read length information. Optionally, application data and user-state custom control data can be read from the user state through direct memory access and the application data and user-state custom control data can be transmitted to the space of the remote memory address corresponding to the remote address registration information. That is, the extended socket interface provided by the embodiments of the present application can not only be able to directly exchange application data, but also allow users to exchange custom control message data, which further improves the performance while greatly improving the flexibility of user coding.

The embodiments of the present application are described below with reference to specific examples. It should be noted that in this example, SMC-R is used as an example for description.

Figure 2 is the overall design framework of this example. It includes user state, kernel state, and hardware (including RDMA devices). SMC-R is a kernel module that implements the SMC-R protocol. SMC-R works in kernel space, supports network behaviors described by user state programs (APP) through socket interfaces, and uses IB verbs interfaces to implement RDMA network transmission. Send_buf is a ring buffer allocated by the SMC-R kernel protocol stack for connecting data transmission. The receive buffer is omitted in Figure 2. The extended socket interface provided in this example allows users to actively create user state memory area mappings and provide them to SMC for direct data exchange. Data transmission no longer depends on kernel state and user state copies. On this basis, in the process of data exchange, the extended socket interface also allows users to exchange custom control messages, which further improves performance while greatly improving the flexibility of user coding. In Figure 2, the left side is the data transmission path proposed in this example, and the right side is the data transmission path of SMC-R in the related technology.

Under the above design framework, a complete host-to-host communication is described as an example. This method includes: handshake establishment (using a standard socket interface to create an SMC connection); resource preparation and registration (the sender applies for a memory local_addr in the local address space; uses the extended socket interface to register the local address memory local_addr to the SMC connection, and obtains the local address registration information local_id; uses the SMC-R general data path to send the local address registration information local_id to the remote end; waits for event notification and obtains the remote memory address registration information remote_id); sends data (in user mode, writes the application data directly to the local memory address local_addr, and uses the extended socket interface to move the data corresponding to the local_addr address to the remote address space corresponding to the remote_id address information; if it is necessary to notify the other end of this write, use the SMC-R general data path to notify the other end and pass the write information, including address information (remote_id), write offset, write length, etc.). Receive data (wait for event notification, obtain the write event information of the other end, such as local address information (local_id), write offset, write length, etc. Directly read and process the data of the local address local_addr corresponding to local_id), see S301~S311 in Figure 3. This example uses a socket-based interface for programming, which greatly reduces the learning cost and development cost. At the same time, the creation or destruction of RDMA-related resources is completely handled by the kernel SMC, and the user process no longer needs to handle complex RDMA verbs operations. The extended socket interface allows users to bypass the data path of the SMC ring buffer and directly exchange data and notify information on the user-defined user-mode memory, eliminating the memory copy and semantic conversion between kernel mode and user mode, and greatly improving communication performance based on the native SMC.

In summary, the embodiment of the present application is based on the kernel protocol stack and places the use of RDMA in the kernel SMC module, which greatly reduces the developer's development cost and the management and maintenance cost of the huge RDMA software stack. In most scenarios, latency and throughput benefits close to user-state verbs can be obtained, which not only avoids the cost of directly using user-state RDMA verbs, but also solves the problem that SMC's performance is not as good as ibverbs in high-performance scenarios. In addition, using the SMC protocol, the control plane is compatible with the socket interface, and TCP-based applications do not need to be modified. The data plane adds a very simple interface based on the SMC socket, with low learning cost, implemented in the kernel state, and strong reusability. There is no need to implement different code logic for different scenarios.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a socket interface. As shown in FIG4 is a structural block diagram of a socket interface of an embodiment of the present application, the interface may include:

The first sub-interface 42 is used to register the local memory address applied by the user state into the kernel to obtain local address registration information when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, wherein the target link corresponds to the underlying hardware device type of the SMC, send the local address registration information to the remote node, and receive the remote address registration information sent by the remote node;

The second sub-interface 44 is used to read the target data from the user state through a direct memory access method and transfer the target data to the space of the remote memory address corresponding to the remote address registration information after the target data is written into the space of the local memory address in the user state;

Through the extended socket interface shown in FIG4, when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, the local memory address applied by the user state is registered in the kernel to obtain the local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC; the local address registration information is sent to the remote node, and the remote address registration information of the remote node is obtained; after the target data is written into the space of the local memory address in the user state, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information. In other words, the extended socket interface of the embodiment of the present application can allow the application to use the user state memory area to directly communicate data without the CPU performing any operation on the communication data path, which solves the technical problem in the related technology that the CPU needs to participate in the memory copy and semantic conversion between the kernel state and the user state when communicating based on the SMC protocol, resulting in reduced communication performance, thereby achieving the technical effect of improving communication performance.

Optionally, the target data includes: application data, user-defined control data, wherein the user-defined control data includes at least one of the following: remote address registration information, write offset information, write length information, read offset information, and read length information.

Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a communication device based on a socket interface. As shown in FIG5 , a structural block diagram of a communication device based on a socket interface in an embodiment of the present application may include:

The registration module 52 is used to register the local memory address applied by the user state into the kernel when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, and obtain the local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC. It should be noted that the above-mentioned local node and remote node can be a local host and a remote host, and the operating system of the local host and the remote host includes user state, kernel state, and underlying hardware devices. Before the local host and the remote host establish SMC communication, the local host protocol stack can first establish a TCP connection with the remote host in the kernel, use a special TCP option during the handshake process to indicate that it supports SMC, and confirm that the remote host also supports SMC. In addition, the local host and the remote host SMC-R protocol stack will create new or reuse existing RDMA resources, establish an available RDMA RC link, so that the network transmission will be completed based on the RDMA network, that is, the target link is reachable. The above-mentioned underlying hardware devices may include: internal shared memory ISM devices, remote direct memory access RDMA devices. Correspondingly, the target link may include: a link implemented based on RDMA technology, a link implemented based on direct memory access DMA technology. Correspondingly, the kernel module may be an SMC-R module, an SMC-D module. The local address registration information may include a local address registration ID.

Processing module 54 is used to send the local address registration information to the remote node and receive the remote address registration information sent by the remote node. Optionally, in an embodiment of the present application, the method of sending the local address registration information to the remote node includes but is not limited to: Method 1, copying the local address registration information to the kernel buffer, reading the local address registration information from the kernel buffer by direct memory access, and sending the local address registration information to the remote node. That is, the local address registration information is sent using the SMC general data path. Method 2, establishing a TCP connection to send the local address registration information. Accordingly, after receiving the local address registration information sent by the local node, the remote node sends the remote address registration information of the remote node to the local node in the same manner as the sending end.

The communication module 56 is used to read the target data from the user state through direct memory access and transmit the target data to the space of the remote memory address corresponding to the remote address registration information after the target data is written into the space of the local memory address in the user state. It should be noted that the above target data may include: application data, user state custom control data, etc.

Through the device shown in Figure 5, the socket interface is extended. When the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, the local memory address applied by the user state is registered in the kernel to obtain the local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC; the local address registration information is sent to the remote node, and the remote address registration information of the remote node is obtained; after the user state writes the target data into the space of the local memory address, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information. In other words, the extended socket interface of the embodiment of the present application can allow the application to use the user state memory area to directly communicate data without the CPU performing any operation on the communication data path, which solves the technical problem that the CPU needs to participate in the memory copy and semantic conversion between the kernel state and the user state when communicating based on the SMC protocol in the related technology, resulting in reduced communication performance, thereby achieving the technical effect of improving communication performance.

In a possible implementation, the communication module 56 includes: a mapping unit for mapping the target data into the kernel to obtain a data mapping relationship; a communication unit for enabling the underlying hardware device to directly read the target data in the user state through the data mapping relationship, and transmit the target data to the space of the remote memory address corresponding to the remote address registration information through the network corresponding to the underlying hardware device. Optionally, in an embodiment of the present application, the implementation of the above S11 can be based on mmap memory mapping to map the target data into the kernel. Since the local memory address applied for by the user state is registered in the kernel to obtain the local address registration information, the target data can be mapped to the space corresponding to the local address registration information based on the local address registration information to obtain the mapping relationship between the target data and the space corresponding to the local address registration information. Optionally, the mapping relationship can include: the starting address of the data mapping, the data mapping length, etc. In the SMC, the underlying hardware reads data from the kernel buffer through the DMA technology, and the embodiment of the present application determines the above mapping relationship, so that the underlying hardware can directly access the target data in the user state and bypass the CPU. Through the above steps S11~S12, while being compatible with the SMC protocol, the application is allowed to use the user state memory area to directly communicate data, avoiding memory copying and semantic conversion between kernel state and user state, and achieving the CPU not needing to perform any operations on the data on the communication data path, thereby greatly improving communication performance.

The processing module 54 includes: a copy unit, which is used to copy the local address registration information to the buffer of the kernel; a processing unit, which is used to read the local address registration information from the kernel buffer by direct memory access, and send the local address registration information to the remote node. That is, in an embodiment of the present application, the SMC general data path can be used to transmit local address registration information and remote address registration information. It should be noted that the SMC general data path can be a local node application (APP) that copies the data to be sent to the ring buffer allocated by the SMC protocol stack on this side for data transmission through the socket interface, and the SMC protocol stack directly and efficiently writes it into the ring buffer of the remote node through the RDMA write operation.

In addition to being able to directly exchange application data on the user-state memory, the above-mentioned device can also implement information notification. Specifically, the above-mentioned target data may include: application data, user-state custom control data, wherein the user-state custom control data includes at least one of the following: remote address registration information, write offset information, write length information, read offset information, read length information. Optionally, the application data and user-state custom control data can be read from the user state through direct memory access and the application data and user-state custom control data can be transmitted to the space of the remote memory address corresponding to the remote address registration information. That is, the extended socket interface provided in the embodiment of the present application can not only be able to directly exchange application data, but also allow users to exchange custom control message data, which further improves the performance while greatly improving the flexibility of user coding.

The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a communication system based on a socket interface. As shown in FIG6 , a structural block diagram of a communication system based on a socket interface in an embodiment of the present application can include:

The local node 62 is used to register the local memory address applied by the user state into the kernel to obtain the local address registration information when the local node and the remote node support the shared memory communication SMC protocol and the target link is reachable, wherein the target link corresponds to the underlying hardware device type of the SMC; send the local address registration information to the remote node, and receive the remote address registration information sent by the remote node; after the target data is written into the space of the local memory address in the user state, read the target data from the user state through direct memory access and transfer the target data to the space of the remote memory address corresponding to the remote address registration information. It should be noted that the above-mentioned local node and remote node can be a local host and a remote host, and the operating systems of the local host and the remote host include user state, kernel state, and underlying hardware devices. Before the local host and the remote host establish SMC communication, the local host protocol stack can first establish a TCP connection with the remote host in the kernel, use a special TCP option in the handshake process to indicate that it supports SMC, and confirm that the remote host also supports SMC. In addition, the local host and the remote host SMC-R protocol stack will create new or reuse existing RDMA resources to establish an available RDMA RC link, so that network transmission will be completed based on the RDMA network, that is, the target link is reachable. The above-mentioned underlying hardware devices may include: internal shared memory ISM devices, remote direct memory access RDMA devices. Correspondingly, the above-mentioned target link may include: a link implemented based on RDMA technology, a link implemented based on direct memory access DMA technology. Correspondingly, the kernel module may be an SMC-R module or an SMC-D module. The above-mentioned local address registration information may include a local address registration ID. Optionally, in an embodiment of the present application, the method of sending the local address registration information to the remote node includes but is not limited to: Method 1, copying the local address registration information to the kernel buffer, reading the local address registration information from the kernel buffer by direct memory access, and sending the local address registration information to the remote node. That is, the local address registration information is sent using the SMC general data path. Method 2, establishing a TCP connection to send the local address registration information. Correspondingly, after receiving the local address registration information sent by the local node, the remote node sends the remote address registration information of the remote node to the local node in the same manner as the sending end.

The remote node 64 is used to determine the remote address registration information, send the remote address registration information to the local node, and after receiving the target data, write the target data into the remote memory address space corresponding to the remote address registration information.

Through the system shown in FIG6 , when the local node 62 and the remote node 64 support the shared memory communication SMC protocol and the target link is reachable, the local memory address applied by the user state is registered in the kernel to obtain the local address registration information, wherein the target link corresponds to the underlying hardware device type of the SMC; the local address registration information is sent to the remote node 64, and the remote address registration information of the remote node 64 is obtained; after the target data is written into the space of the local memory address in the user state, the target data is read from the user state through direct memory access and the target data is transferred to the space of the remote memory address corresponding to the remote address registration information. In other words, the embodiment of the present application extends the socket interface, which allows the application to use the user state memory area to directly communicate data without the CPU performing any operation on the communication data path, thereby solving the technical problem in the related technology that the CPU needs to participate in the memory copy and semantic conversion between the kernel state and the user state when communicating based on the SMC protocol, resulting in reduced communication performance, thereby achieving the technical effect of improving communication performance.

The functions of each module in each system of the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

FIG7 is a block diagram of an electronic device for implementing an embodiment of the present application. As shown in FIG7 , the electronic device includes: a memory 701 and a processor 702, wherein the memory 701 stores a computer program that can be run on the processor 702. When the processor 702 executes the computer program, the method in the above embodiment is implemented. The number of the memory 701 and the processor 702 can be one or more.

The electronic device also includes:

The communication interface 703 is used to communicate with external devices and perform data exchange transmission.

If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the memory 701, the processor 702 and the communication interface 703 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG. 7, but it does not mean that there is only one bus or one type of bus.

Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

The logic and/or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims (14)

1. A socket interface-based communication method, comprising:

Under the condition that a local node and a remote node support a shared memory communication SMC protocol and a target link is reachable, registering a local memory address applied by a user state into a kernel to obtain local address registration information, wherein the target link corresponds to a bottom hardware device type of the SMC;

Transmitting the local address registration information to the remote node, and receiving the remote address registration information transmitted by the remote node;

And after the user state writes the target data into the space of the local memory address, reading the target data from the user state in a direct memory access mode and transmitting the target data into the space of the remote memory address corresponding to the remote address registration information.

2. The method of claim 1, wherein reading the target data from the user state and transmitting the target data to a space of a remote memory address corresponding to the remote address registration information by a direct memory access manner comprises:

Mapping the target data into the kernel to obtain a data mapping relation;

And the bottom hardware device directly reads the target data of the user mode through the data mapping relation, and transmits the target data to a space of a remote memory address corresponding to the remote address registration information through a network corresponding to the bottom hardware device.

3. The method of claim 2, wherein mapping the target data into the kernel to obtain a data mapping relationship comprises:

Acquiring the local address registration information;

And mapping the target data into a space corresponding to the local address registration information to obtain a mapping relation between the target data and the space corresponding to the local address registration information.

4. The method of claim 1, wherein transmitting the home address registration information to the remote node comprises:

copying the local address registration information into the kernel buffer;

and reading the local address registration information from the kernel buffer area in a direct memory access mode, and sending the local address registration information to the remote node.

5. The method of claim 1, wherein the target data comprises: application data and user-mode custom control data, wherein the user-mode custom control data comprises at least one of the following: remote address registration information, write offset information, write length information, read offset information, read length information.

6. The method of any of claims 1 to 5, wherein the underlying hardware device comprises an internal shared memory ISM device or a remote direct memory access RDMA device.

7. The method of claim 6, wherein the target link is a direct memory access, DMA, link when the underlying hardware device is the ISM device and an RDMA link when the underlying hardware device is the RDMA device.

8. A socket interface, comprising:

The first sub-interface is used for registering a local memory address applied by a user mode into a kernel to obtain local address registration information under the condition that a local node and a remote node support a shared memory communication SMC protocol and a target link is reachable, wherein the target link corresponds to the type of bottom hardware equipment of the SMC, the local address registration information is sent to the remote node, and the remote address registration information sent by the remote node is received;

And the second sub-interface is used for reading the target data from the user state in a direct memory access mode after the target data are written into the space of the local memory address by the user state and transmitting the target data into the space of the remote memory address corresponding to the remote address registration information.

9. The socket interface of claim 8, wherein the target data comprises: application data and user-mode custom control data, wherein the user-mode custom control data comprises at least one of the following: remote address registration information, write offset information, write length information, read offset information, read length information.

10. A socket interface-based communication system, comprising:

The local node is used for registering a local memory address applied by a user state into the kernel to obtain local address registration information under the condition that the local node and the remote node support a shared memory communication SMC protocol and a target link is reachable, wherein the target link corresponds to the type of bottom hardware equipment of the SMC; transmitting the local address registration information to the remote node, and receiving the remote address registration information transmitted by the remote node; after the user state writes the target data into the space of the local memory address, reading the target data from the user state in a direct memory access mode and transmitting the target data into the space of the remote memory address corresponding to the remote address registration information;

The remote node is used for determining remote address registration information, sending the remote address registration information to the local node, and writing the target data into a remote memory address space corresponding to the remote address registration information after receiving the target data.

11. The system of claim 10, wherein the underlying hardware device comprises an internal shared memory ISM device or a remote direct memory access RDMA device, the target link being a direct memory access DMA link when the underlying hardware device is the ISM device, and an RDMA link when the underlying hardware device is the RDMA device.

12. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1-7 when the computer program is executed.

13. A computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-7.

14. A computer program product comprising computer instructions which, when executed by a processor, implement the method of any of claims 1-7.