CN113326155A

CN113326155A - Information processing method, device, system and storage medium

Info

Publication number: CN113326155A
Application number: CN202110720942.7A
Authority: CN
Inventors: 周炜; 雍帅; 霍道安; 刘烨
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-06-28
Filing date: 2021-06-28
Publication date: 2021-08-31
Anticipated expiration: 2041-06-28
Also published as: CN113326155B

Abstract

The embodiment of the application discloses an information processing method, which is applied to a client node and comprises the following steps: detecting a first Remote Procedure Call (RPC) request generated by a first target application, and determining a server node and a target communication channel based on the first target application; generating a Remote Direct Memory Access (RDMA) request based on the first RPC request; sending the RDMA request to the server-side node by adopting RDMA bilateral primitive through the target communication channel; receiving an RDMA response sent by the server end node by adopting the RDMA bilateral primitive through the target communication channel; wherein the RDMA reply is generated by the server end node in response to the RDMA request. The embodiment of the application also discloses a client device, a server device, a system and a storage medium.

Description

Information processing method, device, system and storage medium

Technical Field

The present application relates to the field of distributed system technologies, and in particular, to an information processing method, apparatus, system, and storage medium.

Background

With The development of The Internet of Things (IOT) and industry 4.0, cloud computing has also been rapidly emerging. Distributed storage has also gained wide attention and development as the largest application of cloud computing. The distributed storage is used as network storage, clusters are constructed by using a network interconnection mode for each storage node, and more reliable high-performance storage side service is provided to the outside. One commonly used distributed storage service is Remote Direct Memory Access (RDMA), which completely bypasses a Central Processing Unit (CPU) through a network card to realize Memory data exchange between two devices. RDMA is a network transmission technology realized by hardware, which can greatly improve the network transmission efficiency and help network Input/Output (IO) intensive services to obtain lower time delay and larger throughput.

In the existing RDMA-implemented distributed storage system, in the communication process, both communication parties need to copy and analyze the memory, which results in larger resource consumption in the communication process and lower performance of the RDMA-implemented distributed storage system.

Disclosure of Invention

In view of this, embodiments of the present application are expected to provide an information processing method, an apparatus, a system, and a storage medium, which solve the problem that resource consumption is large in the communication process due to the fact that the memory needs to be copied and analyzed in the communication process between two communication parties in a distributed storage system implemented by RDMA at present, and provide a communication information processing method in a distributed system, which implement that the memory does not need to be copied and analyzed in the communication process, reduce resource consumption in the communication process in the distributed storage system, and effectively improve the performance of the distributed storage system implemented by RDMA.

In order to achieve the purpose, the technical scheme of the application is realized as follows:

in a first aspect, an information processing method is applied to a client node, and the method includes:

detecting a first Remote Procedure Call (RPC) request generated by a first target application, and determining a server node and a target communication channel based on the first target application;

generating a Remote Direct Memory Access (RDMA) request based on the first RPC request;

sending the RDMA request to the server-side node by adopting RDMA bilateral primitive through the target communication channel;

receiving an RDMA response sent by the server end node by adopting the RDMA bilateral primitive through the target communication channel; wherein the RDMA reply is generated by the server end node in response to the RDMA request.

In a second aspect, an information processing method is applied to a service end node, and the method includes:

receiving an RDMA request sent by a client node by adopting an RDMA bilateral primitive through a target communication channel; wherein the RDMA request is generated by the client node based on a first RPC request corresponding to a first target application;

generating an RDMA response based on the RDMA request;

sending the RDMA response to the client node using the RDMA bilateral primitive over the target communication channel.

In a third aspect, a client device, the device comprising: the device comprises a first determining unit, a first generating unit, a first sending unit and a first receiving unit; wherein:

the first determining unit is used for detecting a first RPC request generated by a first target application, and determining a server node and a target communication channel based on the first target application;

the first generation unit is used for generating an RDMA request for remote direct memory access based on the first RPC request;

the first sending unit is used for sending the RDMA request to the server end node by adopting RDMA bilateral primitive through the target communication channel;

the first receiving unit is used for receiving the RDMA response sent by the server-end node by adopting the RDMA bilateral primitive through the target communication channel; wherein the RDMA reply is generated by the server end node in response to the RDMA request.

In a fourth aspect, a server-side apparatus, the apparatus comprising: a second receiving unit, a second generating unit and a second transmitting unit; wherein:

the second receiving unit is used for receiving the RDMA request sent by the client node through the target communication channel by adopting the RDMA bilateral primitive;

the second generation unit is used for generating an RDMA response based on the RDMA request;

the second sending unit is configured to send the RDMA response to the client node using the RDMA bilateral primitive through the target communication channel.

In a fifth aspect, an information processing system includes: a client node and a server node; wherein:

the client node, configured to implement the steps of the information processing method according to any one of the above;

the server end node is used for realizing the steps of the information processing method.

A sixth aspect is a storage medium having stored thereon an information processing program which, when executed by a processor, implements the steps of the information processing method according to any one of the above.

According to the information processing method, the device, the system and the storage medium provided by the embodiment of the application, if a first RPC request generated by a first target application is detected, a client node determines a server node and a target communication channel based on the first target application, generates an RDMA request based on the first RPC request, sends the RDMA request to the server node through the target communication channel by adopting an RDMA bilateral primitive, generates an RDMA response based on the RDMA request after the server node receives the RDMA request sent by the client node through the target communication channel by adopting the RDMA bilateral primitive, and sends the RDMA response to the client node through the target communication channel by adopting the RDMA bilateral primitive. Thus, the client node determines the server node and the target communication channel with a first RPC request generated by a first target application, and generates an RDMA request based on the first RPC request, the RDMA request is then sent to the server-side node over the target communication channel to cause the server-side node to respond to the RDMA request, generate a corresponding RDMA reply, the generated RDMA response is sent to the client node through the target communication channel, the problem that the resource consumption in the communication process is large due to the fact that the memory needs to be copied and analyzed in the communication process of two communication parties in the existing RDMA-realized distributed storage system is solved, the communication information processing method under the distributed system is provided, the memory does not need to be copied and analyzed in the communication process, the resource consumption in the communication process of the distributed storage system is reduced, and the performance of the RDMA-realized distributed storage system is effectively improved.

Drawings

Fig. 1 is a schematic flowchart of an information processing method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of another information processing method according to an embodiment of the present application;

fig. 3 is a schematic flowchart of another information processing method according to an embodiment of the present application;

fig. 4 is a schematic flowchart of another information processing method according to an embodiment of the present application;

fig. 5 is a schematic flowchart of an information processing method according to another embodiment of the present application;

fig. 6 is a schematic flowchart of another information processing method according to another embodiment of the present application;

fig. 7 is a schematic flowchart of another information processing method according to another embodiment of the present application;

fig. 8 is a schematic flowchart of another information processing method according to another embodiment of the present application;

fig. 9 is a schematic flowchart of an information processing method according to another embodiment of the present application;

FIG. 10 is a block diagram of an information handling system according to an embodiment of the present disclosure;

FIG. 11 is a diagram illustrating a data structure of an RPC request according to an embodiment of the present application;

fig. 12 is a schematic diagram illustrating a data structure of a request header according to an embodiment of the present application;

fig. 13 is a schematic data structure diagram of request data according to an embodiment of the present application;

fig. 14 is a schematic diagram of a data structure of a memory in request data according to an embodiment of the present application;

fig. 15 is a schematic data structure diagram of an RDMA request according to an embodiment of the present application;

fig. 16 is a schematic data structure diagram of an RDMA transfer request according to an embodiment of the present application;

fig. 17 is a schematic diagram illustrating a data structure of a memory in an RDMA transfer request according to an embodiment of the present application;

FIG. 18 is a diagram illustrating a data structure of an RPC response according to an embodiment of the present application;

fig. 19 is a schematic data structure diagram of an RDMA response according to an embodiment of the present application;

fig. 20 is a schematic structural diagram of a client device according to an embodiment of the present application;

fig. 21 is a schematic structural diagram of a server device according to an embodiment of the present application;

fig. 22 is a schematic structural diagram of an information processing system according to an embodiment of the present application.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

An embodiment of the present application provides an information processing method, which is applied to a client node, and as shown in fig. 1, the method includes the following steps:

step 101, detecting a first Remote Procedure Call (RPC) request generated by a first target application, and determining a server node and a target communication channel based on the first target application.

In the embodiment of the application, the client node and the server node communicate with each other in an RPC form, wherein the first target application is an upper-layer application on the client node side in an RPC framework. The client node is a message initiator, the server node is a message receiver and responder, after receiving the request sent by the client node, the upper layer application at the side of the server node carries out corresponding processing on the request sent by the client node and obtains a corresponding response, and then the obtained corresponding response is returned to the client node. The target communication channel is an independent communication channel which is independently created when the client node communicates with the service end node.

The server node has a corresponding relation with the first target application, namely, the service requested by the first target application can be provided for the first target application.

Step 102, generating a Remote Direct Memory Access (RDMA) request based on the first Remote Procedure Call (RPC) request.

In the embodiment of the application, RPC communication is implemented between the client node and the server node, so that the client node needs to generate an RDMA request from the request content included in the first RPC request.

Illustratively, the first RPC request includes at least a communication Channel (Channel) pointer, a request header, request data, a callback function, and a callback parameter pointer, and the RDMA request is generated based on the request header in the first RPC request and a memory address in the first RPC request.

Step 103, sending the RDMA request to the server node by adopting the RDMA bilateral primitive through the target communication channel.

In an embodiment of the application, an RDMA request is sent to a server node via a target communication channel using an RDMA bilateral primitive, such as RDMA _ SEND.

And step 104, receiving the RDMA response sent by the server-end node by adopting the RDMA bilateral primitive through the target communication channel.

Wherein the RDMA reply is generated by the server end node in response to the RDMA request.

In an embodiment of the application, an RDMA response sent by a serving end node is received via a target communication channel using an RDMA bilateral primitive, such as RDMA _ RECV. Thus, the RDMA bilateral primitive mode is adopted for data transmission, and high-performance communication with low delay, low Central Processing Unit (CPU) overhead and low memory bandwidth overhead is realized.

According to the information processing method provided by the embodiment of the application, if a first RPC request generated by a first target application is detected, a client node determines a server node and a target communication channel based on the first target application, generates an RDMA request based on the first RPC request, transmits the RDMA request to the server node by using an RDMA bilateral primitive through the target communication channel, and finally receives an RDMA response transmitted by the server node by using the RDMA bilateral primitive through the target communication channel. Thus, the client node determines the server node and the target communication channel with a first RPC request generated by a first target application, and generates an RDMA request based on the first RPC request, the RDMA request is then sent to the server-side node over the target communication channel to cause the server-side node to respond to the RDMA request, generate a corresponding RDMA reply, the generated RDMA response is sent to the client node through the target communication channel, the problem that the resource consumption in the communication process is large due to the fact that the memory needs to be copied and analyzed in the communication process of two communication parties in the existing RDMA-realized distributed storage system is solved, the communication information processing method under the distributed system is provided, the memory does not need to be copied and analyzed in the communication process, the resource consumption in the communication process of the distributed storage system is reduced, and the performance of the RDMA-realized distributed storage system is effectively improved.

Based on the foregoing embodiments, an embodiment of the present application provides an information processing method, which is applied to a service end node, and as shown in fig. 2, the method includes the following steps:

step 201, receiving an RDMA request sent by a client node through a target communication channel by using an RDMA bilateral primitive.

Wherein the RDMA request is generated by the client node based on a first RPC request corresponding to the first target application.

In the embodiment of the application, the server node is a specific node that the client node determines from the first RPC request. It should be noted that the client node can establish communication connection with a plurality of server nodes, and each server node can also establish communication connection with a plurality of client nodes. The server node may employ a RDMA bilateral primitive, such as RDMA _ RECV, to receive the first RPC request sent by the client node.

Step 202, generate an RDMA response based on the RDMA request.

In the embodiment of the application, after the server-side node receives the RDMA request, the upper-layer application corresponding to the RDMA request in the server-side node is called to respond to the RDMA request, and the RDMA response is generated by the response information of the upper-layer application in the server-side node.

Step 203, sending RDMA response to the client node by using RDMA bilateral primitive through the target communication channel.

In an embodiment of the application, after the server node generates the RDMA response, it may be sent to the client node via an RDMA bilateral primitive, such as RDMA _ SEND.

According to the information processing method provided by the embodiment of the application, the server-side node receives the RDMA request sent by the client-side node through the target communication channel by adopting the RDMA bilateral primitive, generates the RDMA response based on the RDMA request, and finally sends the RDMA response to the client-side node through the target communication channel by adopting the RDMA bilateral primitive. Thus, the client node determines the server node and the target communication channel with a first RPC request generated by a first target application, and generates an RDMA request based on the first RPC request, the RDMA request is then sent to the server-side node over the target communication channel to cause the server-side node to respond to the RDMA request, generate a corresponding RDMA reply, the generated RDMA response is sent to the client node, the problem that the resource consumption in the communication process is large due to the fact that the memory needs to be copied and analyzed in the communication process of two communication parties in the existing RDMA-realized distributed storage system is solved, the communication information processing method under the distributed system is provided, the memory does not need to be copied and analyzed in the communication process, the resource consumption in the communication process of the distributed storage system is reduced, and the performance of the RDMA-realized distributed storage system is effectively improved.

Based on the foregoing embodiments, an embodiment of the present application provides an information processing method, which is shown in fig. 3 and includes the following steps:

step 301, detecting a first RPC request generated by the first target application, the client node determines an internet protocol, IP, address and port number indicated by the first target application.

Wherein, the first RPC request comprises: the channel pointer, the RPC request header, the request data, the callback function and the callback parameter pointer are used for pointing to the server-side node; wherein the first RPC request header includes at least: the RPC request processing method comprises the following steps of request type, data transmission direction, message version number and an application custom block, wherein the application custom block is used for realizing type conversion of a first RPC request according to requirements, and the request data at least comprises the following steps: at least one of remotely accessible memory information and reference count.

In this embodiment of the application, the IP address and the port number indicated by the first target application are the IP address and the port number corresponding to the server node.

Step 302, the client node creates a connection between the channel creation interface and the server node based on the IP address and the port number, and obtains a target communication channel.

In an embodiment of the application, a client node may invoke an RDMA programming library to create RDMA connection resources based on the determined IP address and port number of the serving node and establish an RDMA communication connection by creating a target communication channel.

In the foregoing embodiment, step 101 can be implemented through steps 301 to 302. Steps 301-302 may be implemented as a separate embodiment, and the target communication channel may be created only when there is information to be transferred between the client node and the server node, and may be removed when there is no information to be transferred between the client node and the server node.

Step 303, the client node generates a remote direct memory access RDMA request based on the first RPC request.

Step 304, the client sends the RDMA request to the server node through the target communication channel by using the RDMA bilateral primitive.

And 305, the server-side node receives the RDMA request sent by the client-side node through the target communication channel by adopting the RDMA bilateral primitive.

Step 306, the server end node generates an RDMA response based on the RDMA request.

Step 307, the server-side node sends the RDMA response to the client-side node through the target communication channel by using the RDMA bilateral primitive.

And step 308, the client node receives the RDMA response sent by the server node by adopting the RDMA bilateral primitive through the target communication channel.

Based on the foregoing embodiments, in other embodiments of the present application, referring to fig. 4, before the client node performs step 303, it is further configured to perform steps 309-310:

step 309, when detecting that the RPC request queue corresponding to the target communication channel is not full, the client node writes the first RPC request into the tail of the RPC request queue.

In the embodiment of the application, the RPC request queue is created when the client node creates the target communication channel, and is used for storing the RPC request issued by the upper layer application on the client node side. The RPC requests stored in the RPC request queue comply with the principle of first-in first-out, last-in last-out.

Step 310, the client node adds 1 to the reference count included in the first RPC request.

In the embodiment of the application, the initial value of the reference count in the first RPC request is 1, 1 is added when the request data in the first RPC request is referenced, 1 is subtracted when the request data in the first RPC request is released, and the memory to which at least one of the request data in the first request and the memory information in the first RPC request points is released when the reference count is subtracted to 0.

Based on the foregoing embodiments, in other embodiments of the present application, step 303 may be implemented by steps 303a to 303 c:

and step 303a, detecting that the state of the target communication channel is a target state, detecting that the first RDMA request queue is not full and the RPC request queue is not empty, and taking out the first RPC request from the queue head of the RPC request queue by the client node.

Wherein the first RDMA request queue is used to record RDMA requests that have been issued but not returned acknowledgements.

In the embodiment of the present application, the state that the target communication channel is the target state means a state that the target communication channel can normally transmit information, that is, a connection normal state. The RDMA requests stored in the first RDMA request queue also follow the first-in first-out, last-in last-out principle. And when the state of the target communication channel is detected to be a connection abnormal state, or the first RDMA request queue is full, or the RPC request queue is empty, the client node exits, and the subsequent operation is not executed. The client node takes the first RPC request out of the head of the RPC request queue, namely when the first RPC request moves to the head of the RPC request queue, the client node takes the first RPC request out of the head of the RPC request queue for subsequent operation.

Step 303b, the client node determines the RDMA identification information of the RDMA request to be generated.

In this embodiment of the present application, the RDMA identification information is identification information for uniquely identifying an RDMA request, and may be denoted as RDMA identity identification information.

Step 303c, the client node obtains the RPC request header, the at least one accessible memory information and the RDMA identification information from the first RPC request, and generates an RDMA request.

In the embodiment of the application, the client node puts an RPC request header in a first RPC request into a created RDMA request, puts at least one piece of accessible memory information into a region where a memory address is placed in the RDMA request, and sets RDMA identification information in the RDMA request, so that the RDMA request is obtained.

Based on the foregoing embodiments, in other embodiments of the present application, referring to fig. 5, before the client node performs step 303c, the client node is further configured to perform steps 311 to 312:

in step 311, the client node obtains at least one memory address included in the at least one accessible memory information.

In this embodiment of the present application, the client node obtains at least one memory address allocated to the memory remotely accessible by the server node, that is, the memory remotely accessible by the server node includes at least one memory.

Step 312, it is detected that at least one of the memory addresses includes at least one target address that is not registered as an RDMA memory, and the client node registers the memory corresponding to the at least one target address as an RDMA memory.

In the embodiment of the present application, the memory registered as the RDMA memory may be represented by a Key-Value (Key-Value) form. In this way, the registered RDMA memories all correspond to one keyword, and therefore, the keywords corresponding to different memory addresses can be searched to determine whether the acquired at least one memory address is registered, and if at least one unregistered target address exists, RDMA memory registration is performed on the at least one target address.

Based on the foregoing embodiments, in the embodiment of the present application, step 312 may be implemented by steps 312a to 312 b:

step 312a, the client node obtains the memory address information including the memory registered as RDMA memory.

In the embodiment of the present application, the memory address information registered as the RDMA memory may be represented in the form of an RDMA memory key map, but may also be represented in other forms, such as a list.

Step 312b, if the memory address information does not include at least one target address of the at least one memory address, registering the memory corresponding to the at least one target address as an RDMA memory.

In this embodiment of the present application, if the key word information corresponding to one target address is not found in the memory address information, it is determined that the memory corresponding to at least one target address is not registered as the RDMA memory, and therefore, the RDMA memory registration processing is performed on the memory corresponding to at least one target address, and specifically, the memory corresponding to at least one target address may be registered by calling an RDMA library function, so as to obtain the key word corresponding to at least one target address.

Based on the foregoing embodiment, in the embodiment of the present application, after the client node performs step 312b, it may further perform step 312 c:

step 312c, the client node updates the memory address information based on the at least one target address.

In the embodiment of the application, after the client node registers at least one target address as the RDMA memory, in order to ensure that the server node and the client node can quickly determine which memories are the RDMA memories, the memories corresponding to the registered at least one target address and the corresponding keywords can be updated into the memory address information, so that the working efficiency of the server node and the client node is effectively improved.

Based on the foregoing embodiments, in other embodiments of the present application, referring to fig. 6, before the client node performs step 308, it is further configured to perform steps 313 to 317, and/or steps 318 to 320:

step 313, the client node detects a first duration from the time of taking out the first RPC request from the queue head of the RPC request queue to the current time and a second duration from the time of sending the RDMA request to the server node to the current time according to a preset time interval.

In the embodiment of the present application, the preset time interval is a time period experience value obtained according to a large number of experiments or according to the needs or experiences of the user. The client node detects a first duration of the first RPC request taken out from the RPC request queue to the current time and detects a second duration of the RDMA request sent to the server node to the current time at intervals of a preset time interval. It should be noted that the preset time interval, i.e. the time period, for detecting the first time duration of the first RPC request may be different from or the same as the preset time interval for detecting the RDMA request, that is, the time for detecting the first time duration of the first RPC request may be the same as or different from the time for detecting the second time duration of the RDMA request.

Step 314, if the first time is greater than the first threshold, the client node determines that the first response result to the first RPC request is a response time-out.

In the embodiment of the present application, the first threshold may be a time-length empirical value obtained according to a number of experiments or user settings according to their own needs or experiences.

Step 315, the client node calls a callback function in the first RPC request to return the first response result to the first target application.

In an embodiment of the present application, the callback function in the first RPC request is specified by the first target application on the client node side. And calling a callback function in the first RPC request by the client node to return a first response result of the first RPC request response timeout to the first target application so that the first target application can perform subsequent processing based on the first response result.

Step 316, the client node deletes the first RPC request from the RPC request queue.

In the embodiment of the application, because the response to the first RPC request is overtime, the first RPC request is deleted from the RPC request queue, so that the client node can quickly execute the operation to other RPC requests in the RPC request queue, and the processing efficiency of the client node is effectively improved.

Step 317, the client node decrements the reference count in the first RPC request by 1.

Step 318, if the second duration is greater than the second threshold, the client node determines that the second response result for the RDMA request is a response timeout.

In the embodiment of the present application, the second threshold may be a time-length empirical value obtained through a large number of experiments or set by a user according to the needs or experiences of the user.

Step 319, the client node calls a callback function in the first RPC request to return the second response result to the first target application.

Step 320, the client node identifies the RDMA request timeout.

The client node may execute steps 313 to 317 and/or steps 318 to 320 simultaneously with the server node executing steps 306 to 307, or may execute the steps before step 306 or step 307, or may execute the steps after step 306, and the specific execution condition may be determined according to the actual situation.

Based on the foregoing embodiments, in other embodiments of the present application, referring to fig. 7, after the client node performs step 304, it is further configured to perform step 321:

step 321, the client node writes the RDMA request into a first RDMA request queue.

In an embodiment of the application, a client node writes an RDMA request to the tail of a first RDMA request queue.

Based on the foregoing embodiments, in other embodiments of the present application, referring to fig. 8, after the client node performs step 308, it is further configured to perform steps 322-325:

step 322, the client node counts a third duration from the time the RDMA request is sent to the server node to the time the RDMA reply sent by the server node is received.

Step 323, if the third duration is less than or equal to the third threshold, the client node calls a callback function in the first RPC request to return an RDMA response to the first target application.

In the embodiment of the present application, the third threshold may be a time-length empirical value obtained through a large number of experiments or set by a user according to the needs or experiences of the user.

Step 324, if the third duration is greater than the third threshold, the client node decrements the reference count in the first RPC request by 1.

Step 325, the client node deletes the RDMA request from the first RDMA request queue.

Based on the foregoing embodiments, in other embodiments of the present application, referring to fig. 9, after the service end node performs step 305, the service end node is further configured to perform step 326:

step 326, the server node writes the RDMA request to the tail of the second RDMA request queue in the server node.

Step 326 may be executed simultaneously with step 321, or may be executed before or after step 321, and the specific execution order may be determined according to the actual situation.

Based on the foregoing embodiments, in other embodiments of the present application, step 306 may be implemented by steps 306a to 306 c:

step 306a, if the second RDMA request queue is not empty, the server end node obtains the RDMA request from the head of the second RDMA request queue.

Step 306b, the server-side node analyzes the RDMA request to obtain an RPC request header and at least one piece of accessible memory information.

Wherein the at least one piece of accessibility information is RDMA memory in the client node.

Step 306c, the server-side node generates an RDMA response based on the RPC request header and the at least one accessible memory information.

Based on the foregoing embodiments, in other embodiments of the present application, step 306c may be implemented by steps a 11-a 16, or steps a 11-a 12 and steps a 17-a 20. If the data transmission direction in the RPC request header is that the client node transmits to the server node, the server node selects to execute the steps a 11-a 16, and if the data transmission direction in the RPC request header is that the server node transmits to the client node, the server node selects to execute the steps a 11-a 12 and the steps a 17-a 20:

step a11, the service end node allocates the memory of the service end node based on at least one accessible memory information, and generates memory allocation information.

And a12, the server-side node generates a second RPC request based on the RPC request header and the memory allocation information.

Step a13, if the data transmission direction in the RPC request header is that the client node transmits to the server node, the server node reads the first target data from the memory corresponding to the at least one accessible memory information in the client node.

Step a14, the server-side node stores the first target data in the memory corresponding to the memory allocation information.

Step a15, the server-side node determines a second target application, and controls the second target application to perform response processing on the second RPC request and the memory allocation information to obtain a first RPC response.

Wherein the second target application has an association relationship with the RPC request header.

In the embodiment of the application, the second target application is determined by the server end node based on the request type in the header of the RPC request.

Step a16, the server end node generates an RDMA reply based on the first RPC reply.

Step a17, if the data transmission direction in the RPC request header is that the server node transmits to the client node, the server node determines a third target application, and controls the third target application to perform response processing on the second RPC request to obtain third target data.

Wherein the third target application has an association relationship with the RPC request header.

In the embodiment of the application, the third target application is determined by the server-side node based on the request type in the header of the RPC request.

Step a18, the server node stores the third target data in the memory corresponding to the memory allocation information, and generates a second RPC response.

Step a19, the server node writes the third target data stored in the memory corresponding to the memory allocation information into the memory corresponding to the at least one accessible memory information of the client node.

Step a20, the server end node generates an RDMA reply based on the second RPC reply.

Based on the foregoing embodiments, the present application provides a system architecture diagram of an information processing method, which is shown in fig. 10 and includes a client node 1, a client node 2, a server node 1, and a server node 2. Wherein, the client node 1 is connected with the server terminal node 1 and the server terminal node 2 in a communication way, and the client node 2 is also connected with the server terminal node 1 and the server terminal node 2 in a communication way.

And each communication channel of each client node is provided with an RPC request queue, an RDMA connection resource and memory address information for storing a registered RDMA memory. The RPC request queue is used for storing requests issued by upper-layer application; the RDMA request queue is used for storing RDMA requests which are sent out in an RDMA communication mode and are not returned; the RDMA connection resource is related resources defined by the RDMA programming library and used for RDMA data transmission, including a request Queue (QP), a Completion Queue (CQ), a Protection Domain (PD), and the like; the memory address information may be denoted as RDMA _ rkey _ map, and is used to store a key word of the registered RDMA memory, and if a remote node, that is, the aforementioned server node, is required to directly access data in the local memory by RDMA, the memory needs to be registered as an RDMA-accessible memory, generally referred to as RDMA memory, the authority is set to be remotely accessible, and the key word is provided to the remote node as a token for access.

For the side of the server end node, an RDMA request queue and an RDMA connection resource are also provided, wherein the RDMA request queue on the side of the server end node is used for storing the received client RDMA requests which are not replied yet, and the RDMA connection resource is related resources defined by an RDMA programming library and used for RDMA data transmission, and comprises a QP, a CP, a PD and the like.

The multiple communication channels at the side of the server-side node share a memory pool at the side of the server-side node to perform data memory allocation, and a memory in the memory pool is an RDMA memory.

Based on the system architecture diagram shown in fig. 10, the target communication channel between the client node and the server node is created in advance by the following method: after the upper layer application on the client node side appoints the IP address and the port number of the server node, the client node calls the RPC to create a communication channel interface to create a communication channel, and RDMA connection between the client node and the server node is established. Further, the RPC also calls the RDMA programming library to create a corresponding RDMA connection resource between the client node and the server node.

When the upper layer application on the client node side, namely the first target application, initiates a first RPC request, a target communication channel from the client node to the server node, for example, phxrpc channel, a request header, request data, for example, RPC _ iobuf, a callback function and a callback parameter are specified in the first RPC request. In some special cases, there may be request types that may set rpc _ iobuf to null if no data needs to be transferred. It should be noted that, after the first target application sends the first RPC request, the memory pointed by RPC _ iobuf cannot be operated until the first RPC request completes the callback. In this way, when the client node receives a first RPC request submitted by a first target application, firstly, whether an RPC request queue corresponding to the phxrpc channel is full is judged, and if so, a callback function is called to return an error; and if not, adding the first RPC request to the end of the RPC request queue, increasing the reference count of RPC _ iobuf in the RPC request, and then executing the operation of sending the first RPC request on the phxrpc channel.

The structure of the first RPC request can be as shown in fig. 11, and includes a channel pointer, a request header, request data RPC _ iobuf, a callback function, and a callback parameter pointer. Wherein:

a channel pointer in the first RPC request to specify a target communication channel of the client node for sending the message to the server node.

The request header, the metadata of the request, as shown with reference to fig. 12, includes at least a request type, a data transmission direction, a message version number, and an application custom block. Wherein: two data transmission directions are provided, one is that data are transmitted from the client to the server and can be recorded as C2S, and the other is that data are transmitted from the server to the client and can be recorded as S2C; the application self-defining block is a section of area with fixed length, the area is set by a first target application at the side of a client node, different request types can be set into different structures, and the structure inside the RPC is not sensed; the message version number is used for compatibility among different versions; RPC _ iobuf, that is, a structure for describing request data, realizes that no data copy is performed between the RPC and the first target application between the client nodes, that is, no memory copy is performed whether the upper layer application on the client side sends the first RPC request through the client side, or the server node hands the first RPC request to the upper layer application on the server side for processing, and only the reference of data to the corresponding memory region is transferred through RPC _ iobuf.

rpc _ iobuf, namely, for storing data to be transferred, as shown in fig. 13, at least includes: reference count, memory count and memory 1, memory 2, … …, memory n, n being a positive integer greater than or equal to 1. For example, the representation structure of the memory in rpc _ iobuf can be shown in fig. 14. The data transmitted by rpc _ iobuf may be a continuous memory or may be composed of multiple discrete memories, in rpc _ iobuf, a memory may be represented by a symbol iovec, multiple iovecs may be set to set multiple memories, the number of iovecs cannot exceed a set threshold, for example, the threshold may be 16, and iovec identifies a memory by a memory address and a data length. And the reference count is used for controlling the life cycle of the memory, namely, the reference count is set to be 1 when rpc _ iobuf is initialized, 1 is added when rpc _ iobuf is referenced, 1 is subtracted when rpc _ iobuf is released, and rpc _ iobuf and the memory pointed by each iovec in the memory are released when the reference count is reduced to be 0.

And the callback function is the callback function after the first RPC request is executed, and is specified by the first target application.

And the callback parameter pointer is a parameter transmitted when the callback function is called after the first RPC request is executed, and is specified by the first target application.

The operation procedure of sending the first RPC request may specifically be that when the first RPC request reaches the head of the RPC request queue, the first RPC request is sent to the server node in an RDMA manner. The specific implementation steps of the client node for sending the first RPC request may be as follows:

step b11, the client node judges whether the phxrpc channel state is normal, and exits if the connection of the phxrpc channel state is abnormal;

step b12, if the phxrpc channel state connection is normal, the client node judges whether the RDMA request queue is full, if the RDMA request queue is full, the client node quits;

the RDMA request queue stores the RDMA requests which are sent out and are not completed;

step b13, the client node judges whether the RPC request queue is empty, and exits if the RPC request queue is empty;

step b14, if the RPC request queue is not empty, the client node takes out the first RPC request located at the head position of the RPC request queue, creates a corresponding RDMA request for the first RPC request, fills the request head of the first RPC request into the RDMA request, sets RDMA _ iovec according to iovec in RPC _ iobuf of the first RPC request, searches corresponding rkey, namely the keyword, in the RDMA _ rkey _ map, namely the memory address information according to the memory address in iovec, calls an RDMA library function to register the RDMA memory to obtain the rkey if the corresponding rkey is not found, and updates the rkey into the RDMA _ rkey _ map.

The structure of the RDMA request may be as shown in fig. 15, and includes: RDMA identification information, RPC request pointer, and RDMA transfer request. The RDMA identification information can be RDMA request identity identification information, is a unique identification of the RDMA request, and is generated when the RDMA request is created by the client node; the RPC request pointer is a channel pointer of a first RPC request sent by a first target application; the RDMA transfer request is the part actually required to be transferred to the server node by RDMA, and referring to FIG. 16, the RDMA identification information of the RDMA request, the request header of the first RPC request, the RDMA _ iovec number describing the request data area, and at least one RDMA _ iovec are included; the RDMA _ iovec is encapsulation of iovecs in phx _ iobuf of the first RPC request, and is in one-to-one correspondence with iovecs in phx _ iobuf of the first RPC request, wherein a memory address and data length are values in iovec, and RDMA _ rkey is a keyword obtained after the RDMA programming library function is called by the segment of memory and is registered as an RDMA memory, and the RDMA _ rkey is provided to the server node by the client node and is used as an access token when the subsequent server node accesses the segment of memory on the client node. The structure of each rdma _ iovec can be seen with reference to fig. 17, including memory address, data length, and key.

Step b15, the client node SENDs the RDMA request to the client node by adopting RDMA bilateral primitive RDMA _ SEND, and adds the RDMA request to the tail of the RDMA request queue, and exits.

Step b16, the server node receives the RDMA request using RDMA bilateral primitive RDMA _ RECV, puts the RDMA request at the end of the RDMA request queue at the server node side, and then performs the operation of processing the RDMA request. Wherein: when the RDMA request is processed, when the RDMA request is positioned at the head of an RDMA request queue at the side of a server-side node, the RDMA request is taken out, the RDMA request is analyzed to obtain a first RPC request, a required memory is distributed according to the first RPC request, then the first RPC request is delivered to an upper-layer application at the side of the server-side node, namely the second target application and the third target application, processed to obtain a response, and the response is sent to the client-side node. The specific implementation steps of the operation of the server end node to process the RDMA request may be as follows:

step c11, the server end node judges whether the RDMA request queue is empty, if so, the server end node exits;

step c12, if the RDMA request queue is not empty, the server node takes out the RDMA request at the head of the RDMA request queue, performs memory allocation on the memory pool at the side of the server node according to each RDMA _ iovec in the RDMA request, creates a phx _ iobuf structure to record the information of the memory allocation, and exits if the memory allocation fails;

step c13, if the memory allocation is successful, the server-side node creates an RPC request, namely the second RPC request, and fills the RPC request header in the RDMA request and the phx _ iobuf created in step c 12;

step C14, the service end node judges the data transmission direction in the RPC request header, if the data transmission direction is C2S, the steps C15-C17 and C21 are executed; if S2C, executing steps c 18-c 21;

step c15, the service end node READs the data in the client memory indicated by each RDMA _ iovec from the client node by using the RDma single-edge primitive RDMA _ READ, and stores the obtained data in the memory of phx _ iobuf record allocated by the service end node;

step c16, the server node submits the second RPC request and the phxrpc _ iobfuf created in step c12 to an upper layer application at the server node side, namely the second target application, for processing, namely, according to the request type of the second RPC request, a corresponding processing program registered by the upper layer application is called to process the second RPC request;

step c17, after the upper layer application at the side of the server-side finishes the processing of the second RPC request, the server-side receives the RPC response returned by the upper layer application at the side of the server-side, fills the RPC response into the RDMA response, SENDs the RDMA bilateral primitive RDMA _ SEND to the client-side, and exits;

the structure of the RPC response may be as shown in fig. 18, and includes a return value of a response phxrpc request returned by an upper layer application at the server end node side, a message version number, and an application custom block; the message version number and the application custom block are consistent with the above description and are set by the upper layer application at the side of the server node. Correspondingly, the structure of the obtained RDMA response may be as shown in fig. 19, and includes RDMA identification information and RPC response, where the RDMA identification information is identification information of the corresponding RDMA request.

Step c18, the client node sends the second RPC request to the upper layer application at the side of the server node, namely the third target application, for processing, namely, the corresponding processing program registered by the application is called for processing according to the request type in the second RPC request;

step c19, after the third target application finishes processing the second RPC application, the service end node sends the data in the memory of the service end node recorded by phxrpc _ iobuf to the memory of the client end node recorded by RDMA _ iovec through RDMA single-edge operation RDMA _ WRITE;

and step c20, the server node fills the RPC response returned by the third target application into the RDMA response, SENDs the RDMA bilateral primitive RDMA _ SEND to the client node, and exits.

Step c21, the client receives RDMA responses through a bilateral primitive RDMA _ RECV, then obtains the RDMA requests sent before through RDMA _ req _ id, judges whether the RDMA requests are marked as timeout or not, if the RDMA requests are not marked as timeout, calls a callback function of a first RPC request, returns the RPC responses obtained by analyzing the RDMA responses to an upper layer application at the client side, reduces the phx _ iobuf reference count in the first RPC request by 1, and removes the RDMA requests from an RDMA request queue at the client side; if the RDMA request has been marked for timeout, then no callback is made, the phx _ iobuf reference count for the second RPC request contained in the RDMA request is decremented by 1, and the RDMA request is removed from the client node side RDMA request queue.

The client node records the time when the application submits the first RPC request, periodically detects whether the execution time of any request in the RPC request queue and the RDMA request queue exceeds a threshold value, and the request exceeding the threshold value is judged as an overtime request. Wherein:

if the request in the RPC request queue is overtime, an RPC response structure is created to set an overtime error code, a callback function in a first RPC request is called to return a result to an upper-layer application at the client node side, then the first RPC request is removed from the RPC request queue, and the reference count of phx _ iobuf in the first RPC request is reduced by 1;

if the RDMA request in the RDMA request queue is overtime, a phxrpc response structure is created to set a timeout error code, a callback function in the RDMA request is called to return the result to an upper application on the client node side, the RDMA request is marked to be overtime, the RDMA request is still kept in the RDMA request queue, and the reference count of phx _ iobuf in the RDMA request is not changed. The reason is that after the sent RDMA request is judged to be timeout, the server node may read or write to the memory area pointed by phx _ iobuf in the RDMA request, and the early release of the memory may cause data errors. It should be noted that, when receiving the RDMA response of the server node and processing the RDMA response, the client node also determines whether the RDMA request is marked as timeout and performs corresponding processing.

In the foregoing description, an application-defined block with a fixed length is set in the request header of the first RPC request, so that the upper layer application on the client node side can define the data structure in this block according to the message type, and when in use, the upper layer application only needs to perform a type conversion using the pointer of the application-defined block. When the upper layer application of the client node side sends the first RPC request through the client node, the upper layer application of the client node side directly assigns the structure obtained through type conversion based on the application custom block, after the server node receives the DRMA request comprising the first RPC request, the server node calls a corresponding processing program registered by the upper layer application according to the request type in the DRMA request to perform request processing, and the upper layer application of the server node side performs type conversion based on the application custom block to obtain a required structure. Compared with the traditional RPC coding and decoding method, the method has higher efficiency, and the length of the head of the first RPC request is fixed, so that the method is suitable for transmission by using RDMA bilateral primitives.

By way of example, assume that the data structure expected by the upper layer application at the server end node side is as follows:

struct phxrpc_header{

enum req_type type；

enum trans flag flag；

uint32_t version；

char appdata[MAXLEN]；

}；

however, the data structure in the first RPC request sent by the client node is as follows:

struct app_msg_type1{

uint32 t a；

uint32_t b；

uint64 t c；

}；

in this way, the upper layer application in the server-side node calls the following conversion method in the custom area block to convert the data structure in the first RPC request, and the desired data structure is obtained for subsequent analysis.

struct phxrpc_header header；

struct app_msg_type1*msg；

msg＝(struct app_msg_type1)&header.appdata。

It should be noted that, communication can be performed between the client node and the upper layer application on the client node side, or between the server node and the upper layer application on the server node side through the set RPC interface, that is, the upper layer application submits an RPC request to the client node or the server node through the RPC interface, where the RPC request is executed asynchronously, and an execution result for the RPC request is returned through a callback function. When the RDMA memory is registered, the RDMA memory can be registered through a preset RDMA memory registration interface. And after receiving the second RPC request, the server node calls the registered upper application processing program according to the request type in the second RPC request to process, and when the structure of the user-defined block of the head of the RPC request is analyzed according to the request type, the method can be realized through a preset message processing program registration interface at the server node side.

It should be noted that, for the explanation of the same steps or concepts in the present embodiment as in the other embodiments, reference may be made to the description in the other embodiments, and details are not described here.

In the information processing method provided by the embodiment of the application, the client node determines the server node and the target communication channel through a first RPC request generated by a first target application, generates an RDMA request based on the first RPC request, and then sends the RDMA request to the server node through the target communication channel by adopting RDMA bilateral primitives, so that the server node responds to the RDMA request to generate a corresponding RDMA response, and sends the generated RDMA response to the client node through the target communication channel by adopting RDMA bilateral primitives, thereby solving the problem that the resource consumption in the communication process is large because the memory needs to be copied and analyzed in the communication process of two communication parties in the existing RDMA realized distributed storage system, providing a communication information processing method under the distributed system, realizing that the memory does not need to be copied and analyzed in the communication process, and reducing the resource consumption in the communication process of the distributed storage system, the performance of the RDMA realized distributed storage system is effectively improved. And the RDMA bilateral primitive mode is adopted for data transmission, so that high-performance communication with low delay, low CPU (central processing unit) overhead and low memory bandwidth overhead is realized.

Based on the foregoing embodiments, the present application provides a client device 4, where the client device 4 can be applied to the embodiments corresponding to fig. 1 to 9 and other methods, and referring to fig. 20, the client device 4 includes: a first determining unit 41, a first generating unit 42, a first transmitting unit 43, and a first receiving unit 44; wherein:

a first determining unit 41, configured to detect a first Remote Procedure Call (RPC) request generated by a first target application, and determine, based on the first target application, a server node and a target communication channel;

a first generating unit 42, configured to generate a remote direct memory access RDMA request based on the first RPC request;

a first sending unit 43, configured to send an RDMA request to the server node by using an RDMA bilateral primitive through the target communication channel;

a first receiving unit 44, configured to receive an RDMA response sent by the server-end node through the target communication channel by using an RDMA bilateral primitive; wherein the RDMA reply is generated by the server end node in response to the RDMA request.

In other embodiments of the present application, the first RPC request includes: the channel pointer, the RPC request header, the request data, the callback function and the callback parameter pointer are used for pointing to the server-side node; wherein the first RPC request header includes at least: the RPC request processing method comprises the following steps of request type, data transmission direction, message version number and an application custom block, wherein the application custom block is used for realizing type conversion of a first RPC request, and request data at least comprise: at least one of remotely accessible memory information and reference count.

In other embodiments of the present application, the first determining unit includes: a first determining module and a creating module; wherein:

the first determining module is used for detecting a first RPC request generated by a first target application and determining an Internet Protocol (IP) address and a port number indicated by the first target application;

and the creating module is used for creating the connection between the channel creating interface and the service end node based on the IP address and the port number to obtain the target communication channel.

In other embodiments of the present application, before the first generating unit, the apparatus further includes: a first writing unit and a processing unit; wherein:

the first write-in unit is used for detecting that an RPC request queue corresponding to the target communication channel is not full, and writing the first RPC request into the tail of the RPC request queue;

and the processing unit is used for adding 1 to the reference count included in the first RPC request.

In other embodiments of the present application, the first generating module comprises: the device comprises a taking-out module, a second determining module and a first generating module; wherein:

the taking-out module is used for detecting that the state of the target communication channel is a target state, detecting that the first RDMA request queue is not full and the RPC request queue is not empty, and taking out the first RPC request from the queue head of the RPC request queue; wherein the first RDMA request queue is used for recording RDMA requests which are sent out but not returned with responses;

the second determining module is used for determining the RDMA identification information of the RDMA request to be generated;

the first generation module is used for acquiring an RPC request header, at least one piece of accessible memory information and RDMA identification information from the first RPC request and generating the RDMA request.

In other embodiments of the present application, before the first generating module, the first generating unit further includes: the system comprises a first acquisition module and a registration module; wherein:

the first acquisition module is used for acquiring at least one memory address included in at least one piece of accessible memory information;

the registration module is configured to detect that at least one memory address includes at least one target address that is not registered as an RDMA memory, and register a memory corresponding to the at least one target address as the RDMA memory.

In other embodiments of the present application, the registration module is specifically configured to implement the following steps:

acquiring memory address information which is registered as an RDMA memory;

at least one target address in at least one memory address is not included in the memory address information, and the memory corresponding to at least one target address is registered as an RDMA memory.

Correspondingly, after the module is registered, the first generating unit further includes: an update module; wherein:

and the updating module is used for updating the memory address information based on at least one target address.

In other embodiments of the present application, before the first sending unit, the apparatus further includes: the device comprises a detection unit, a second determination unit, a calling unit and a deleting unit; wherein:

the detection unit is used for detecting a first time length from the moment of taking out the first RPC request from the head of the RPC request queue to the current moment and a second time length from the moment of sending the RDMA request to the server node to the current moment according to a preset time interval;

a second determining unit, configured to determine that a first response result to the first RPC request is response timeout if the first duration is greater than the first threshold;

the call unit is used for calling a callback function in the first RPC request and returning a first response result to the first target application;

the deleting unit is used for deleting the first RPC request from the RPC request queue;

a processing unit further configured to decrement the reference count in the first RPC request by 1;

and/or the second determining unit is configured to determine that a second response result for the RDMA request is response timeout if the second duration is greater than a second threshold;

the call unit is used for calling a callback function in the first RPC request and returning a second response result to the first target application;

a processing unit to also identify an RDMA request timeout.

In other embodiments of the present application, the first writing unit is further configured to perform the following steps:

the RDMA request is written to a first RDMA request queue.

In other embodiments of the present application, after the first receiving unit, the apparatus further includes: a counting unit; wherein:

the statistic unit is used for counting a third time length from the moment of sending the RDMA request to the server node to the moment of receiving the RDMA response sent by the server node;

the call unit is further used for calling a callback function in the first RPC request to return an RDMA response to the first target application if the third duration is less than or equal to a third threshold;

the processing unit is further used for subtracting 1 from the reference count in the first RPC request if the third duration is greater than a third threshold;

a delete unit to delete the RDMA request from the first RDMA request queue.

It should be noted that, in the embodiment, the interaction process between the steps implemented by each unit and the module may refer to the interaction processes in the embodiments corresponding to fig. 1 to 9 and the information processing method provided in the foregoing embodiments, and details are not described here.

In the client node provided by the embodiment of the application, the client node determines the server node and the target communication channel through a first RPC request generated by a first target application, generates an RDMA request based on the first RPC request, and then sends the RDMA request to the server node through the target communication channel by using RDMA bilateral primitives, so that the server node responds to the RDMA request to generate a corresponding RDMA response, and sends the generated RDMA response to the client node through the target communication channel, thereby solving the problem that the memory needs to be copied and analyzed in the communication process of two communication parties in the existing RDMA-implemented distributed storage system, which causes large resource consumption in the communication process, providing a communication information processing method under the distributed system, realizing that the memory does not need to be copied and analyzed in the communication process, and reducing the resource consumption in the communication process of the distributed storage system, the performance of the RDMA realized distributed storage system is effectively improved.

Based on the foregoing embodiments, the present application provides a server apparatus 5, where the server apparatus 5 can be applied to the embodiments corresponding to fig. 1 to 9 and other methods, and referring to fig. 21, the server apparatus 5 includes: a second receiving unit 51, a second generating unit 52, and a second transmitting unit 53; wherein:

a second receiving unit 51, configured to receive, through a target communication channel, an RDMA request sent by a client node by using an RDMA bilateral primitive;

a second generating unit 52 configured to generate an RDMA response based on the RDMA request;

a second sending unit 53, configured to send the RDMA response to the client node using the RDMA bilateral primitive through the target communication channel.

In other embodiments of the present application, after the second receiving unit, the server apparatus further includes: a second writing unit; wherein:

a second write unit to write the RDMA request to a tail of a second RDMA request queue in the server node.

In other embodiments of the present application, the second generating unit includes: the system comprises a second acquisition module, an analysis module and a second generation module; wherein:

the second acquisition module is used for acquiring the RDMA request from the head of the second RDMA request queue if the second RDMA request queue is not empty;

the analysis module is used for analyzing the RDMA request to obtain an RPC request head and at least one piece of accessible memory information; wherein the at least one piece of accessible information is RDMA memory in the client node;

a second generating module to generate an RDMA response based on the RPC request header and the at least one accessible memory information.

In other embodiments of the present application, the second generating module may be specifically configured to implement the following steps:

allocating the memory of the service end node based on at least one piece of accessible memory information to generate memory allocation information;

generating a second RPC request based on the RPC request header and the memory allocation information;

if the data transmission direction in the RPC request header is that the client node transmits the data to the server node, reading first target data from a memory corresponding to at least one piece of accessible memory information in the client node;

storing the first target data to a memory corresponding to the memory allocation information;

determining a second target application, and controlling the second target application to respond to the second RPC request and the memory allocation information to obtain a first RPC response; the second target application and the RPC request header have an incidence relation;

based on the first RPC response, an RDMA response is generated.

In other embodiments of the present application, the second generating module may be further specifically configured to implement the following steps:

if the data transmission direction in the RPC request header is that the server node transmits the data to the client node, determining a third target application, and controlling the third target application to perform response processing on the second RPC request to obtain third target data; the third target application and the RPC request header have an incidence relation;

storing the third target data into a memory corresponding to the memory allocation information, and generating a second RPC response;

writing third target data stored in the memory corresponding to the memory allocation information into the memory corresponding to the at least one accessible memory information of the client node;

generating an RDMA response based on the second RPC response.

In the server device provided by the embodiment of the application, the client node determines the server node and the target communication channel through the first RPC request generated by the first target application, generates the RDMA request based on the first RPC request, and then sends the RDMA request to the server node through the target communication channel by using the RDMA bilateral primitive, so that the server node responds to the RDMA request to generate a corresponding RDMA response, and sends the generated RDMA response to the client node, thereby solving the problem that the memory needs to be copied and analyzed in the communication process of two communication parties in the distributed storage system realized by the existing RDMA, which causes large resource consumption in the communication process, providing a communication information processing method under the distributed system, realizing that the memory does not need to be copied and analyzed in the communication process, and reducing the resource consumption in the communication process in the distributed storage system, the performance of the RDMA realized distributed storage system is effectively improved.

Based on the foregoing embodiments, the present application provides an information processing system 6, where the information processing system 6 may be applied to the embodiments corresponding to fig. 1 to 9 and other methods, and as shown in fig. 22, the information processing system 6 includes: client node 61 and service node 62; wherein:

the client node 61 is configured to implement the information processing method provided in the embodiments corresponding to fig. 1 and fig. 3 to 9, and details are not repeated here;

the service end node 62 is configured to implement the information processing method provided in the embodiments corresponding to fig. 2 to 9, and details are not described here again.

Based on the foregoing embodiments, embodiments of the present application provide a computer-readable storage medium, referred to as a storage medium for short, where one or more information processing programs are stored in the computer-readable storage medium, and the one or more information processing programs can be executed by one or more processors to implement the information processing method provided in the embodiments corresponding to fig. 1 and fig. 3 to 9, or fig. 2 to 9, and details are not repeated here.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, an air conditioner, or a network communication link device) to execute the methods described in the embodiments of the present application.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims

1. An information processing method applied to a client node, the method comprising:

2. The method of claim 1, wherein the first RPC request comprises: the channel pointer, the RPC request header, the request data, the callback function and the callback parameter pointer are used for pointing to the server-side node; wherein the first RPC request header includes at least: request type, data transmission direction, message version number and application custom block, the application custom block is used for realizing carrying out type conversion to the first RPC request, the request data include at least: at least one of remotely accessible memory information and reference count.

3. The method according to any one of claims 1 to 2, wherein the detecting of the first Remote Procedure Call (RPC) request generated by the first target application and the determining of the server node and the target communication channel based on the first target application comprise:

detecting the first RPC request generated by a first target application, and determining an Internet Protocol (IP) address and a port number indicated by the first target application;

and establishing connection between a channel establishing interface and the service end node based on the IP address and the port number to obtain the target communication channel.

4. The method of claim 3, wherein prior to generating a remote memory direct access (RDMA) request based on the first RPC request, the method further comprises:

detecting that an RPC request queue corresponding to the target communication channel is not full, and writing the first RPC request into the tail of the RPC request queue;

and adding 1 to the reference count included in the first RPC request.

5. The method of claim 4, wherein generating a Remote Direct Memory Access (RDMA) request based on the first RPC request comprises:

detecting that the state of the target communication channel is a target state, detecting that a first RDMA request queue is not full and the RPC request queue is not empty, and taking out the first RPC request from the head of the RPC request queue; wherein the first RDMA request queue is to record RDMA requests that have been issued but not returned acknowledgements;

determining RDMA identification information of the RDMA request to be generated;

and acquiring the RPC request header, at least one piece of accessible memory information and the RDMA identification information from the first RPC request, and generating the RDMA request.

6. The method of claim 5, wherein the RPC request header, at least one accessible memory information, and the RDMA identification information are obtained from the first RPC request, and wherein prior to generating the RDMA request, the method further comprises:

acquiring at least one memory address included in the at least one accessible memory information;

detecting that at least one target address included in the at least one memory address is not registered as an RDMA memory, and registering a memory corresponding to the at least one target address as the RDMA memory.

7. The method of claim 6, wherein the detecting that at least one of the at least one memory address is not registered as RDMA memory comprises detecting that at least one target address is not registered in RDMA memory, and registering memory corresponding to the at least one target address as the RDMA memory comprises:

acquiring memory address information which is registered as an RDMA memory;

the memory address information does not include the at least one target address in the at least one memory address, and a memory corresponding to the at least one target address is registered as the RDMA memory;

correspondingly, the at least one target address in the at least one memory address is not included in the memory address information, and after the memory corresponding to the at least one target address is registered as the RDMA memory, the method further includes:

and updating the memory address information based on the at least one target address.

8. The method of claim 5, wherein prior to receiving the RDMA reply sent by the server end node using the RDMA bilateral primitive over the target communication channel, the method further comprises:

detecting a first time length from the moment of taking out the first RPC request from the head of the RPC request queue to the current moment and a second time length from the moment of sending the RDMA request to the server-side node to the current moment according to a preset time interval;

if the first duration is greater than a first threshold, determining that a first response result aiming at the first RPC request is response overtime;

calling the callback function in the first RPC request to return the first response result to the first target application;

deleting the first RPC request from the RPC request queue;

subtracting 1 from the reference count in the first RPC request;

and/or if the second duration is greater than a second threshold, determining that a second response result for the RDMA request is response timeout;

calling the callback function in the first RPC request to return the second response result to the first target application;

identifying the RDMA request timeout.

9. The method of any of claims 4 to 8, wherein after sending the RDMA request to the server node over the target communication channel using RDMA bilateral primitives, the method further comprises:

writing the RDMA request into the first RDMA request queue.

10. The method of claim 2 wherein after receiving the RDMA reply sent by the server end node using RDMA bilateral primitives over the target communication channel, the method further comprises:

counting a third time length from the time of sending the RDMA request to the server end node to the time of receiving the RDMA response sent by the server end node;

if the third duration is less than or equal to a third threshold, calling the callback function in the first RPC request to return the RDMA response to the first target application;

if the third duration is greater than the third threshold, subtracting 1 from the reference count in the first RPC request;

deleting the RDMA request from the first RDMA request queue.

11. An information processing method, applied to a service end node, the method comprising:

generating an RDMA response based on the RDMA request;

12. The method of claim 11, wherein after receiving the RDMA request sent by the client node over the target communication channel, the method further comprises:

writing the RDMA request to a tail of a second RDMA request queue in the serving end node.

13. The method of claim 12, wherein generating the RDMA reply based on the RDMA request comprises:

if the second RDMA request queue is not empty, acquiring the RDMA request from the head of the second RDMA request queue;

analyzing the RDMA request to obtain an RPC request head and at least one piece of accessible memory information; wherein the at least one piece of accessible information is RDMA memory in the client node;

generating the RDMA response based on the RPC request header and the at least one accessible memory information.

14. The method of claim 13, wherein generating the RDMA reply based on the RPC request header and the at least one accessible memory information comprises:

allocating the memory of the service end node based on the at least one piece of accessible memory information to generate memory allocation information;

if the data transmission direction in the RPC request header is that the client node transmits the data to the server node, reading first target data from a memory corresponding to the at least one accessible memory information in the client node;

determining a second target application, and controlling the second target application to respond to the second RPC request and the memory allocation information to obtain a first RPC response; wherein the second target application has an association with the RPC request header;

generating the RDMA reply based on the first RPC reply.

15. The method of claim 14, further comprising:

if the data transmission direction in the RPC request header is that the server node transmits the data to the client node, determining a third target application, and controlling the third target application to perform response processing on the second RPC request to obtain third target data; wherein the third target application has an association with the RPC request header;

writing the third target data stored in the memory corresponding to the memory allocation information into the memory corresponding to the at least one accessible memory information of the client node;

generating the RDMA reply based on the second RPC reply.

16. A client device, the device comprising: the device comprises a first determining unit, a first generating unit, a first sending unit and a first receiving unit; wherein:

17. A server-side apparatus, the apparatus comprising: a second receiving unit, a second generating unit and a second transmitting unit; wherein:

18. An information processing system characterized by comprising: a client node and a server node; wherein:

the client node for implementing the steps of the information processing method according to any one of claims 1 to 10;

the server end node for implementing the steps of the information processing method according to any of claims 11 to 15.

19. A storage medium, characterized in that an information processing program is stored thereon, which when executed by a processor implements the steps of the information processing method according to any one of claims 1 to 10 or claims 11 to 15.