CN113242250B

CN113242250B - Multiplexing protocol and transmission method

Info

Publication number: CN113242250B
Application number: CN202110544820.7A
Authority: CN
Inventors: 冯杰; 李嘉伟; 周谊成
Original assignee: Suzhou Ruilisi Technology Co ltd
Current assignee: Suzhou Ruilisi Technology Co ltd
Priority date: 2021-05-19
Filing date: 2021-05-19
Publication date: 2023-10-24
Anticipated expiration: 2041-05-19
Also published as: CN113242250A

Abstract

The invention discloses a multiplexing protocol and a transmission method. The protocol adopts a UDP protocol as a communication protocol of a bottom layer, wherein an outer layer protocol of the protocol comprises a load protocol, a reliable transmission head and inner layer protocol data, the load protocol is one of TCP or UDP, if the load protocol is TCP, the reliable transmission head takes effect, and if the load protocol is UDP, the reliable transmission head does not take effect; the reliable transmission head comprises a data packet type, a sequence number, a time stamp and a window size; the inner layer protocol data includes a command, a connection ID, a data length, and data. The multiplexing protocol provided by the embodiment simplifies the content of the multiplexing protocol in the prior art, and distinguishes the TCP protocol and the UDP protocol through the data packet type, so that the protocol provided by the embodiment can support multiplexing of TCP and UDP connection at the same time.

Description

Multiplexing protocol and transmission method

Technical Field

The embodiment of the invention relates to the technical field of data transmission, in particular to a multiplexing protocol and a transmission method.

Background

Multiplexing is a technology that combines multiple low-speed channels into one high-speed channel, which can effectively increase the utilization rate of data links, so that one high-speed backbone link simultaneously serves multiple low-speed access links, that is, the network backbone can simultaneously carry a large amount of voice and data transmissions. Multiplexing is a technique for establishing a plurality of communication channels on one line in order to make full use of transmission media. The essence of multiplexing technology is that a plurality of user data in one area are collected by a transmitting multiplexer, and then the collected data are transmitted by one line, and then the data are separated by a receiving multiplexer and distributed to a plurality of users.

However, the header fields of the existing transmission control protocol (Transmission Control Protocol, TCP) and user datagram protocol (User Datagram Protocol, UDP) multiplexing protocols are complex, the performance loss is large compared to the case of no multiplexing, and the existing TCP/UDP multiplexing protocol can support only one of the TCP and UDP protocols.

Disclosure of Invention

The present invention provides a multiplexing protocol and a transmission method to provide a simplified multiplexing protocol capable of supporting both TCP and UDP protocols.

The technical scheme of the invention is as follows:

in a first aspect, an embodiment of the present invention provides a multiplexing protocol, where the protocol uses a UDP protocol as a communication protocol of a bottom layer, and an outer layer of the protocol includes a payload protocol, a reliable transport header, and inner layer protocol data, where the payload protocol is one of TCP and UDP, and if the payload protocol is TCP, the reliable transport header is in effect, and if the payload protocol is UDP, the reliable transport header is not in effect;

the reliable transmission head comprises a data packet type, a sequence number, a time stamp and a window size;

the inner layer protocol data includes a command, a connection ID, a data length, and data.

Optionally, the command includes a new connection control command, a transmission data control command, and a close connection control command.

Optionally, the connection ID is used to split data on one link to distinguish between different connections.

In a second aspect, an embodiment of the present invention further provides a multiplexing transmission method, where the method is based on the protocol in any of the foregoing implementations of the present invention, and the method is performed by a client, and includes:

step 1, establishing UDP monitoring at a server, and maintaining a mapping table from a source IP and a source port to a connection ID;

step 2, when a new data connection request is received, matching a source IP and a source port in the data connection request with the source IP and the source port stored in the mapping table, if the matching is successful, continuing to execute the step 3, otherwise, executing the step 5;

step 3, self-increasing the connection ID, and newly creating a mapping between the source IP and the source end in the data connection request and the self-increased connection ID in the mapping table;

step 4, sending a control command for newly-built connection to the server;

step 5, determining a corresponding connection ID based on the source IP and the source port in the data connection request and constructing a data command, wherein the data command comprises the connection ID, the data length and the data itself;

and step 6, if the data packet type in the data connection request is UDP, the data packet is directly sent to the server, and if the data packet type in the data connection request is TCP, the data packet is sent to the server through a reliable transmission head so as to ensure the sending of the data packet.

Optionally, when the connection is closed, the method further includes sending a close connection control command to the server, and deleting the mapping of the source IP and the source port to the connection ID in the data connection request in the mapping table.

Optionally, before step 6, the method further includes: and encrypting the data packet in the data connection request by adopting an encryption algorithm.

In a third aspect, an embodiment of the present invention further provides a multiplexing transmission method, where the method is based on the protocol in any one of the foregoing implementations of the present invention, and the method is executed by a server, and includes:

step 1, monitoring UDP port, and maintaining a mapping table from connection ID to connection;

step 2, receiving a data packet sent by a client, and decrypting the data packet;

step 3, analyzing the data packet type, if the data packet type is TCP, sending a confirmation character by using a reliable transmission protocol, continuing to analyze the control command field, and if the data packet type is UDP, directly analyzing the control command field;

and 4, executing the command obtained by analyzing the control command field.

The multiplexing protocol provided by the embodiment simplifies the content of the multiplexing protocol in the prior art, and distinguishes the TCP protocol and the UDP protocol through the data packet type, so that the protocol provided by the embodiment can support multiplexing of TCP and UDP connection at the same time.

Drawings

Fig. 1 is a general architecture diagram of a multiplexing protocol according to an embodiment of the present invention.

Detailed Description

The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.

Examples

Discussed in this embodiment is connection oriented multiplexing and demultiplexing, i.e., the host-to-host delivery service provided by the network layer extends to the process-to-process delivery service provided by the application running on the host.

In order to solve the problems existing in the multiplexing protocol in the prior art, the multiplexing protocol supporting both TCP and UDP provided in this embodiment, in order to maintain the low latency characteristic of UDP, the multiplexing protocol uses the UDP protocol as the communication protocol of the bottom layer, and meanwhile, a guarantee measure for reliable connection is added to the multiplexing protocol, so as to adapt to the requirements of TCP connection.

Specifically, the outer layer protocol of the multiplexing protocol is shown in the following table:

PROTOCOL	REL_HEADER	DATA
			load protocol (1 byte)	Reliable transmission head (9 bytes)	Inner layer protocol data (variable length)

The outer layer PROTOCOL is a UDP based reliable transport PROTOCOL portion including a payload PROTOCOL (PROTOCOL), a reliable transport HEADER (rel_header), and inner layer PROTOCOL DATA (DATA).

The load protocol is one of TCP or UDP, if the load protocol is TCP, the reliable transmission head is effective, and if the load protocol is UDP, the reliable transmission head is not effective. The reliable transport header is similar to the header in the TCP protocol, and includes packet types such as SYN, PSH, ACK, etc. for data reliability assurance.

The reliable transport header includes a packet TYPE (msg_type), SEQUENCE number (SEQUENCE), timestamp (TIMESTAMP), and window SIZE (wnd_size), as shown in the following table:

MSG_TYPE	SEQUENCE	TIMESTAMP	WND_SIZE
				bag type (1 byte)	Serial number (2 bytes)	Time stamp (4 bytes)	Window size (2 bytes)

The inner layer protocol DATA is common to TCP and UDP, including Command (CMD), connection ID (connjd), DATA Length (LEN), and DATA (DATA), as shown in the following table:

CMD	CONN_ID	LEN	DATA
				command (1 byte)	Connection ID (2 bytes)	Load length (2 bytes)	Load(s)

CMD is a control command for the connection, including three commands:

1. newly-built connection control command OPEN: newly creating a connection;

2. transmitting a DATA control command DATA: transmitting a data packet;

3. closing the connection control command CLOSE: the connection is closed.

Connjd is a 2-byte number that is self-increasing with connection creation. The conn_id remains consistent in the same connection, and the multiplexing protocol splits data on one link through the conn_id, distinguishing between different connections. LEN is the length of the actual carried DATA of the present packet, and DATA is the DATA itself.

Further, when data transmission is performed based on the multiplexing protocol, the transmitted data is encrypted through a configurable encryption algorithm, so that the security of the data transmission is ensured. For example, the data may be transmitted using the AES-256-GCM algorithm encrypted using the KEY agreed upon by the client and server as the encryption KEY and the random 16-byte data as the initial matrix, placed at the start of the data.

With continued reference to fig. 1, fig. 1 is a general architecture diagram of a multiplexing protocol according to an embodiment of the present invention, where after multiplexing and encrypting, multiple TCP and UDP connections are uniformly transmitted based on a stream, and after decrypting, the connections are separated by multiple paths to obtain different connections. Compared with the existing protocol, the protocol establishes a bottom multiplexing channel based on multiplexing of streams, improves the problems of uneven distribution, partial server pressure overload and the like which are easy to occur in the conventional server distribution process, and plays roles of balancing server resources and balancing link speed.

The multiplexing protocol provided by the embodiment simplifies the content of the multiplexing protocol in the prior art, and distinguishes the TCP protocol and the UDP protocol through the data packet type, so that the protocol provided by the embodiment can support multiplexing of TCP and UDP connection at the same time. Furthermore, the security of the connection is achieved by means of configurable encryption.

Further, based on the multiplexing protocol described above, the present invention provides three alternative embodiments:

as an alternative embodiment, the embodiment includes a remote proxy server supporting multiplexing, which can reduce the initialization and handshaking of a new connection and reduce server consumption.

As another alternative implementation mode, the implementation mode comprises a rapid reverse proxy which can be used for exposing intranet service to public network, thereby realizing intranet penetration, supporting tcp and udp traffic forwarding, realizing the safety of data transmission by encrypting the data in the transmission process, and ensuring that the data is not eavesdropped and tampered.

As a further alternative embodiment, the embodiment comprises a webapi server, light-weight high-performance HTTP request routing is realized, after multiplexing, the A station interface accesses the B interface to request data, a TCP long link is made, and all requests go through one channel, so that no matter how many client requests are connected too many problems are avoided.

On the basis of the multiplexing protocol, the transmission process of the client and the server is continuously introduced. First introducing the client:

step 1, client side communicates with Server, need to set up UDP monitor in the Server, and maintain a mapping table of source IP and source port to connection ID, namely maintain a mapping table Z of source IP: source port > to CONN_ID;

step 2, when a new data connection request is received, matching a source IP and a source port in the data connection request with a source IP and a source port stored in a mapping table Z, namely searching whether a < source IP: source port > in the data connection request exists in the mapping table Z, if the matching is successful, continuing to execute the step 3, otherwise, executing the step 5;

step 3, enabling the connection ID (CONN_ID) to be self-increased, wherein a fixed constant can be self-increased each time, such as self-increase 1, and a mapping between a source IP and a source end in the data connection request and the self-increased connection ID is newly established in the mapping table;

step 4, sending a control command OPEN for newly-built connection to the server, wherein the OPEN command comprises CONN_ID, LEN and DATA are empty;

step 5, for each incoming client DATA, determining a corresponding connection ID (i.e. conn_id) based on the source IP and source port in the DATA connection request and constructing a DATA (DATA) command comprising the connection ID (conn_id), the DATA Length (LEN) and the DATA itself (DATA);

and 6, if the data packet TYPE (MSG_TYPE) in the data connection request is UDP, directly sending the data packet to a server, and if the MSG_TYPE in the data connection request is TCP, sending the data packet to the server through a reliable transmission head so as to ensure the delivery of the data packet.

Further, the method further comprises: when the client connection is closed, sending a CLOSE connection control command CLOSE to the server, wherein the CLOSE command comprises a conn_id, LEN and DATA to be empty; and deleting the mapping records of the source IP and the source port to connection ID in the data connection request in the mapping table.

Before step 6, the method further comprises: and encrypting the data packet in the data connection request by adopting an encryption algorithm. The encryption algorithm may be, for example, an AES-256-GCM algorithm.

When the server is used as the execution end, the corresponding transmission process is as follows:

step 1, monitoring UDP port, maintaining a mapping table Y from connection ID (CONN_ID) to Connection (CONN), and starting to receive data;

step 3, analyzing the data packet TYPE MSG_TYPE, if the data packet TYPE is TCP, continuing to analyze the command field after sending the acknowledgement character ACK by using the reliable transmission protocol, and if the data packet TYPE is UDP, directly analyzing the command field CMD;

and 4, executing the command obtained by analyzing the command field.

Specifically, if the analyzed command is OPEN, locally creating CONN, and binding the CONN to the service processing logic of the server;

if the command obtained through analysis is DATA, searching CONN_ID in Y, and if the record exists, sending the DATA packet to the corresponding CONN; if no record exists, the packet is ignored;

if the analyzed command is CLOSE, CONN is found through CONN_ID, CONN is closed, and the mapping relation of CONN_ID and CONN is deleted.

Further, in this embodiment, the session defines a multiplexing connection for the stream, and carries multiple TCP/UDP connections on the same link between the client and the server.

In addition, the present embodiment can realize bidirectional transmission of data, allowing connection to be established in either direction. Once a connection is established, the parties can communicate equally. One party may simultaneously create a connection as an active initiator to begin communication. Communication may also be initiated as a passive recipient receiving a connection.

The technical scheme of the invention can effectively lighten the load of the server, reduce the times of connection creation and destruction on the server and save server resources. In practical application, the purpose of bypassing the firewall can be achieved by establishing a flow (the flow can also be called a virtual channel), a proxy server is arranged at the legal port of the firewall, a TCP/UDP channel of the protected server is maintained, the validity of the client is identified through the proxy server, and the flow between the server and a plurality of clients is established, so that multiplexing is realized.

Head of line blocking may occur in some cases due to multiplexing of streams into a single physical channel, which problem may be alleviated by increasing the overall de-multiplexing buffer to a larger value, which obviously would take more memory. Limiting per-stream memory usage, by limiting the stream buffer of the receiving party, will apply back pressure to the sending party and limit reading, and eventually prevent the source from sending too much data to occupy every bit of the buffer on the link.

A balance is maintained between concurrency and resources, and if there are many clients to serve and at the same time a powerful server is obtained, this value can be increased to increase concurrency and also reduced to serve only 1 or 2 clients.

Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims

1. The transmission method of the multiplexing protocol is characterized in that the protocol adopts a UDP protocol as a communication protocol of a bottom layer, an outer layer protocol of the protocol comprises a load protocol, a reliable transmission head and inner layer protocol data, the load protocol is one of TCP or UDP, the reliable transmission head takes effect if the load protocol is TCP, and the reliable transmission head takes effect if the load protocol is UDP; the reliable transmission head comprises a data packet type, a sequence number, a time stamp and a window size; the inner layer protocol data comprises a command, a connection ID, a data length and data; the data packet type is TCP protocol or UDP protocol;

the transmission method of the multiplexing protocol comprises the following steps:

step 4, sending a control command for newly-built connection to the server;

step 5, determining a corresponding connection ID based on the source IP and the source port in the data connection request and constructing a data command, wherein the data command comprises the connection ID, the data length and the data per se, and analyzing the data packet type in the data connection request through a server;

2. The protocol of claim 1, wherein the commands include a new connection control command, a transfer data control command, and a close connection control command.

3. The protocol of claim 1, wherein the connection ID is used to split data on one link to distinguish between different connections.

4. The method according to claim 1, wherein the method further comprises:

and when the connection is closed, sending a connection closing control command to a server, and deleting the mapping from the source IP and the source port to the connection ID in the data connection request in the mapping table.

5. The method of claim 1, further comprising, prior to step 6:

and encrypting the data packet in the data connection request by adopting an encryption algorithm.

6. The method of claim 1, wherein the method is performed by a server, comprising:

monitoring the UDP port, and maintaining a mapping table from a connection ID to connection;

receiving a data packet sent by a client, and decrypting the data packet;

analyzing the data packet type, if the data packet type is TCP, sending a confirmation character by using a reliable transmission protocol, continuing to analyze the command field, and if the data packet type is UDP, directly analyzing the command field;

executing the command obtained by analyzing the command field.