CN101159687A - A system, device and method for multi-channel communication data transmission - Google Patents

Abstract

The invention discloses a system, device and method for multi-channel communication and data transmission. The system includes a plurality of nodes and a plurality of switches, and the nodes include a plurality of network interfaces, and the outputs of the plurality of network interfaces are connected to the plurality of switches; the plurality of switches form a physically independent multi-layer interconnection network, forming a node to Multiple channels between nodes; the node is used to send messages to the target address through the interconnection network composed of multiple switches with multiple channels; the node is also used to receive messages sent by the target address as itself through the interconnection network of multiple channels The data. It reduces the delay of data transmission and increases the network bandwidth.

Description

A system, device and method for multi-channel communication data transmission

technical field

The invention relates to the field of computer communication, in particular to a system, device and method for transmitting data through multi-layer independent network parallel multi-channel communication.

Background technique

With the improvement of CPU performance and the development of multi-core technology, inter-node communication has gradually become the performance bottleneck of high-performance computing systems, and a high-bandwidth, low-latency network system has become an urgent need for people.

Specialized interconnection technologies such as Myrinet, Infiniband, and Quadrics, which emerged at the historic moment, alleviate the pressure of network communication to a certain extent, and their high performance provides strong support for the construction of supercomputers.

In order to further increase the bandwidth of the multiprocessor system interconnection network and improve the reliability of the interconnection network, on the one hand, many multiprocessor system interconnection networks begin to use multiple identical networks to connect nodes, such as cluster systems using two sets of Infiniband Network or other commercial network interconnection, etc., such a system has multiple network interfaces to connect to the interconnection network, and uses software to realize the synchronization of communication between different networks.

On the other hand, in order to make up for the difficulty in improving communication performance in a single-link environment, some multi-processor systems use multiple links to increase network bandwidth, such as Cray XT3, Infiniband networks, etc. These multi-links are limited to two points. When data packets are transmitted on these multi-links, the endpoints of each link must be synchronized before being sent to the next link.

Combining the advantages of the above two methods, the prior art provides a multi-channel communication technology.

Multi-channel communication technology is a new technology that uses multiple network links in parallel to improve communication performance and break through the performance bottleneck in the single-link interconnection environment. Multi-channel communication technology directly doubles the physical communication performance available at the application layer by increasing the number of network links between nodes, and integrates multiple network interfaces and multiple network links to achieve multi-channel communication. At the same time, the cost is reduced, which has become one of the development directions of today's communication technology.

However, for multi-network interfaces, software is used to achieve synchronization between different networks. For multi-links, it is necessary to synchronize at the end of each link before downloading, which increases the time for network synchronization processing. This increases network latency.

Contents of the invention

The problem to be solved by the present invention is to provide a system, device and method for multi-channel communication and data transmission, which solves the problem of increased delay caused by the need for synchronization in multi-channel and multi-link in the prior art, and reduces data transmission. The delay increases the network bandwidth.

A system for multi-channel data transmission provided to achieve the purpose of the present invention includes a plurality of nodes and a plurality of switches, the nodes include a plurality of network interfaces, and the output of the plurality of network interfaces is connected to the plurality of switches ;

The multiple switches form a physically independent multi-layer interconnection network, forming multiple channels from node to node;

The node is used to send a message to the target address through an interconnection network composed of multiple switches and having multiple channels.

The node is also used to receive the data sent by the node as the target address through the interconnection network of multiple channels.

The node includes a message splitting module, a distribution module, and a sending module, wherein:

The message splitting module is used to select multiple channels according to the target address and length of the message, and split the message into multiple subtask messages according to the cache size of the switches at each layer of the interconnection network;

The distribution module is configured to distribute the data packets of the multiple subtask messages split by the splitting module to multiple channels respectively;

The sending module is configured to independently send the data packets of the multiple subtask messages allocated by the allocation module to the target address node by using an interconnection network with multiple channels.

The node also includes a receiving module, a recording module, and a judgment processing module, wherein:

The receiving module is used to receive a data packet whose target address is itself and a plurality of subtask messages sent from an interconnection network with a plurality of channels;

The recording module is configured to record a plurality of subtask message packets received by the receiving module through a receiving message recording table;

The judging processing module is configured to judge whether the entire message reception is completed according to the received multiple subtask message data packets recorded by the recording module, and process according to the judging result.

The subtask data packet includes a message header, and the message header includes the type of the packet, the unique identifier of the message to which the packet belongs, the destination address of the data segment, and the data length information.

The packet type is the first packet, the last packet, the middle packet or the first and last packet.

The tail packet in the subtask data packet also includes tail information, and the tail information includes the total number of subpackets transmitted by the message in this channel, the total size of the message and indication information.

The message is a long message.

In order to realize the purpose of the present invention, a multi-channel data transmission node is also provided, and the node sends a message to the target address through an interconnection network composed of multiple switches with multiple channels; and receives the target address as its own through multiple channels data sent from the Internet.

The node includes a message splitting module, a distribution module, and a sending module, wherein:

The message splitting module is used to select multiple channels according to the target address and length of the message, and split the message into multiple subtask messages according to the cache size of the switches at each layer of the interconnection network;

The distribution module is configured to distribute the data packets of the multiple subtask messages split by the splitting module to multiple channels respectively;

The sending module is configured to independently send the data packets of the multiple subtask messages allocated by the allocation module to the target address node by using an interconnection network with multiple channels.

The node also includes a receiving module, a recording module, and a judgment processing module, wherein:

The receiving module is used to receive a data packet whose target address is itself and a plurality of subtask messages sent from an interconnection network with a plurality of channels;

The recording module is configured to record a plurality of subtask message packets received by the receiving module through a receiving message recording table;

The judging processing module is configured to judge whether the entire message reception is completed according to the received multiple subtask message data packets recorded by the recording module, and process according to the judging result.

In order to realize the purpose of the present invention, a method for multi-channel data transmission is further provided, in which a plurality of nodes and a plurality of switches form a physically independent multi-layer interconnection network, forming a plurality of channels from node to node, including the following step:

Step A, the sending end node selects one or more channels according to the target address and length of the message, and splits the message into multiple subtask messages according to the cache size of the switches at each layer of the interconnection network;

In step B, the data packets of multiple subtask messages are allocated to multiple channels respectively, and are independently sent to the target address node by multiple switches of multiple physical layers of multiple channels.

The method for multi-channel data transmission also includes the following steps:

Step C, the receiving end node corresponding to the target address receives data packets of multiple subtask messages sent from the interconnection network with multiple channels;

Step D, the receiving end node records the received multiple subtask message packets through the received message record table;

Step E, judging whether the reception of the whole message is completed, and processing according to the judging result.

Described step A comprises the following steps:

Step A1, the network interface of the sending end node receives a message sending request;

Step A2, selecting the interconnection network channel used according to the length of the message;

Step A3, splitting the message into multiple subtask messages.

Said step D comprises the following steps:

Step D1, receiving the first first packet or first and last packet of a long message from a channel;

Step D2, creating a received message record, and uploading the data packet to the memory of the receiving end node;

Step D3, receiving the intermediate packet of the long message from the same channel or the first packet or intermediate packet from other channels;

Step D4, changing the record of the received message record table, and uploading the data packet to the memory of the receiving end node;

Step D5, receiving tail packets arriving from all used channels;

Step D6, changing the record in the received message record table, uploading the data packet to the memory of the receiving end node, marking the end of channel receiving, and checking whether the packet is lost.

The message is a long message.

The beneficial effects of the present invention are: the system, device and method for multi-channel communication data transmission of the present invention, the data packets in each channel are transmitted independently, no waiting and synchronization are needed, and the multi-network interface and multi-network link can be better utilized The advantage of reducing the correlation between network communications and data packets transmitted on it, and all data operations are completed by hardware, reducing the delay of data transmission. In addition, multiple parallel independent transmissions increase network bandwidth.

Description of drawings

Fig. 1 is a schematic diagram of the system structure of multi-channel communication transmission data;

Fig. 2 is a schematic diagram of the node structure of multi-channel communication data transmission in the present invention;

Fig. 3 is a flow chart of processing a long message sending request by a sending end node in the multi-channel communication data transmission method of the present invention;

Fig. 4 is a schematic diagram of splitting a long message into subtask packages and distributing them to each channel;

Fig. 5 is a schematic diagram of the content contained in the message header of the subtask package;

Fig. 6 is a schematic diagram of the content contained in the tail information in the tail packet type subtask packet;

FIG. 7 is a flowchart of receiving a complete long message by a receiving end node according to an embodiment of the present invention;

Fig. 8 is a schematic structural diagram of a message receiving record table of a receiving end node;

FIG. 9 is a flow chart of processing a subtask packet by a receiving end node according to another embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, a system, device and method for multi-channel communication data transmission of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

As shown in Figure 1, in the system of multi-channel communication transmission data of the present invention, comprise a plurality of nodes 2 and a plurality of switches 1, comprise a plurality of network interfaces 27 in the node 2, each network interface 27 passes I/ The O bus connects to Node 2 memory.

The outputs of multiple network interfaces 27 are connected to multiple switches 1, and these switches 1 form a physically independent multi-layer interconnection network, forming multiple channels from node to node.

As shown in FIG. 2 , the node 2 includes a message splitting module 21 , a distribution module 22 , and a sending module 23 , configured to send a message to a target address through an interconnection network with multiple channels formed by multiple switches 1 .

The message splitting module 21 is used to select multiple channels according to the target address and length of the message, and split the message into multiple subtask messages according to the cache size of the switch 1 at each layer of the interconnection network.

The distribution module 22 is configured to distribute the data packets of the multiple subtask messages split by the splitting module to multiple channels respectively.

The sending module 23 is configured to independently send the data packets of the multiple subtask messages distributed by the distribution module 22 to the target address node 2 through an interconnection network with multiple channels.

At the same time, the node 2 also includes a receiving module 24, a recording module 25, and a judging processing module 26, which are used to receive the message data sent by the node 2 through the interconnection network with multiple channels.

The receiving module 24 is configured to receive data packets of multiple subtask messages sent from the interconnection network with multiple channels whose target address is itself.

The recording module 25 is configured to record the multiple subtask message packets received by the receiving module 24 through the receiving message recording table.

The judging processing module 26 is configured to judge whether the entire message reception is completed according to the received multiple subtask message data packets recorded by the recording module 25, and process according to the judging result.

As a possible implementation manner, as shown in FIG. 1 , the switches Switch_A, Switch_B, Switch_C and Switch_D are four independent switches 1, which form a physically independent four-layer interconnection network. Each layer of switching network is composed of one-level or multi-level switching chips, and each layer of switching network has multiple interfaces (4 in the illustration), which are respectively connected to multiple nodes 2 .

The network interface 27 of each node 0-3 is divided into four channels A, B, C, and D, which are respectively connected to the four-layer switching network. Node 0 and node 3 can communicate independently through 4 links.

After the network interface 27 obtains the message data from the memory of the node 2 through the I/O bus, it decomposes the message into data packets according to the size of the message, and distributes the message to the multi-layer interconnection network composed of multiple switches 1 for transmission.

Data packets on different physical layers on the interconnection network do not need to be forwarded to the next link after the endpoints of each link are synchronized, but are transmitted independently until reaching the network interface 27 of the target node 2 .

The network interface 27 of the target node 2 records the number of data packets that have been written into the internal memory. When the data packets of the entire message are all received, write the received message completion event in the internal memory of the target node 2 to notify the completion of a message data. arrive.

The working process of the multi-channel communication data transmission system of the present invention is described in detail below, that is, the method process of a multi-channel communication data transmission method provided by the present invention includes the following steps:

Step S100, the sender node 2 selects multiple channels according to the destination address and length of the message, and splits the message into multiple subtask messages according to the cache size of the switch 1 at each layer of the interconnection network.

How to realize parallel transmission of a long message data is taken as an example below to illustrate the system, device and method for multi-channel communication data transmission of the present invention. However, it should be noted that it is not a limitation of the present invention, and the present invention is also applicable to short messages. parallel transfer process.

As shown in Figure 3, it is the processing flow of the sending end node 2 to the long message sending request, including the following steps:

Step S110, the sending end node 2 receives a sending request of a long message;

Step S120, selecting the interconnection network channel to be used according to the message length;

At the sending end node 2, a long message usually includes a message header and a large piece of data to be transmitted (the data can be an immediate value, or a piece of data in the memory to be fetched). Select the channel to go according to the target address and the length of the data to be transmitted;

According to the target address of the message and the message length, one, two, three, four channels, or more channels can be selected. In the embodiment of the present invention, all four channels are selected.

Step S130, splitting the long message into multiple subtask messages.

Affected by the size of the sending and receiving buffer of switch 1, the long message is divided into a series of subtasks, so as to facilitate the segmented transmission of the message.

As shown in Figure 4, it is a schematic diagram of splitting long message data into subtasks.

Determine the size of the sub-task packet according to the size of the sending and receiving buffer of switch 1, divide the long message into a series of sub-task messages, and send them to the four channels A, B, C, and D respectively, that is, the data is transmitted independently in sequence in Figure 2 Block 1, data block 2, ..., data block N.

Each subtask package contains a clear message header, which contains information such as the package type, the unique identifier of the long message to which this package belongs, the destination address and data length of this segment of data, as shown in Figure 5.

Wherein, the package type is the first package, the last package, the middle package or the first and last package.

The first and last packets refer to the fact that the long message has only one subpacket in this channel.

The tail packet also includes tail information, and the tail information includes the total number of sub-packets of the long message transmitted in this channel, the total size of the long message and indication information, etc., as shown in Figure 6,

The last subtask independently retrieves data from the memory through Direct Memory Access (DMA), constructs a data packet, and sends it to the interconnection network.

In step S200, the data packets of multiple subtask messages are distributed to multiple channels respectively, and are sent to the target address node 2 independently by multiple switches 1 of multiple physical layers of multiple channels.

The data packets of multiple subtask messages are assigned to multiple channels respectively, the first subtask packet allocated to each channel is marked as the first packet of this channel, and the last one is marked as the last packet of this channel.

Each subtask packet is transmitted independently on the interconnection network, and arrives at the network interface 27 of the receiving end node 2 one after another.

Step S300, the receiving end node 2 corresponding to the target address sends data packets of multiple subtask messages from the interconnection network with multiple channels;

On the target node 2, the data on different physical layers on the interconnection network are directly written into the memory by DMA according to the target address it carries, without waiting and synchronizing with each other.

Step S400, the receiving end node 2 records the received multiple subtask message data packets through the received message recording table;

As shown in FIG. 7 , it is a flow chart of the receiving end node 2 receiving a complete long message.

Step S400 includes the following steps:

Step S410, receiving the first first packet or the first and last packets of a long message from a channel;

In the case that the network link does not fail, the receiving end first receives a first packet or a first and last packet from a physical channel.

In the present invention, in the receiving end node 2, a received message record table is maintained, as shown in FIG. 8 .

Step S420, create a received message record, and upload the data packet to the memory of the receiving end node 2;

After the receiving end node 2 receives the first packet or the first and last packets, because this is the first packet of a long message, it needs to create a new record in the received message record table to record the arrival of a long message, and uniquely Fill in the message record ID for the property ID. And from the message header of the first packet or the first and last packet, you can know which physical channels the long message will pass through, and then set the valid flag bit of the corresponding channel to valid 1 in the record table, and clear the receiving count of each channel accordingly .

Then upload the data packet to the memory of the receiving end node 2 according to the data length and the destination address.

Step S430, receiving the intermediate packet of the long message from the same channel or the first packet or intermediate packet from other channels;

Next, the first packet of another physical channel, or the middle or last packet of the same physical channel may be received.

After the data transmission is completed, add 1 to the receiving count of the channel that the data packet traveled in the receiving message record table.

At this time, the record of the same long message is already contained in the received message record table, and the data is uploaded according to the data length and destination address in the message header, and the corresponding channel receiving count of the corresponding record in the received message record table is increased by 1, indicating a new message the arrival of data packets.

Step S440, changing the record in the received message record table, and uploading the data packet to the memory of the receiving end node 2;

Step S450, receiving tail packets arriving from all used channels;

Step S460, changing the record in the received message record table, uploading the data packet to the memory of the receiving end node 2, marking the end of channel receiving, and checking whether the packet is lost.

For the tail packet, after completing the same operation as the middle packet, compare the number of subtask packets of this channel in the tail packet information with the reception count in the received message record table. If there is a difference, it indicates that there is a data packet loss, and the packet is lost. deal with.

As another possible implementation manner, a more detailed process of the processing flow after the receiving end node 2 receives a data packet is shown in FIG. 9 . First judge the type of the received data packet, then do corresponding processing according to the type of the packet, then upload the data and maintain the received message record table at the same time, finally judge that the long message has been received, and notify the receiving end node 2 to clear the received message record of this message surface.

Step S500, judging whether the reception of the entire message is completed, and processing according to the judging result.

After all the tail packets of the physical channel used have arrived, it indicates that a long message has been received. At this time, according to the long message indication information in the tail information of the last tail packet, it is judged whether all subtask packets have been received, and according to the judgment result, select Yes Whether the sending is completed or interrupted, and the received message record table of this message is cleared.

In the system, device and method for multi-channel communication data transmission of the present invention, the data packets in each channel are transmitted independently, no waiting and synchronization are required, the advantages of multi-network interfaces and multi-network links are better utilized, and the network communication time is reduced. And the correlation between data packets transmitted on it, and all data operations are completed by hardware, reducing the delay of data transmission. In addition, multiple parallel independent transmissions increase network bandwidth.

Other aspects and features of the present invention will be apparent to those skilled in the art from the above description of specific embodiments of the present invention in conjunction with the accompanying drawings.

The specific embodiments of the present invention have been described and illustrated above, and these embodiments should be considered as exemplary only, and are not used to limit the present invention, and the present invention should be interpreted according to the appended claims.

Claims (16)

1. A system for multi-channel data transmission, comprising a plurality of nodes and a plurality of switches, wherein the node includes a plurality of network interfaces, and the output of the plurality of network interfaces is connected to the plurality of switches; The multiple switches form a physically independent multi-layer interconnection network, forming multiple channels from node to node; The node is used to send a message to the target address through an interconnection network composed of multiple switches and having multiple channels. 2 . The system for multi-channel data transmission according to claim 1 , wherein the node is further configured to receive the data sent by the node itself through the interconnection network of multiple channels. 3 . 3. The system according to claim 1 or 2, wherein said node comprises a message splitting module, a distribution module, and a sending module, wherein: The message splitting module is used to select multiple channels according to the target address and length of the message, and split the message into multiple subtask messages according to the cache size of the switches at each layer of the interconnection network; The distribution module is configured to distribute the data packets of the multiple subtask messages split by the splitting module to multiple channels respectively; The sending module is configured to independently send the data packets of the multiple subtask messages allocated by the allocation module to the target address node by using an interconnection network with multiple channels. 4. The system for multi-channel data transmission according to claim 3, wherein the node further comprises a receiving module, a recording module, and a judgment processing module, wherein: The receiving module is used to receive a data packet whose target address is itself and a plurality of subtask messages sent from an interconnection network with a plurality of channels; The recording module is configured to record a plurality of subtask message packets received by the receiving module through a receiving message recording table; The judging processing module is configured to judge whether the entire message reception is completed according to the received multiple subtask message data packets recorded by the recording module, and process according to the judging result. 5. The system for multi-channel data transmission according to claim 4, wherein the subtask data packet includes a message header, and the message header includes the unique identification of the packet type, the message to which the packet belongs, the message header, and the message header. Destination address and data length information of segment data. 6. The system for multi-channel data transmission according to claim 5, characterized in that, the packet type is a first packet, a tail packet, a middle packet or a first and last packet. 7. The system for multi-channel data transmission according to claim 6, wherein the tail packet in the subtask data packet also includes tail information, and the tail information includes the total number of messages transmitted in this channel. Number of subpackets, total message size and indication. 8. The system for multi-channel data transmission according to claim 4, wherein the message is a long message. 9. A node for multi-channel data transmission, characterized in that, the node sends a message to the target address through an interconnection network with multiple channels formed by a plurality of switches; Data sent from the Internet. 10. The node for multi-channel data transmission according to claim 9, wherein the node comprises a message splitting module, an allocation module, and a sending module, wherein: The message splitting module is used to select multiple channels according to the target address and length of the message, and split the message into multiple subtask messages according to the cache size of the switches at each layer of the interconnection network; The distribution module is configured to distribute the data packets of the multiple subtask messages split by the splitting module to multiple channels respectively; The sending module is configured to independently send the data packets of the multiple subtask messages allocated by the allocation module to the target address node by using an interconnection network with multiple channels. 11. The node for multi-channel data transmission according to claim 10, wherein the node further comprises a receiving module, a recording module, and a judgment processing module, wherein: The receiving module is used to receive a data packet whose target address is itself and a plurality of subtask messages sent from an interconnection network with a plurality of channels; The recording module is configured to record a plurality of subtask message packets received by the receiving module through a receiving message recording table; The judging processing module is configured to judge whether the entire message reception is completed according to the received multiple subtask message data packets recorded by the recording module, and process according to the judging result. 12. A method for multi-channel data transmission, characterized in that, a plurality of nodes and a plurality of switches form a physically independent multi-layer interconnection network, forming a plurality of channels from node to node, including Follow the steps below: Step A, the sending end node selects multiple channels according to the target address and length of the message, and splits the message into multiple subtask messages according to the cache size of the switches at each layer of the interconnection network; In step B, the data packets of multiple subtask messages are allocated to multiple channels respectively, and are independently sent to the target address node by multiple switches of multiple physical layers of multiple channels. 13. The method for multi-channel data transmission according to claim 12, further comprising the following steps: Step C, the receiving end node corresponding to the target address sends data packets of multiple subtask messages from the interconnection network with multiple channels; Step D, the receiving end node records the received multiple subtask message packets through the received message record table; Step E, judging whether the reception of the whole message is completed, and processing according to the judging result. 14. The method for multi-channel data transmission according to claim 12 or 13, wherein said step A comprises the following steps: Step A1, the network interface of the sending end node receives a message sending request; Step A2, selecting the interconnection network channel used according to the length of the message; Step A3, splitting the message into multiple subtask messages. 15. The method for multi-channel data transmission according to claim 13, wherein said step D comprises the following steps: Step D1, receiving the first first packet or first and last packet of a message from a channel; Step D2, creating a received message record, and uploading the data packet to the memory of the receiving end node; Step D3, receiving the intermediate packet of the message from the same channel or the first packet or intermediate packet from other channels; Step D4, changing the record of the received message record table, and uploading the data packet to the memory of the receiving end node; Step D5, receiving tail packets arriving from all used channels; Step D6, changing the record in the received message record table, uploading the data packet to the memory of the receiving end node, marking the end of channel receiving, and checking whether the packet is lost. 16. The method for multi-channel data transmission according to claim 12 or 13, characterized in that the message is a long message.

Images