CN118433174A - Multichannel multilink file transmission method and system - Google Patents
Multichannel multilink file transmission method and system Download PDFInfo
- Publication number
- CN118433174A CN118433174A CN202410656956.0A CN202410656956A CN118433174A CN 118433174 A CN118433174 A CN 118433174A CN 202410656956 A CN202410656956 A CN 202410656956A CN 118433174 A CN118433174 A CN 118433174A
- Authority
- CN
- China
- Prior art keywords
- file
- queue
- transmission
- queues
- receiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 239
- 238000000034 method Methods 0.000 title claims abstract description 41
- 108091026890 Coding region Proteins 0.000 claims abstract description 14
- 238000004891 communication Methods 0.000 claims description 22
- 238000012546 transfer Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 239000000872 buffer Substances 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the technical field of file transmission, and particularly provides a multichannel and multilink file transmission method and system, wherein the method comprises the following steps: detecting a file to be transmitted from a transmission catalog, and when the file to be transmitted is detected, converting the file to be transmitted into file messages and respectively storing the file messages into two transmission queues; randomly selecting one sending queue from the two queues as a first sending queue, taking out the file messages from the first sending queue, distributing a code for each file message as a packet head to be overlapped in the file message, and distributing the file message to N sending sub-queues; respectively establishing N channels to orderly transmit file messages in N transmission sub-queues of a transmitting node to N corresponding receiving sub-queues in a receiving node; and acquiring the file messages from the N receiving sub-queues, acquiring the codes of each file message, and storing the acquired file messages into the first receiving queue according to the coding sequence. The transmission performance and the real-time performance are better, and the reliability of file transmission can be ensured.
Description
Technical Field
The invention relates to the technical field of file transmission, in particular to a multichannel and multilink file transmission method and system.
Background
The existing file transmission technology such as FTP protocol, file uploading and downloading based on HTTP and the like can finish file transmission under good network conditions, but under unstable network environments (such as mobile networks), abnormal interruption of file transmission is often caused by high delay, packet loss and the like, and although an application layer can retransmit a file through retransmission and other mechanisms, the reliability and real-time of the whole file transmission process are unsatisfactory. The file transmission needs to take account of two different types of communication requirements of control of file transmission states and transmission of file data. The message data used for controlling the file transmission state is generally shorter and smaller, and the requirements on the reliability and the real-time performance of the communication are very high because the state consistency of the file transmitting and receiving parties needs to be coordinated. In the unstable network environment, if reliability of file data transmission is pursued, when network packet loss occurs or there is no response for a long time, retrying to send file data often generates invalid retries or repeatedly sends data to waste network bandwidth resources, resulting in reduced instantaneity of file transmission.
The existing file transmission technology often adopts single-channel communication, and cannot well balance two different communication requirements of control of file transmission state and transmission of file data, and generally, the real-time performance is sacrificed for guaranteeing the reliability of file transmission or the reliability is lost for pursuing the real-time performance of file transmission.
Through the above, in the conventional technology, the file data is transmitted on one channel, and in the normal condition of the network, if the amount of the file data to be transmitted is large (such as live broadcast and retransmission of a large event, transmission of a high-definition video file, etc.), the file data is easily affected by factors such as network bandwidth, transmission delay, link stability, etc., so that the transmission efficiency is low, even if the transmission fails, even if the server where the file is located has multiple transmission links, the conventional technology cannot use multiple links for transmission at the same time, and the performance and instantaneity of file transmission cannot be guaranteed.
Disclosure of Invention
Aiming at the problem that a plurality of links cannot be used for transmission simultaneously in the related art, and the performance and instantaneity of file transmission cannot be guaranteed, the invention provides a multichannel multilink file transmission method and a multichannel multilink file transmission system.
On one hand, the technical scheme of the invention provides a multichannel multilink file transmission method which is applied to file transmission between a sending node and a receiving node, and the multilink communication connection between the sending node and the receiving node is realized; the method comprises the following steps:
Creating two sending queues and N sending sub-queues at a sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node; n 2;
Detecting a file to be transmitted from a transmission catalog, and when the file to be transmitted is detected, converting the file to be transmitted into file messages and respectively storing the file messages into two transmission queues;
Randomly selecting one sending queue from the two queues as a first sending queue, taking out the file messages from the first sending queue, distributing a code for each file message as a packet head to be overlapped in the file message, and distributing the file message to N sending sub-queues; the remaining one of the two transmit queues is used as a second transmit queue;
establishing a channel to orderly transmit file messages in a second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; respectively establishing N channels to orderly transmit file messages in N transmission sub-queues of a transmitting node to N corresponding receiving sub-queues in a receiving node; wherein one channel uses one link; a receiving queue corresponding to the second sending queue in the two receiving queues is defined as a second receiving queue, and the other receiving queue is a first receiving queue;
And acquiring the file messages from the N receiving sub-queues, acquiring the codes of each file message, and storing the acquired file messages into the first receiving queue according to the coding sequence.
As a further limitation of the technical solution of the present invention, the step of creating two transmit queues and N transmit sub-queues at the transmitting node, and creating corresponding two receive queues and N receive sub-queues at the receiving node includes:
Setting weights of the two transmission queues according to the sum of the available bandwidth of the link used by the second transmission queue and the available bandwidth of the link used by each transmission sub-queue;
The weight of each transmit sub-queue is set according to the available bandwidth of the link used by each transmit sub-queue.
As a further limitation of the technical solution of the present invention, the step of detecting a file to be transmitted from a transmission directory, and when the file to be transmitted is detected, converting the file to be transmitted into file messages and storing the file messages in two transmission queues respectively includes:
detecting a file to be transmitted from a transmission directory, and converting the file to be transmitted into a file message when the file to be transmitted is detected;
Dividing the file messages according to the quantity of the file messages based on the set weights, and storing the file messages in a first transmission queue and a second transmission queue respectively.
As a further limitation of the technical solution of the present invention, the step of randomly selecting one transmission queue from the two queues as the first transmission queue, taking out the file message from the first transmission queue, allocating a code as a header to each file message to be added to the file message, and distributing the file message to N transmission sub-queues includes:
Randomly selecting one transmission queue from the two queues as a first transmission queue, taking out the file messages from the first transmission queue, distributing a code for each file message as a packet header to be overlapped in the file message, and distributing the file message to N transmission sub-queues according to the weight set by the transmission sub-queues.
As a further limitation of the technical solution of the present invention, the method further comprises: in the transmission process, when one of the links fails, transferring the channel established on the failed link to a normal link;
when the failed link is restored, the tunnel is re-established on the restored link.
As a further limitation of the technical solution of the present invention, the steps of acquiring the file message in the N receiving sub-queues and acquiring the code of each file message, and storing the acquired file message in the first receiving queue according to the coding sequence include:
acquiring file messages from N receiving sub-queues;
And obtaining message codes from the file message packet heads, and storing the file messages with the packet heads overlapped by the sending nodes removed in a first receiving queue according to the message coding sequence, so that the content and the sequence of the file messages stored in the first receiving queue are completely consistent with those of the file messages stored in the first sending queue.
As a further limitation of the technical solution of the present invention, the steps of acquiring the file messages in the N receiving sub-queues and acquiring the codes of each file message, and storing the acquired file messages in the first receiving queue according to the coding sequence include:
File messages are obtained from the first receive queue and the second receive queue and reassembled into files in a receive directory for storage at a receiving node.
On the other hand, the technical scheme of the invention also provides a multichannel multilink file transmission system, which comprises a sending node and a receiving node, wherein the sending node is connected with the receiving node in a multilink manner; the sending node and the receiving node are respectively provided with a communication middleware with a message queue service function; the sending node is provided with a double-channel double-link file sending service module, a file transmission service module and a multi-channel multi-link file transmission message distribution service module; the receiving node is provided with a multichannel multilink file transmission message merging service module;
Creating two sending queues and N sending sub-queues at a sending node through a communication middleware of the sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node through a communication middleware of the receiving node; n 2;
The dual-channel double-link file sending service module detects files to be sent from a sending directory, and when the files to be sent are detected, the files to be sent are converted into file messages and are respectively stored in two sending queues;
The multi-channel multi-link file transmission message distribution service module randomly selects one transmission queue from the two queues as a first transmission queue, takes out file messages from the first transmission queue, distributes a code for each file message as a packet head to be overlapped in the file message and distributes the file message to N transmission sub-queues; the remaining one of the two transmit queues is used as a second transmit queue;
The file transmission service module is used for establishing a channel to orderly transmit the file information in the second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; the method is also used for respectively establishing N channels to orderly transmit the file messages in N transmission sub-queues of the transmitting node to N corresponding receiving sub-queues in the receiving node; wherein one channel uses one link; a receiving queue corresponding to the second sending queue in the two receiving queues is defined as a second receiving queue, and the other receiving queue is a first receiving queue;
The multi-channel multi-link file transmission message merging service module is used for acquiring file messages in N receiving sub-queues and acquiring codes of each file message, and storing the acquired file messages into a first receiving queue according to the coding sequence.
As a further limitation of the technical scheme of the invention, a double-channel double-link file receiving service module is also arranged on the receiving node;
the dual-channel double-link file receiving service module acquires file messages from the first receiving queue and the second receiving queue and reassembles the file messages into a file in the receiving catalog.
As a further limitation of the technical scheme of the present invention, the system further comprises a weight setting module, which sets the weights of the two transmission queues according to the sum of the available bandwidth of the link used by the second transmission queue and the available bandwidth of the link used by each transmission sub-queue; setting the weight of each transmission sub-queue according to the available bandwidth of the link used by each transmission sub-queue;
The double-channel double-link file sending service module converts a file to be sent into a file message when detecting the file to be sent; dividing the file message according to the quantity of the file message based on the set weight, and respectively storing the file message into a first transmission queue and a second transmission queue;
The multi-channel multi-link file transmission message distribution service module takes out the file messages from the first transmission queue, distributes a code for each file message as a packet header to be overlapped in the file message, and distributes the file message to N transmission sub-queues according to the weight set by the transmission sub-queues.
From the above technical scheme, the invention has the following advantages: the multi-channel multi-link transmission is arranged, the available bandwidth of file transmission is higher, the transmission performance and the real-time performance are better, the reliability of file transmission can be ensured, and the method is more suitable for the scene of high-bandwidth multi-link high-speed file transmission.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
It can be seen that the present invention has outstanding substantial features and significant advances over the prior art, as well as its practical advantages.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a method of one embodiment of the invention.
Fig. 2 is a schematic flow chart of a method of another embodiment of the invention.
FIG. 3 is a schematic block diagram of a system of one embodiment of the present invention.
Detailed Description
According to the invention, the message queue of the IROS is utilized for data caching, the multichannel multilink is utilized for file data transmission, the file transmission performance is improved by several times, and even if one link is disconnected, the file can be reliably transmitted. A system is provided that includes a transmitting node and a receiving node. An IROS, a double-channel double-link file sending service module, a file transmission service module and a multi-channel multi-link file transmission message distribution service module are installed on a sending node to provide file sending service and distribution service, and the functions are that files to be sent in a sending directory are converted into file messages, channels are established and the file messages are sent to a receiving node. An IROS, a multi-channel multi-link file transmission message merging service module and a dual-channel double-link file receiving service module are installed on a receiving node to provide file merging and receiving service, and the functions are that file messages sent by a sending sub-queue are merged and written into a corresponding receiving queue, and the file messages in the receiving queue are reassembled into an original file format and stored in a designated receiving directory. In the invention, each channel of the multi-channel can use different links, such as a channel uses a private line link, another channel uses an internet link, another channel uses a connected private line link, another channel uses a mobile private line link and the like, if the links are all normal, a plurality of links can be used for transmission at the same time, if one of the links fails, and the rest of the links can also independently complete transmission tasks. In order to make the technical solution of the present invention better understood by those skilled in the art, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that, IROS is fully called an intelligent robot operating system, is similar to an open source ROS in function, can support development and operation of an intelligent robot, and is a set of communication middleware, which is similar to ROS in function, but has higher performance, reliability and safety than ROS.
The channel is as follows: here, a logical concept in data transmission is that a channel is established between two nodes, and data such as a file can be transmitted in the channel, where the channel can be based on TCP, UDP protocols, and the like.
And (3) link: the link is a transmission medium in which two nodes are actually located, for example, two servers transmit through two network lines or one network line and wireless transmission, and transmit between different places through special lines and the internet.
As shown in fig. 1, an embodiment of the present invention provides a multi-channel multi-link file transmission method, which is applied to file transmission between a sending node and a receiving node, where the sending node and the receiving node are in multi-link communication connection; the method comprises the following steps:
Step 1: creating two sending queues and N sending sub-queues at a sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node; n 2;
In the step, a sending node establishes two IROS sustainable storage queues at the sending node by installing a communication middleware with a message queue service function, and the two IROS sustainable storage queues are used for caching file messages to be sent; two IROS persistable store queues are created at the receiving node for caching received file messages.
N IROS sustainable storage queues-sending sub-queues are newly created at the sending node and are used for caching file messages to be sent and corresponding to a new sending channel; newly creating N IROS sustainable storage queues-receiving sub-queues at the receiving node for caching the received file messages; the newly added transmitting sub-queues and receiving sub-queues are in one-to-one correspondence, and a transmitting channel is established between every two sub-queues.
Step 2: detecting a file to be transmitted from a transmission catalog, and when the file to be transmitted is detected, converting the file to be transmitted into file messages and respectively storing the file messages into two transmission queues;
Step 3: randomly selecting one sending queue from the two queues as a first sending queue, taking out the file messages from the first sending queue, distributing a code for each file message as a packet head to be overlapped in the file message, and distributing the file message to N sending sub-queues; the remaining one of the two transmit queues is used as a second transmit queue;
After the file message is fetched from the first sending queue, a number is allocated to the message, the number is an unsigned integer value, the number is increased according to a step value of 1, and starts from 0 again after reaching the maximum value, and the number is used as a packet header to be newly overlapped into the original file message to be cached and sent together.
Step 4: establishing a channel to orderly transmit file messages in a second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; respectively establishing N channels to orderly transmit file messages in N transmission sub-queues of a transmitting node to N corresponding receiving sub-queues in a receiving node; wherein one channel uses one link; a receiving queue corresponding to the second sending queue in the two receiving queues is defined as a second receiving queue, and the other receiving queue is a first receiving queue;
In the invention, each channel of the multi-channel can use different links, such as a channel uses a private line link, a channel uses an internet link, a channel uses a connected private line link, a channel uses a mobile private line link and the like, if the links are all normal, a plurality of links can be used for transmission at the same time, if one of the links fails, and the rest of the links can also independently complete transmission tasks.
Step 5: and acquiring the file messages from the N receiving sub-queues, acquiring the codes of each file message, and storing the acquired file messages into the first receiving queue according to the coding sequence.
In some embodiments, as shown in fig. 2, the steps of creating two transmit queues and N transmit sub-queues at the transmitting node, and creating corresponding two receive queues and N receive sub-queues at the receiving node, include:
s1-2: the weights of the two sending queues are set according to the sum of the available bandwidth of the second sending queue using link and the available bandwidth of each sending sub-queue using link, and the weights of the sending sub-queues are set according to the available bandwidth of the sending sub-queues using link.
When the weight is set, calculating the sum of the available bandwidths of the links used by all the transmission sub-queues and the available bandwidth of the links used by the second transmission queue; setting weights of the first transmission queue and the second transmission queue according to the ratio of the sum of the available bandwidths of the transmission sub-queues to the available bandwidth of the link used by the second transmission queue; wherein the larger the value of the usable bandwidth, the higher the weight, and the stronger the transmission capability of the queue;
correspondingly, the available bandwidth of the link used by each transmission sub-queue is calculated, and the weight of each transmission sub-queue is set according to the ratio of each available bandwidth.
Correspondingly, detecting the file to be sent from the sending directory, and when the file to be sent is detected, converting the file to be sent into file messages and respectively storing the file messages into two sending queues, wherein the steps comprise:
Detecting a file to be transmitted from a transmission directory, and converting the file to be transmitted into a file message when the file to be transmitted is detected; dividing the file messages according to the quantity of the file messages based on the set weights, and storing the file messages in a first transmission queue and a second transmission queue respectively. Randomly selecting one transmission queue from the two queues as a first transmission queue, taking out the file messages from the first transmission queue, distributing a code for each file message as a packet header to be overlapped in the file message, and distributing the file message to N transmission sub-queues according to the weight set by the transmission sub-queues.
Specifically, the first transmission queue and the second transmission queue may set different weights, where the weight value is a positive integer, and the default value is 1, and the larger the value, the stronger the transmission capability of the queue. If the sum of the available bandwidths of all the sending sub-queues using links is 100Mb, the second sending queue uses one link of 20Mb for transmission, the weight of the first sending queue may be set to 10, the weight of the second sending queue may be set to 2, if there are 12 file messages to be transmitted, the first sending queue will buffer 10 file messages for transmission, and the second sending queue buffers 2 file messages for transmission. The weight setting mode of each sending sub-queue is approximately the same, and is only set according to the ratio of the bandwidths of the links used by each sending sub-queue.
In some embodiments, the method further comprises: in the transmission process, when one of the links fails, transferring the channel established on the failed link to a normal link;
when the failed link is restored, the tunnel is re-established on the restored link.
In the file transmission process, transmission faults are timely found and processed by monitoring the transmission states of the channel and the link in real time, so that the reliability of file transmission is ensured. Specifically, a heartbeat mechanism is established between the sending node and the receiving node and is used for detecting the link connection state, and when the link disconnection is detected, a channel established on the fault link is transferred to another normal link; and after the fault recovery, recovering to establish a channel and recovering file transmission.
The link connection state is detected through a heartbeat mechanism, and when the link is detected to be disconnected, a channel is automatically transferred and file transmission is restored, so that the method has stronger adaptability and robustness. Even under the condition of unstable network environment, smooth file transmission can be ensured.
In some embodiments, the method further comprises: the multiple channels established are set up to use the same transmission algorithm or different transmission algorithms or not all the same transmission algorithms.
In the file transmission process, different links can process file messages in different modes, so that the safety of file transmission is ensured.
In some embodiments, the steps of obtaining the file message in the N receiving sub-queues and obtaining the code of each file message, and storing the obtained file message in the first receiving queue according to the code order include:
Step 51: acquiring file messages from N receiving sub-queues;
step 52: and obtaining message codes from the file message packet heads, and storing the file messages with the packet heads overlapped by the sending nodes removed in a first receiving queue according to the message coding sequence, so that the content and the sequence of the file messages stored in the first receiving queue are completely consistent with those of the file messages stored in the first sending queue.
In some embodiments, the steps of obtaining the file messages in the N receiving sub-queues and obtaining the codes of each file message, and storing the obtained file messages in the first receiving queue according to the coding order include:
Step 6: file messages are obtained from the first receive queue and the second receive queue and reassembled into files in a receive directory for storage at a receiving node.
The number of the channels and the number of the links can be the same or different, and one channel can select one link to use.
The method utilizes the ordered characteristic of the IROS message queue, and utilizes an algorithm similar to merging and sorting to quickly merge into the target file from the first receiving queue and the second receiving queue, and does not need new buffering, message sorting and the like in the process, so that the method has excellent performance. After receiving the file message, the receiving node checks the file message to ensure the integrity of the file message. The file message in the embodiment of the invention is a data sheet with a set size range.
When the file messages in each receiving sub-queue are combined, when the situation that the file messages sent by different channels firstly are possible to arrive later is met, the file messages sent earlier and later are cached in each receiving sub-queue in the IROS message queue and are combined according to the number sequence, and then the file messages are put into the first receiving queue.
The method provided in some embodiments is based on a file transfer system, the system including a sending node and a receiving node, the sending node being coupled to the receiving node in multiple links; the sending node and the receiving node are respectively provided with a communication middleware with a message queue service function; the sending node is provided with a double-channel double-link file sending service module, a file transmission service module and a multi-channel multi-link file transmission message distribution service module; the receiving node is provided with a multichannel multilink file transmission message merging service module; creating two sending queues and N sending sub-queues at a sending node through a communication middleware of the sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node through a communication middleware of the receiving node; n2; The dual-channel double-link file sending service module detects files to be sent from a sending directory, and when the files to be sent are detected, the files to be sent are converted into file messages and are respectively stored in two sending queues; the multi-channel multi-link file transmission message distribution service module randomly selects one transmission queue from the two queues as a first transmission queue, takes out file messages from the first transmission queue, distributes a code for each file message as a packet head to be overlapped in the file message and distributes the file message to N transmission sub-queues; the file transmission service module establishes a channel to orderly transmit the file information in the second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; the method is also used for respectively establishing N channels to orderly transmit the file messages in N transmission sub-queues of the transmitting node to N corresponding receiving sub-queues in the receiving node; the multi-channel multi-link file transmission message merging service module is used for acquiring file messages in N receiving sub-queues and acquiring codes of each file message, and storing the acquired file messages into a first receiving queue according to the coding sequence. The dual-channel double-link file receiving service module acquires file messages from the first receiving queue and the second receiving queue and reassembles the file messages into a file in the receiving catalog. The file messages in the two receiving queues are reassembled into the original file format and stored in the designated receiving directory.
As shown in fig. 3, the embodiment of the present invention further provides a multi-channel multi-link file transmission system, which includes a sending node and a receiving node, where the sending node is connected with the receiving node in a multi-link manner; the sending node and the receiving node are respectively provided with a communication middleware with a message queue service function; the sending node is provided with a double-channel double-link file sending service module, a file transmission service module and a multi-channel multi-link file transmission message distribution service module; the receiving node is provided with a multichannel multilink file transmission message merging service module;
Creating two sending queues and N sending sub-queues at a sending node through a communication middleware of the sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node through a communication middleware of the receiving node; n 2;
The dual-channel double-link file sending service module detects files to be sent from a sending directory, and when the files to be sent are detected, the files to be sent are converted into file messages and are respectively stored in two sending queues;
The multi-channel multi-link file transmission message distribution service module randomly selects one transmission queue from the two queues as a first transmission queue, takes out file messages from the first transmission queue, distributes a code for each file message as a packet head to be overlapped in the file message and distributes the file message to N transmission sub-queues; the remaining one of the two transmit queues is used as a second transmit queue;
The file transmission service module is used for establishing a channel to orderly transmit the file information in the second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; the method is also used for respectively establishing N channels to orderly transmit the file messages in N transmission sub-queues of the transmitting node to N corresponding receiving sub-queues in the receiving node; wherein one channel uses one link; a receiving queue corresponding to the second sending queue in the two receiving queues is defined as a second receiving queue, and the other receiving queue is a first receiving queue;
The multi-channel multi-link file transmission message merging service module is used for acquiring file messages in N receiving sub-queues and acquiring codes of each file message, and storing the acquired file messages into a first receiving queue according to the coding sequence.
In the embodiment of the invention, the communication middleware with the message queue service function can be IROS, the IROS has the message queue service function, and two IROS sustainable storage queues are created at the sending node and used for caching file messages to be sent; two IROS persistable store queues are created at the receiving node for caching received file messages.
A multi-channel multi-link file transmission message distribution service module is installed at a transmitting node, a plurality of transmitting sub-queues are created, and the function is to transmit files to be transmitted in a transmitting directory to a receiving node in a file message mode through a plurality of channels/links. A multi-channel multi-link file transmission message merging service module is installed at a receiving node, a plurality of receiving sub-queues are created, and the function is to merge file messages sent by a sending node through a plurality of channels/links and then write the file messages into the receiving queues. The number of the channels and the number of the links can be the same or different, and one channel can select one link to use.
In some embodiments, a dual-channel dual-link file receiving service module is further installed on the receiving node;
The dual-channel double-link file receiving service module acquires file messages from the first receiving queue and the second receiving queue and reassembles the file messages into a file in the receiving catalog. The file messages in the two receiving queues are reassembled into the original file format and stored in the designated receiving directory.
In some embodiments, the system further comprises a weight setting module that sets weights of the two transmit queues according to a sum of an available bandwidth of the link used by the second transmit queue and an available bandwidth of the link used by each transmit sub-queue; setting the weight of each transmission sub-queue according to the available bandwidth of the link used by each transmission sub-queue;
The double-channel double-link file sending service module converts a file to be sent into a file message when detecting the file to be sent; dividing the file message according to the quantity of the file message based on the set weight, and respectively storing the file message into a first transmission queue and a second transmission queue;
The multi-channel multi-link file transmission message distribution service module takes out the file messages from the first transmission queue, distributes a code for each file message as a packet header to be overlapped in the file message, and distributes the file message to N transmission sub-queues according to the weight set by the transmission sub-queues.
The same physical link can also be used by a plurality of channels in the channels established by the file transmission service module, for example, when one of the physical links fails, the channel established by the file transmission service module before the failed link can be transferred to the other physical link, and when the failed link is recovered, the channel is re-established.
The multi-channel multi-link file transmission message distribution service module takes out messages from one transmission queue (the first transmission queue in fig. 3) and distributes different messages according to different weight values of the transmission sub-queues to store the messages in a plurality of transmission sub-queues (the transmission queues 1a, 1b and 1c in fig. 3).
The multi-channel multi-link file transmission message distribution service module is characterized in that after a message is taken out from a sending queue, a code is allocated to the message, the code is an unsigned integer value, the code is increased according to a step value of 1, and after the code reaches the maximum value, the code is restarted from 0, and the code is newly overlapped into the original message as a packet head to be cached and sent together.
The multi-channel multi-link file transmission message merging service module takes out file messages from a plurality of receiving sub-queues (a receiving queue 1a, a receiving queue 1b and a receiving queue 1c in fig. 3), obtains message codes from message packet heads, stores the message codes in the receiving queue (a first receiving queue in fig. 2) according to the message coding sequence after removing the packet heads overlapped by the sending nodes, and completely accords with the message content and sequence stored in the first sending queue through the processing of the multi-channel multi-link file transmission message merging service module.
The dual-channel double-link file receiving service module acquires file messages from the first receiving queue and the second receiving queue and reassembles the file messages into a file in the receiving catalog. The dual-channel double-link file receiving service module utilizes the ordered characteristic of the IROS message queue, and utilizes an algorithm similar to merging and sorting to quickly merge into a target file from the first receiving queue and the second receiving queue, and new buffering, message sorting and the like are not needed in the process, so that the performance is excellent.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
Embodiments of the system provided by the embodiments of the present invention belong to the same inventive concept as the method embodiments of the above embodiments, and reference may be made to the embodiments of the above method for details that are not described in detail in the embodiments of the system.
The system is the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein, and may be embodied in electronic hardware, computer software, or a combination of the two, with the elements and steps of the examples having been generally described in terms of functionality in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Those skilled in the art will appreciate that the various aspects of the methods may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
Although the present invention has been described in detail by way of preferred embodiments with reference to the accompanying drawings, the present invention is not limited thereto. Various equivalent modifications and substitutions may be made in the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and it is intended that all such modifications and substitutions be within the scope of the present invention/be within the scope of the present invention as defined by the appended claims.
Claims (10)
1. The multichannel multilink file transmission method is characterized by being applied to file transmission between a sending node and a receiving node, and the sending node and the receiving node are in multilink communication connection; the method comprises the following steps:
Creating two sending queues and N sending sub-queues at a sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node; n 2;
Detecting a file to be transmitted from a transmission catalog, and when the file to be transmitted is detected, converting the file to be transmitted into file messages and respectively storing the file messages into two transmission queues;
Randomly selecting one sending queue from the two queues as a first sending queue, taking out the file messages from the first sending queue, distributing a code for each file message as a packet head to be overlapped in the file message, and distributing the file message to N sending sub-queues; the remaining one of the two transmit queues is used as a second transmit queue;
establishing a channel to orderly transmit file messages in a second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; respectively establishing N channels to orderly transmit file messages in N transmission sub-queues of a transmitting node to N corresponding receiving sub-queues in a receiving node; wherein one channel uses one link; a receiving queue corresponding to the second sending queue in the two receiving queues is defined as a second receiving queue, and the other receiving queue is a first receiving queue;
And acquiring the file messages from the N receiving sub-queues, acquiring the codes of each file message, and storing the acquired file messages into the first receiving queue according to the coding sequence.
2. The multi-channel multi-link file transmission method according to claim 1, wherein the step of creating two transmit queues and N transmit sub-queues at the transmitting node and creating corresponding two receive queues and N receive sub-queues at the receiving node comprises:
Setting weights of the two transmission queues according to the sum of the available bandwidth of the link used by the second transmission queue and the available bandwidth of the link used by each transmission sub-queue;
The weight of each transmit sub-queue is set according to the available bandwidth of the link used by each transmit sub-queue.
3. The multi-channel, multi-link file transfer method of claim 2, wherein the step of detecting a file to be transmitted from the transmission directory, and converting the file to be transmitted into file messages to be stored in the two transmission queues, respectively, when the file to be transmitted is detected, comprises:
detecting a file to be transmitted from a transmission directory, and converting the file to be transmitted into a file message when the file to be transmitted is detected;
Dividing the file messages according to the quantity of the file messages based on the set weights, and storing the file messages in a first transmission queue and a second transmission queue respectively.
4. A multi-channel, multi-link file transfer method according to claim 3 wherein the step of randomly selecting a transmit queue from two queues as the first transmit queue, retrieving file messages from the first transmit queue, assigning a code to each file message as a header to be superimposed in the file message and distributing the file message to the N transmit sub-queues comprises:
Randomly selecting one transmission queue from the two queues as a first transmission queue, taking out the file messages from the first transmission queue, distributing a code for each file message as a packet header to be overlapped in the file message, and distributing the file message to N transmission sub-queues according to the weight set by the transmission sub-queues.
5. The multi-channel, multi-link file transfer method of any one of claims 1-4, further comprising: in the transmission process, when one of the links fails, transferring the channel established on the failed link to a normal link;
when the failed link is restored, the tunnel is re-established on the restored link.
6. The multi-channel, multi-link file transfer method of claim 5 wherein the steps of obtaining file messages in N receive sub-queues and obtaining a code for each file message, storing the obtained file messages in the first receive queue in the code order, comprise:
acquiring file messages from N receiving sub-queues;
And obtaining message codes from the file message packet heads, and storing the file messages with the packet heads overlapped by the sending nodes removed in a first receiving queue according to the message coding sequence, so that the content and the sequence of the file messages stored in the first receiving queue are completely consistent with those of the file messages stored in the first sending queue.
7. The multi-channel, multi-link file transfer method of claim 1 wherein the steps of obtaining file messages in N receive sub-queues and obtaining a code for each file message, storing the obtained file messages in the first receive queue in the code order, comprise:
File messages are obtained from the first receive queue and the second receive queue and reassembled into files in a receive directory for storage at a receiving node.
8. The multichannel multilink file transmission system is characterized by comprising a sending node and a receiving node, wherein the sending node is connected with the receiving node in a multilink manner; the sending node and the receiving node are respectively provided with a communication middleware with a message queue service function; the sending node is provided with a double-channel double-link file sending service module, a file transmission service module and a multi-channel multi-link file transmission message distribution service module; the receiving node is provided with a multichannel multilink file transmission message merging service module;
Creating two sending queues and N sending sub-queues at a sending node through a communication middleware of the sending node, and creating corresponding two receiving queues and N receiving sub-queues at a receiving node through a communication middleware of the receiving node; n 2;
The dual-channel double-link file sending service module detects files to be sent from a sending directory, and when the files to be sent are detected, the files to be sent are converted into file messages and are respectively stored in two sending queues;
The multi-channel multi-link file transmission message distribution service module randomly selects one transmission queue from the two queues as a first transmission queue, takes out file messages from the first transmission queue, distributes a code for each file message as a packet head to be overlapped in the file message and distributes the file message to N transmission sub-queues; the remaining one of the two transmit queues is used as a second transmit queue;
The file transmission service module is used for establishing a channel to orderly transmit the file information in the second transmission queue of the transmitting node to a receiving queue corresponding to the second transmission queue in the receiving node; the method is also used for respectively establishing N channels to orderly transmit the file messages in N transmission sub-queues of the transmitting node to N corresponding receiving sub-queues in the receiving node; wherein one channel uses one link; a receiving queue corresponding to the second sending queue in the two receiving queues is defined as a second receiving queue, and the other receiving queue is a first receiving queue;
The multi-channel multi-link file transmission message merging service module is used for acquiring file messages in N receiving sub-queues and acquiring codes of each file message, and storing the acquired file messages into a first receiving queue according to the coding sequence.
9. The multi-channel, multi-link file transfer system of claim 8 wherein the receiving node is further provided with a dual-channel, dual-link file receiving service module;
the dual-channel double-link file receiving service module acquires file messages from the first receiving queue and the second receiving queue and reassembles the file messages into a file in the receiving catalog.
10. The multi-channel, multi-link file transfer system of claim 8 or 9 further comprising a weight setting module that sets weights for the two transmit queues based on a sum of the available bandwidth of the link used by the second transmit queue and the available bandwidth of the link used by each transmit sub-queue; setting the weight of each transmission sub-queue according to the available bandwidth of the link used by each transmission sub-queue;
The double-channel double-link file sending service module converts a file to be sent into a file message when detecting the file to be sent; dividing the file message according to the quantity of the file message based on the set weight, and respectively storing the file message into a first transmission queue and a second transmission queue;
The multi-channel multi-link file transmission message distribution service module takes out the file messages from the first transmission queue, distributes a code for each file message as a packet header to be overlapped in the file message, and distributes the file message to N transmission sub-queues according to the weight set by the transmission sub-queues.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410656956.0A CN118433174A (en) | 2024-05-24 | 2024-05-24 | Multichannel multilink file transmission method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410656956.0A CN118433174A (en) | 2024-05-24 | 2024-05-24 | Multichannel multilink file transmission method and system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN118433174A true CN118433174A (en) | 2024-08-02 |
Family
ID=92337076
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202410656956.0A Pending CN118433174A (en) | 2024-05-24 | 2024-05-24 | Multichannel multilink file transmission method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN118433174A (en) |
-
2024
- 2024-05-24 CN CN202410656956.0A patent/CN118433174A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR100334788B1 (en) | Method and apparatus for connecting a node to a wireless network using standard protocols | |
US6853617B2 (en) | System and method for TCP connection protection switching | |
US6954797B1 (en) | Data Communication method, terminal equipment, interconnecting installation, data communication system and recording medium | |
EP1175064B1 (en) | Method and system for improving network performance using a performance enhancing proxy | |
JP5286346B2 (en) | Process control system and method for communicating application information | |
US20150222444A1 (en) | System and method for reliable multicast data transport | |
US7649909B1 (en) | Adaptive tunnel transport protocol | |
US11303737B2 (en) | Method and device for data transmission | |
JPWO2007063585A1 (en) | Communication apparatus and frame control method | |
US6947435B1 (en) | Radio communication system and apparatus, communication method and program recording medium therefor | |
WO2008020428A2 (en) | Method and system for saving bandwidth during broadcasting/multicasting | |
US20070291782A1 (en) | Acknowledgement filtering | |
US20060259845A1 (en) | Method and apparatus for acknowledging a bitwise data chunk in wireline and wireless communication systems | |
CN117176594A (en) | High-speed data transmission system and transmission method | |
JP2000156706A (en) | Data transmission reception method, medium with data transmission program stored therein and medium data reception program stored therein | |
Koch et al. | Transparent TCP Connection Failover. | |
KR20040074434A (en) | method and apparatus for reliable routing information exchange in distributed router | |
JP4656415B2 (en) | COMMUNICATION DEVICE, COMMUNICATION METHOD, AND PROGRAM | |
CN105763375B (en) | A kind of data packet sending method, method of reseptance and microwave station | |
JP4229807B2 (en) | Data transfer method, TCP proxy device, and network system using the same | |
CN118433174A (en) | Multichannel multilink file transmission method and system | |
JP2008113327A (en) | Network interface device | |
CN116233243A (en) | Communication system and method in weak network environment | |
JPH11177621A (en) | Data transmission equipment and data communication system | |
US20050086383A1 (en) | Optimizing the compression efficiency in a packet data communication |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |