CN115277497B - Transmission delay time measurement method, device, electronic equipment and storage medium - Google Patents
Transmission delay time measurement method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115277497B CN115277497B CN202210716495.2A CN202210716495A CN115277497B CN 115277497 B CN115277497 B CN 115277497B CN 202210716495 A CN202210716495 A CN 202210716495A CN 115277497 B CN115277497 B CN 115277497B
- Authority
- CN
- China
- Prior art keywords
- time interval
- data packet
- historical
- transmission delay
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 179
- 238000000691 measurement method Methods 0.000 title claims description 18
- 238000000034 method Methods 0.000 claims abstract description 93
- 238000012549 training Methods 0.000 claims abstract description 58
- 239000000872 buffer Substances 0.000 claims description 70
- 238000012360 testing method Methods 0.000 claims description 52
- 230000006854 communication Effects 0.000 claims description 36
- 238000004891 communication Methods 0.000 claims description 35
- 230000008569 process Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 15
- 230000015654 memory Effects 0.000 claims description 12
- 238000005538 encapsulation Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 4
- 238000005259 measurement Methods 0.000 abstract description 28
- 238000013528 artificial neural network Methods 0.000 abstract description 5
- 125000004122 cyclic group Chemical group 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 3
- 230000003139 buffering effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
技术领域technical field
本发明涉及网络通信技术领域,尤其涉及一种传输延迟时间测量方法、装置、电子设备及存储介质。The invention relates to the technical field of network communication, in particular to a transmission delay time measuring method, device, electronic equipment and storage medium.
背景技术Background technique
传输延迟作为视频监控系统中一项重要技术指标,通常表明端到端传输过程中存在的编码延时、传输延时及解码显示延时,其不仅影响用户体验,也影响通信质量;比如用户使用终端查看视频时存在传输延迟,则会影响用户的主观体验,在闭环视频监控过程中若存在传输延迟,则会影响视频信息的实时性,影响判断分析和决策结果。因此,如何准确测量传输延迟就显得尤为重要。As an important technical indicator in video surveillance system, transmission delay usually indicates the encoding delay, transmission delay and decoding display delay in the end-to-end transmission process, which not only affects the user experience, but also affects the communication quality; for example, users use If there is a transmission delay when the terminal views the video, it will affect the user's subjective experience. If there is a transmission delay in the closed-loop video monitoring process, it will affect the real-time performance of the video information, and affect the judgment analysis and decision-making results. Therefore, how to accurately measure transmission delay is particularly important.
相关技术中,通常在实验室环境下以直连方式实现对监控系统的延迟时间进行测量,也即将发送端和客户端直接连接,通过计算同一帧视频图像的时间差值计算传输延迟时间。In related technologies, the delay time of the monitoring system is usually measured in a direct connection mode in a laboratory environment, that is, the sending end and the client are directly connected, and the transmission delay time is calculated by calculating the time difference of the same frame of video images.
然而,由于实验室环境较为简单,并不适用于实际应用环境,同时测量过程中要求发送端和客户端具有相同时间基准,从而导致测量传输延迟时间的适用性和精确性不高。However, because the laboratory environment is relatively simple, it is not suitable for the actual application environment. At the same time, the transmitter and the client are required to have the same time reference during the measurement process, resulting in low applicability and accuracy of measuring transmission delay time.
发明内容Contents of the invention
本发明提供一种传输延迟时间测量方法、装置、电子设备及存储介质,用以解决现有技术中测量传输延迟时需要同步发送端和客户端所导致的测量传输延迟时间的适用性和精确性不高的缺陷,无需发送端和客户端时间同步也能在实际应用环境中实现传输延迟测量的适用性和精确性。The present invention provides a method, device, electronic device and storage medium for measuring transmission delay time, which are used to solve the applicability and accuracy of measuring transmission delay time caused by the need to synchronize the sending end and the client when measuring transmission delay in the prior art The defect is not high, and the applicability and accuracy of transmission delay measurement can be realized in the actual application environment without time synchronization between the sender and the client.
本发明提供一种传输延迟时间测量方法,包括:The present invention provides a method for measuring transmission delay time, comprising:
获取服务器逐帧传输的历史数据包所对应的历史时间间隔;Obtain the historical time interval corresponding to the historical data packets transmitted frame by frame by the server;
将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔;其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型;The historical time interval is input into a preset delay prediction model to obtain a prediction time interval corresponding to the current data packet transmitted frame by frame; wherein, the preset delay prediction model is obtained by training an LSTM model based on a sample historical time interval Model;
基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Outputting a transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
根据本发明提供的一种传输延迟时间测量方法,所述基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间,包括:According to a method for measuring transmission delay time provided by the present invention, the output transmission delay time is based on the predicted time interval, the actual time interval corresponding to the current data packet and the data packet transmission time interval of the sending end, including:
将所述预测时间间隔与所述当前数据包对应的实际时间间隔进行大小比较;comparing the predicted time interval with the actual time interval corresponding to the current data packet;
确定所述实际时间间隔大于所述预测时间间隔时,基于所述实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。When it is determined that the actual time interval is greater than the predicted time interval, outputting a transmission delay time based on the actual time interval and the data packet sending time interval of the sending end.
根据本发明提供的一种传输延迟时间测量方法,所述获取服务器逐帧传输的历史数据包所对应的历史时间间隔,包括:According to a method for measuring transmission delay time provided by the present invention, the acquisition of the historical time interval corresponding to the historical data packets transmitted frame by frame by the server includes:
接收服务器逐帧传输的历史数据包;Receive historical data packets transmitted by the server frame by frame;
基于预设的数据包接收协议,对所述历史数据包进行过滤;Filtering the historical data packets based on a preset data packet receiving protocol;
按照预设文件格式,将过滤后的剩余历史数据包存储为数据包文件;According to the preset file format, store the filtered remaining historical data packets as data packet files;
解析所述数据包文件中的时间戳信息并进行时间格式转化,得到所述剩余历史数据包所对应的历史时间间隔。Analyzing the timestamp information in the data packet file and performing time format conversion to obtain the historical time interval corresponding to the remaining historical data packets.
根据本发明提供的一种传输延迟时间测量方法,所述基于预设的数据包接收协议,对所述历史数据包进行过滤,包括:According to a method for measuring transmission delay time provided by the present invention, the filtering of the historical data packets based on the preset data packet receiving protocol includes:
将所述历史数据包依次写入接收缓冲区;Writing the historical data packets into the receiving buffer in sequence;
基于预设的数据包通信协议,从所述接收缓冲区中的所述历史数据包中筛选出第一历史数据包;Filter out a first historical data packet from the historical data packets in the receiving buffer based on a preset data packet communication protocol;
基于预设的数据包标识信息,将所述第一历史数据包中与所述数据包标识信息不匹配的数据包进行过滤。Based on the preset data packet identification information, the data packets in the first historical data packets that do not match the data packet identification information are filtered.
根据本发明提供的一种传输延迟时间测量方法,在所述将所述第一历史数据包中与所述数据包标识信息不匹配的数据包进行过滤之后,所述方法还包括:According to a method for measuring transmission delay time provided by the present invention, after filtering the data packets that do not match the data packet identification information in the first historical data packets, the method further includes:
将过滤后所得的第二历史数据包依次写入播放缓冲区;Writing the filtered second historical data packets into the playback buffer in sequence;
指示所述播放缓冲区对所述第二历史数据包进行整序后播放。Instructing the playing buffer to sort the second historical data packets and then play them.
根据本发明提供的一种传输延迟时间测量方法,在所述将所述历史数据包依次写入接收缓冲区之后,所述方法还包括:According to a method for measuring transmission delay time provided by the present invention, after the historical data packets are sequentially written into the receiving buffer, the method further includes:
获取所述历史数据包写入所述接收缓冲区的缓冲区容量;Obtain the buffer capacity of the historical data packet written into the receiving buffer;
若确定所述缓冲区容量大于预设容量最大值,则指示发送端降低数据包发送速率;If it is determined that the buffer capacity is greater than the maximum preset capacity, instruct the sender to reduce the data packet sending rate;
若确定所述缓冲区容量小于预设容量最小值,则指示所述发送端增加数据包发送速率。If it is determined that the buffer capacity is smaller than the preset minimum capacity, instruct the sending end to increase the data packet sending rate.
根据本发明提供的一种传输延迟时间测量方法,在所述获取服务器逐帧传输的历史数据包所对应的历史时间间隔的步骤之前,所述方法还包括:According to a method for measuring transmission delay time provided by the present invention, before the step of obtaining the historical time interval corresponding to the historical data packets transmitted frame by frame by the server, the method further includes:
建立与发送端之间的数据通信链路;Establish a data communication link with the sender;
向所述发送端发送数据传输请求,所述数据传输请求携带数据封装要求和数据发送方式;Sending a data transmission request to the sender, where the data transmission request carries a data encapsulation requirement and a data transmission method;
接收所述发送端针对所述数据传输请求发送的数据包。receiving the data packet sent by the sending end in response to the data transmission request.
根据本发明提供的一种传输延迟时间测量方法,所述预设延迟预测模型的训练过程包括:According to a method for measuring transmission delay time provided by the present invention, the training process of the preset delay prediction model includes:
基于样本历史时间间隔,确定训练集和测试集;Based on the sample history time interval, determine the training set and test set;
基于所述训练集对LSTM模型进行训练,得到训练后的LSTM模型;The LSTM model is trained based on the training set to obtain the trained LSTM model;
基于所述测试集对所述训练后的LSTM模型进行测试,得到测试结果;Testing the trained LSTM model based on the test set to obtain a test result;
基于所述测试结果与所述测试集之间的误差,确定所述训练后的LSTM模型为预设延迟预测模型。Based on the error between the test result and the test set, it is determined that the trained LSTM model is a preset delay prediction model.
根据本发明提供的一种传输延迟时间测量方法,所述方法还包括:According to a method for measuring transmission delay time provided by the present invention, the method further includes:
基于预设时长内接收到的新的历史时间间隔,对所述训练集和所述测试集分别进行更新。The training set and the test set are respectively updated based on new historical time intervals received within a preset time period.
本发明还提供一种传输延迟时间测量装置,包括:The present invention also provides a transmission delay time measuring device, comprising:
获取模块,用于获取服务器逐帧传输的历史数据包所对应的历史时间间隔;The obtaining module is used to obtain the historical time interval corresponding to the historical data packets transmitted frame by frame by the server;
预测模块,用于将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔;其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型;A prediction module, configured to input the historical time interval into a preset delay prediction model to obtain a prediction time interval corresponding to the current data packet transmitted frame by frame; wherein, the preset delay prediction model is based on a sample historical time interval The model obtained by training the LSTM model;
输出模块,用于基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。An output module, configured to output the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
本发明还提供一种电子设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,所述处理器执行所述程序时实现如上述任一种所述传输延迟时间测量方法。The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, the transmission delay time as described in any of the above is realized. Measurement methods.
本发明还提供一种非暂态计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现如上述任一种所述传输延迟时间测量方法。The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for measuring transmission delay time as described in any one of the above-mentioned methods is implemented.
本发明还提供一种计算机程序产品,包括计算机程序,所述计算机程序被处理器执行时实现如上述任一种所述传输延迟时间测量方法。The present invention also provides a computer program product, including a computer program, when the computer program is executed by a processor, the method for measuring transmission delay time as described in any one of the above methods is implemented.
本发明提供的传输延迟时间测量方法、装置、电子设备及存储介质,其中传输延迟时间测量方法,通过将服务器逐帧传输的历史数据包所对应的历史时间间隔输入至预设延迟预测模型的方式,预测逐帧传输的当前数据包对应的预测时间间隔,并基于该预测时间间隔、当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。以此结合逐帧传输方式降低数据发送延时,实现兼顾整个传输系统的实时性和带宽利用率的目的,无需发送端和客户端时间同步,即可在实际应用环境中实现传输延迟测量,提高了传输延迟测量的适用性;进一步的,由于预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型,以此能够结合训练循环神经网络的方式,实现提高预测时间间隔的精准度的目的,从而也能有效提高传输延迟测量的精确性和可靠性。The transmission delay time measurement method, device, electronic equipment, and storage medium provided by the present invention, wherein the transmission delay time measurement method is by inputting the historical time interval corresponding to the historical data packets transmitted by the server frame by frame into the preset delay prediction model , predict the predicted time interval corresponding to the current data packet transmitted frame by frame, and output the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the sending time interval of the data packet at the sending end. In this way, combined with the frame-by-frame transmission method, the data transmission delay is reduced, and the purpose of taking into account the real-time and bandwidth utilization of the entire transmission system is realized. The transmission delay measurement can be realized in the actual application environment without time synchronization between the sender and the client, and the improvement can be improved. Furthermore, since the preset delay prediction model is based on the model obtained by training the LSTM model based on the sample historical time interval, it can be combined with the training of the cyclic neural network to realize the improvement of the accuracy of the prediction time interval In order to effectively improve the accuracy and reliability of transmission delay measurement.
附图说明Description of drawings
为了更清楚地说明本发明或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍,显而易见地,下面描述中的附图是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are the present invention. For some embodiments of the invention, those skilled in the art can also obtain other drawings based on these drawings without creative effort.
图1是本发明提供的传输延迟时间测量方法的流程示意图;Fig. 1 is a schematic flow chart of the method for measuring transmission delay time provided by the present invention;
图2是本发明提供的传输延迟测量系统的结构示意图;Fig. 2 is a schematic structural diagram of a transmission delay measurement system provided by the present invention;
图3是本发明提供的传输延迟时间测量方法的总体流程图;Fig. 3 is the overall flowchart of the method for measuring transmission delay time provided by the present invention;
图4是本发明提供的传输延迟测量装置的结构示意图;4 is a schematic structural diagram of a transmission delay measurement device provided by the present invention;
图5是本发明提供的电子设备的结构示意图。Fig. 5 is a schematic structural diagram of an electronic device provided by the present invention.
具体实施方式Detailed ways
为使本发明的目的、技术方案和优点更加清楚,下面将结合本发明中的附图,对本发明中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention , but not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
传输延迟作为视频监控系统中一项重要技术指标,通常表明端到端传输过程中存在的编码延时、传输延时及解码显示延时,其不仅影响用户体验,也影响通信质量;比如大型监控系统基于摄像机、服务器和客户端,无时无刻不存在着大量的视频调用和控制,并在视频调用过程中需要服务器和摄像机做出反应且将视频流传输至客户端,从客户端发出数据传输请求到服务器和摄像机做出响应,再到视频流到达指定客户端,需要经过大量的交换机和路由器,不可避免的导致传输延时。此外,视频流传输过程中存在摄像机和客户端时钟不同步的情况,这也给延迟时间的测量造成了不便和困难。As an important technical indicator in video surveillance systems, transmission delay usually indicates the encoding delay, transmission delay and decoding display delay in the end-to-end transmission process, which not only affects user experience, but also affects communication quality; such as large-scale surveillance The system is based on cameras, servers and clients, and there are a lot of video calls and controls all the time. During the video call process, the server and cameras need to respond and transmit the video stream to the client, and send data transmission requests from the client to the The server and the camera respond, and then the video stream reaches the designated client, it needs to go through a large number of switches and routers, which inevitably leads to transmission delay. In addition, during the video streaming process, the clocks of the camera and the client are not synchronized, which also brings inconvenience and difficulty to the measurement of the delay time.
因此,为了提高视频监控系统的可靠性和安全性,通常在网络设计中部署备份线路,当主线路意外中断或者出现故障时自动使用备份线路,切换数据流,切换数据流的过程也即通信链路的切换过程,在这一切换过程中,如果客户端在切换前后经由不同的通信链路连接,则服务器与客户端之间的数据通信也必然会出现中断问题,通信链路的切换可能导致一定时间内的通信中断。对于通信质量要求不高的系统,短时间的通信中断是可以接受的,若视频监控系统对通信的可靠性和实时性要求较高且严格限制系统通信的中断时间,则由于通信的突然中断而导致实时信息无法实时传送,将进一步导致难以发现并解决故障或突发事件,甚至严重时有可能会造成安全事故。因此,视频监控系统的传输延迟时间测量与计算具有重要的意义和价值。Therefore, in order to improve the reliability and security of the video surveillance system, backup lines are usually deployed in the network design. When the main line is unexpectedly interrupted or fails, the backup line is automatically used to switch data streams. The process of switching data streams is also a communication link. In this switching process, if the client is connected via different communication links before and after the switching, the data communication between the server and the client will inevitably be interrupted, and the switching of the communication link may cause certain Communication is interrupted for a period of time. For systems that do not require high communication quality, short-term communication interruptions are acceptable. If the video surveillance system has high requirements for communication reliability and real-time performance and strictly limits the interruption time of system communication, the sudden interruption of communication As a result, real-time information cannot be transmitted in real time, which will further make it difficult to find and solve faults or emergencies, and may even cause safety accidents in severe cases. Therefore, the measurement and calculation of the transmission delay time of the video surveillance system is of great significance and value.
现有的实验室环境下传输延迟时间测量方法是以直连方式实现对监控系统的延迟时间进行测量,也即将发送端和客户端直接相连,通过计算同一视频帧图像的时间差值计算传输延迟时间,测试场景较为简单,并未考虑到实际应用过程中路由分布、网络拥塞、通信干扰等均会影响视频流延时,使得在实验室环境这一理想环境下测出的延迟时间往往显著低于实际应用场景。同时测量过程中还要求发送端(比如摄像头)和客户端具有共同的时间基准,但实际很多视频监控系统并没有共同的时间基准信号,无法同步发送端和客户端。因此,更加便捷且更加精准的延迟时间测量显得尤为重要。The existing transmission delay time measurement method in the laboratory environment is to measure the delay time of the monitoring system in a direct connection mode, that is, to directly connect the sender and the client, and calculate the transmission delay by calculating the time difference of the same video frame image Time, the test scene is relatively simple, and it does not take into account that routing distribution, network congestion, communication interference, etc. will affect the video stream delay in the actual application process, so the delay time measured in the ideal environment of the laboratory environment is often significantly lower in practical application scenarios. At the same time, the measurement process also requires the sender (such as a camera) and the client to have a common time reference. However, many video surveillance systems do not have a common time reference signal and cannot synchronize the sender and client. Therefore, more convenient and more accurate delay time measurement is particularly important.
基于此,本发明提供一种传输延迟时间测量方法、装置、电子设备及存储介质,其中传输延迟时间测量方法可以应用于由多个发送端、服务器和多个客户端形成的传输延迟时间测量系统的场景中,传输延迟时间测量方法的执行主体为传输延迟时间测量系统中任意一个客户端,该客户端可以为个人计算机(Personal Computer,PC)、便携式设备、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备等其它电子设备,且该客户端至少具备数据包接收播放功能、数据包抓取功能、存储功能、解析功能及统计分析功能。本发明对客户端的具体形式不做限定。Based on this, the present invention provides a transmission delay time measurement method, device, electronic equipment and storage medium, wherein the transmission delay time measurement method can be applied to a transmission delay time measurement system formed by multiple senders, servers and multiple clients In the scenario of , the execution subject of the transmission delay time measurement method is any client in the transmission delay time measurement system, and the client can be a personal computer (Personal Computer, PC), a portable device, a notebook computer, a smart phone, a tablet computer and Portable wearable devices and other electronic devices, and the client has at least functions of receiving and playing data packets, capturing data packets, storing, analyzing, and statistically analyzing. The present invention does not limit the specific form of the client.
需要说明的是,下述方法实施例的执行主体可以是传输延迟时间测量系统中任意一个客户端的部分或者全部。下述方法实施例以执行主体为客户端为例进行说明。It should be noted that the subject of execution of the following method embodiments may be part or all of any client in the transmission delay time measurement system. The following method embodiments are described by taking the execution subject as a client as an example.
图1为本发明提供的传输延迟时间测量方法,如图1所示,该传输延迟时间测量方法,包括以下步骤:Fig. 1 is the transmission delay time measurement method provided by the present invention, as shown in Fig. 1, this transmission delay time measurement method comprises the following steps:
步骤110、获取服务器逐帧传输的历史数据包所对应的历史时间间隔。Step 110, acquiring the historical time interval corresponding to the historical data packets transmitted frame by frame by the server.
其中,历史数据包可以为预设时长T内客户端已接收到的各个数据包,T=Ft,F为大于1的整数,t为数据包发送时间间隔,数据包发送时间间隔t可以基于以太网所能支持的最大吞吐量设置,以便于判定预设时长T内是否发生丢包及发生丢包时丢包率。Wherein, the historical data packet can be each data packet that the client has received within the preset time length T, T=Ft, F is an integer greater than 1, t is the data packet sending time interval, and the data packet sending time interval t can be based on the Ethernet The maximum throughput setting that the network can support is used to determine whether packet loss occurs within the preset time period T and the packet loss rate when packet loss occurs.
具体的,在如图2所示的传输延迟时间测量系统中,服务器可以为高并发流媒体服务器且具备中转作用,其主要任务是对所接收到的数据包进行传输、调度分发和缓冲存储等,当传输延迟时间测量系统应用于视频监控系统中时,发送端可以为一个或多个摄像头且发送端可以与服务器采用有线连接,发送端为多个时,可以如图2所示的发送端1、发送端2和发送端3;客户端通常为多个且每个客户端均通过互联网与服务器连接,比如图2中的客户端1、客户端2和客户端3;当传输延时时间测量系统应用于通信系统中时,发送端可以为与客户端相同的终端设备且发送端可以与服务器无线连接。此处不作具体限定。Specifically, in the transmission delay measurement system shown in Figure 2, the server can be a high-concurrency streaming media server with a relay function, and its main task is to transmit, schedule, distribute, and buffer the received data packets, etc. , when the transmission delay time measurement system is applied to a video surveillance system, the sending end can be one or more cameras and the sending end can be connected to the server by wire. When there are multiple sending ends, the sending end as shown in Figure 2 can be used 1. Sending end 2 and sending end 3; there are usually multiple clients and each client is connected to the server through the Internet, such as client 1, client 2 and client 3 in Figure 2; when the transmission delay time When the measurement system is applied to a communication system, the sending end may be the same terminal device as the client and the sending end may be wirelessly connected to the server. No specific limitation is made here.
基于此,客户端可以通过服务器接收发送端传输的数据包,当服务器逐帧向客户端发送数据包时,客户端也可以逐帧接收数据包。因此,客户端每接收到一个数据包,则对应解析其到达时间,并基于每相邻两个数据包到达客户端的时间之差,确定每相邻两个数据包所对应的时间间隔,从而确定出服务器逐帧传输的历史数据包所对应的历史时间间隔。Based on this, the client can receive the data packets transmitted by the sender through the server, and when the server sends data packets to the client frame by frame, the client can also receive data packets frame by frame. Therefore, each time the client receives a data packet, it analyzes its arrival time correspondingly, and determines the time interval corresponding to each adjacent two data packets based on the difference between the arrival time of each adjacent two data packets to the client, thereby determining The historical time interval corresponding to the historical data packets transmitted by the outgoing server frame by frame.
步骤120、将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔。Step 120: Input the historical time interval into a preset delay prediction model to obtain the predicted time interval corresponding to the current data packet transmitted frame by frame.
其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型,当前数据包为正在传输中的数据包。Wherein, the preset delay prediction model is a model obtained by training an LSTM model based on sample historical time intervals, and the current data packet is a data packet being transmitted.
具体的,客户端将所确定的历史时间间隔输入至预设延迟预测模型,预测服务器逐帧传输的当前数据包对应的预测时间间隔,比如,可以将n个历史数据包对应的n-1个历史时间间隔输入至预设延迟预测模型中,预测第n个历史数据包和正在传输的第n+1个当前数据包所对应的第n个时间间隔,即为第n个预测时间间隔;第n+1个当前数据包到达客户端的时间与第n个历史数据包到达客户端的时间之差,为第n+1个当前数据对应的第n个实际时间间隔,n为大于1的整数。Specifically, the client inputs the determined historical time interval into the preset delay prediction model to predict the predicted time interval corresponding to the current data packet transmitted by the server frame by frame. For example, n-1 data packets corresponding to n historical data packets can be The historical time interval is input into the preset delay prediction model, and the nth time interval corresponding to the nth historical data packet and the n+1th current data packet being transmitted is predicted, which is the nth predicted time interval; The difference between the time when the n+1 current data packet arrives at the client and the time when the nth historical data packet arrives at the client is the nth actual time interval corresponding to the n+1th current data packet, and n is an integer greater than 1.
需要说明的是,考虑到长短期记忆模型(Long-Short Term Memory,LSTM)是一种循环神经网络且具有长短期记忆能量,适用于较长时间序列的预测且能解决梯度消失问题。因此,本发明通过采用样本历史时间间隔对LSTM模型训练所得到的预设延迟预测模型,能够提高预测时间间隔的准确性和精度。It should be noted that considering that the Long-Short Term Memory (LSTM) model is a recurrent neural network with long-term and short-term memory energy, it is suitable for the prediction of longer time series and can solve the problem of gradient disappearance. Therefore, the present invention can improve the accuracy and precision of the prediction time interval by using the preset delay prediction model obtained by training the LSTM model with the sample historical time interval.
步骤130、基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Step 130: Output the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
具体的,基于预测的当前数据包到达客户端的预测时间间隔和该当前数据包所对应到达客户端的实际时间间隔,客户端可以先判断是否发生传输延迟,如果基于预测时间间隔和实际时间间隔判定未发生传输延迟,则输出传输延迟时间为0;如果基于预测时间间隔和实际时间间隔判定发生传输延迟,可以再进一步结合发送端的数据包发送时间间隔输出传输延迟时间。基于此,本发明可以适用于有线和无线视频监控系统中传输延迟时间的测量,也可以用于通信系统中链路中断、断网恢复时间测量以及传输异常的判断。Specifically, based on the predicted time interval for the current data packet to arrive at the client and the actual time interval for the current data packet to arrive at the client, the client can first determine whether transmission delay occurs. If a transmission delay occurs, the output transmission delay time is 0; if it is determined that a transmission delay occurs based on the predicted time interval and the actual time interval, the transmission delay time can be further combined with the data packet transmission time interval of the sender. Based on this, the present invention can be applied to the measurement of transmission delay time in wired and wireless video surveillance systems, and can also be used in communication systems for link interruption, network disconnection recovery time measurement and transmission abnormality judgment.
本发明提供的传输延迟时间测量方法,通过将服务器逐帧传输的历史数据包所对应的历史时间间隔输入至预设延迟预测模型的方式,预测逐帧传输的当前数据包对应的预测时间间隔,并基于该预测时间间隔、当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。以此结合逐帧传输方式降低数据发送延时,实现兼顾整个传输系统的实时性和带宽利用率的目的,无需发送端和客户端时间同步,即可在实际应用环境中实现传输延迟测量,提高了传输延迟测量的适用性;进一步的,由于预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型,以此能够结合训练循环神经网络的方式,实现提高预测时间间隔的精准度的目的,从而也能有效提高传输延迟测量的精确性和可靠性。The transmission delay time measurement method provided by the present invention predicts the prediction time interval corresponding to the current data packet transmitted frame by frame by inputting the historical time interval corresponding to the historical data packet transmitted frame by frame by the server into the preset delay prediction model, And output the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval at the sending end. In this way, combined with the frame-by-frame transmission method, the data transmission delay is reduced, and the purpose of taking into account the real-time and bandwidth utilization of the entire transmission system is realized. The transmission delay measurement can be realized in the actual application environment without time synchronization between the sender and the client, and the improvement can be improved. Furthermore, since the preset delay prediction model is based on the model obtained by training the LSTM model based on the sample historical time interval, it can be combined with the training of the cyclic neural network to realize the improvement of the accuracy of the prediction time interval In order to effectively improve the accuracy and reliability of transmission delay measurement.
可选的,在步骤110之前,所述方法还可以包括:Optionally, before step 110, the method may also include:
首先,建立与发送端之间的数据通信链路;然后,向所述发送端发送数据传输请求,所述数据传输请求携带数据封装要求和数据发送方式;再进一步接收所述发送端针对所述数据传输请求发送的数据包。First, establish a data communication link with the sending end; then, send a data transmission request to the sending end, the data transmission request carrying data encapsulation requirements and data transmission mode; then further receive the sending end for the The data packet sent by the data transfer request.
具体的,在图2所示的传输延迟时间测量系统中,每个发送端与服务器之间可以采用有线连接,有线以太网的带宽高且信号稳定,并且测量传输延迟时间时,发送端以以太网所能支持的最大吞吐量设置合适的数据包发送时间间隔和数据包大小;如果数据包发送速率超过最大吞吐量,则会造成网络延迟,并且传输延迟与数据包长度也密切相关,数据包长度不能超过网络最大传输单元,数据包长度过小也容易出现数据传输排队或丢包问题,导致延迟问题更加突出。因此,当客户端与发送端之间建立通信连接且向发送端发送数据传输请求时,发送端可以采用无应答模式且针对数据传输请求携带数据的封装要求和数据发送方式封装数据包并向客户端发送。也即,发送端使用FFmpeg框架对发送端所需的数据进行采集、解码、编码和压缩等处理,比如采用H.264编码算法将所采集的数据压缩编码为网络传输特性较好的数据包格式,再经由FLV221封装规范将压缩编码后的数据封装为一个个数据包,最后采用用户数据报协议(User Datagram Protocol,UDP)将封装后所得的数据包逐帧发送至服务器,以此结合FFmpeg框架具备的高移植性、实现灵活性和数据处理高效性,确定数据的采集频率和图像帧数,从而可以通过采用实时传输控制协议(Real-timeTransport Control Protocol,RTCP)在数据源头控制流媒体质量和码率,实现了数据延迟可控的目的。Specifically, in the transmission delay time measurement system shown in Figure 2, a wired connection can be used between each sending end and the server. The bandwidth of the wired Ethernet is high and the signal is stable. When measuring the transmission delay time, the sending end uses Ethernet The maximum throughput that the network can support sets the appropriate data packet sending time interval and data packet size; if the data packet sending rate exceeds the maximum throughput, it will cause network delay, and the transmission delay is also closely related to the length of the data packet. The length cannot exceed the maximum transmission unit of the network. If the data packet length is too small, data transmission queuing or packet loss may easily occur, resulting in more prominent delay problems. Therefore, when a communication connection is established between the client and the sender and a data transmission request is sent to the sender, the sender can adopt the non-response mode and encapsulate the data packet according to the encapsulation requirements and data transmission method of the data transmission request and send it to the client. end to send. That is to say, the sender uses the FFmpeg framework to collect, decode, encode, and compress the data required by the sender. For example, the H.264 encoding algorithm is used to compress and encode the collected data into a packet format with better network transmission characteristics. , and then encapsulate the compressed and encoded data into individual data packets through the FLV221 encapsulation specification, and finally use the User Datagram Protocol (User Datagram Protocol, UDP) to send the encapsulated data packets to the server frame by frame, so as to combine the FFmpeg framework With high portability, flexibility and high efficiency of data processing, the frequency of data collection and the number of image frames can be determined, so that the quality and quality of streaming media can be controlled at the source of data by using Real-time Transport Control Protocol (RTCP). The code rate realizes the purpose of controllable data delay.
在图2中,发送端发送的数据包先逐帧传输至服务器,再经由服务器缓冲后逐帧转发时客户端;客户端与服务器之间可以一对一通信,也可以多对一通信,以此结合服务器缓冲方式降低传输延迟风险。In Figure 2, the data packets sent by the sender are first transmitted to the server frame by frame, and then buffered by the server and then forwarded to the client frame by frame; the client and the server can communicate one-to-one or many-to-one, so that This combined with server buffering reduces the risk of transmission delays.
本发明提供的传输延迟时间测量方法,通过客户端向已建立通信连接的发送端发送携带数据封装要求和数据发送方式的数据传输请求的方式,确保客户端接收满足网络最大吞吐量及不超过网络最大传输单元带宽的数据包,实现了兼顾数据包传输实时性和流畅性的目的。The method for measuring the transmission delay time provided by the present invention ensures that the client receives the maximum throughput of the network and does not exceed the maximum throughput of the network through the method that the client sends a data transmission request carrying data encapsulation requirements and a data transmission method to the sender that has established a communication connection. The data packet with the maximum transmission unit bandwidth realizes the purpose of taking into account the real-time and fluency of data packet transmission.
可选的,步骤110的实现过程可以包括:Optionally, the implementation process of step 110 may include:
首先,接收服务器逐帧传输的历史数据包;然后,基于预设的数据包接收协议,对所述历史数据包进行过滤;进一步按照预设文件格式,将过滤后的剩余历史数据包存储为数据包文件;最后,解析所述数据包文件中的时间戳信息并进行时间格式转化,得到所述剩余历史数据包所对应的历史时间间隔。First, receive the historical data packets transmitted by the server frame by frame; then, based on the preset data packet receiving protocol, filter the historical data packets; further store the filtered remaining historical data packets as data according to the preset file format package file; finally, analyze the time stamp information in the data package file and perform time format conversion to obtain the historical time interval corresponding to the remaining historical data packages.
具体的,由于历史数据包为已接收到的各个数据包,比如服务器接收到的各个数据包。因此,当服务器接收到发送端发送的数据包时,可以逐帧向客户端发送,客户端也可以逐帧接收,并且,针对逐帧接收的数据包,客户端基于预设的数据包接收协议,将所接收到的数据包中与数据包接收协议不匹配的数据包过滤,以此得到过滤后的剩余历史数据包,预设的数据包接收协议可以用于表征客户端所需数据包的接收指示。Specifically, since the historical data packets are each data packet that has been received, such as each data packet received by the server. Therefore, when the server receives the data packet sent by the sender, it can send it to the client frame by frame, and the client can also receive it frame by frame, and, for the data packet received frame by frame, the client receives the packet based on the preset protocol , filter the data packets that do not match the data packet receiving protocol in the received data packets, so as to obtain the remaining filtered historical data packets, the preset data packet receiving protocol can be used to characterize the data packets required by the client Receive instructions.
然后,客户端再将过滤后的数据包以pacp文件格式进行存储,以此生成数据包文件,当需要计算历史时间间隔时,再将所存储的数据包文件进行解析,并将数据包文件中的时间戳信息转换为客户端的本机时间且精确到微妙,比如转为的时间格式为年/月/日/时/分/秒/微秒,以此准确记录每个历史数据包到达客户端的时间,并基于已到达的数据包中相邻两数据包达到客户端的时间之差,确定过滤后的剩余历史数据包所对应的历史时间间隔。Then, the client stores the filtered data packet in pacp file format to generate a data packet file. When it is necessary to calculate the historical time interval, the stored data packet file is parsed, and the The time stamp information of the client is converted to the local time of the client and is accurate to subtle. For example, the converted time format is year/month/day/hour/minute/second/microsecond, so as to accurately record the time when each historical data packet arrives at the client Time, and based on the time difference between two adjacent data packets arriving at the client in the arrived data packets, determine the historical time interval corresponding to the remaining historical data packets after filtering.
需要说明的是,客户端中还可以设置有数据包抓取单元,数据包抓取单元用于根据与发送端之间的数据包传输协议、服务器输出数据包的端口地址、发送端的源地址,抓取服务器转发至客户端的历史数据包并以pacp文件格式进行存储。It should be noted that, the client may also be provided with a data packet capture unit, which is used to, according to the data packet transmission protocol with the sender, the port address of the server output data packet, and the source address of the sender, Grab the historical data packets forwarded by the server to the client and store them in the pacp file format.
本发明提供的传输延迟时间测量方法,客户端针对服务器逐帧发送的数据包,通过先进行过滤、再以文件格式存储、后解析并进行时间格式转换的方式,获取历史数据包所对应的历史时间间隔。以此结合逐帧发送和逐帧接收的方式,避免了发送端和客户端必须同步的限制,并进一步结合过滤、存储和格式转换的方式,确保了客户端记录数据包达到客户端时间的精确性和易识别性,从而为后续精确测量传输延迟时间奠定基础。In the method for measuring transmission delay time provided by the present invention, the client obtains the historical data corresponding to the historical data packets by first filtering the data packets sent by the server frame by frame, then storing them in a file format, and then parsing and converting the time format. time interval. In this way, the combination of frame-by-frame sending and frame-by-frame receiving avoids the restriction that the sender and client must be synchronized, and further combines filtering, storage and format conversion to ensure the accuracy of the time when the client records the data packet reaching the client. and easy identification, thus laying the foundation for the subsequent accurate measurement of transmission delay time.
可选的,当发送端的数量为多个时,所述基于预设的数据包接收协议,对所述历史数据包进行过滤,包括:Optionally, when there are multiple senders, the historical data packets are filtered based on the preset data packet receiving protocol, including:
首先,将所述历史数据包依次写入接收缓冲区;然后,基于预设的数据包通信协议,从所述接收缓冲区中的所述历史数据包中筛选出第一历史数据包;最后,基于预设的数据包标识信息,将所述第一历史数据包中与所述数据包标识信息不匹配的数据包进行过滤。First, write the historical data packets into the receiving buffer in sequence; then, based on the preset data packet communication protocol, filter out the first historical data packets from the historical data packets in the receiving buffer; finally, Based on the preset data packet identification information, the data packets in the first historical data packets that do not match the data packet identification information are filtered.
具体的,客户端中可以预设缓冲区,且将该缓冲区具体划分为接收缓冲区和播放缓冲区,接收缓冲区可以采用先进先出队列结构。并且,当发送端有多个时,服务器可以接收每个发送端逐帧发送的数据包以及将对应发送端发送的数据包再分别转发至客户端,当客户端逐帧接收到各个历史数据包时,可以先将每个历史数据包依次插入到该先进先出队列结构直至其队尾,但是客户端可能并不需要接收每个发送端的数据包,因此,此时可以通过设置数据包通信协议限制所要接收的数据包及过滤不需要的数据包,该数据包通信协议可以用于表征客户端所请求发送数据包的发送端,以此实现过滤接收缓冲区内历史数据包中与数据包通信协议不匹配的数据包的目的,从而筛选出第一历史数据包。比如,发送端有三个且分别为发送端1、发送端2及发送端3,如果数据包通信协议表明客户端请求接收发送端2的数据包,则将从历史数据包中过滤发送端1和发送端3发送的数据包以及保留发送端2发送的数据包,此时所保留的发送端2发送的数据包即为第一历史数据包。Specifically, a buffer can be preset in the client, and the buffer can be specifically divided into a receiving buffer and a playing buffer, and the receiving buffer can adopt a first-in-first-out queue structure. Moreover, when there are multiple senders, the server can receive the data packets sent by each sender frame by frame and forward the data packets sent by the corresponding sender to the client respectively. When the client receives each historical data packet frame by frame , you can insert each historical data packet into the first-in-first-out queue structure until the end of the queue, but the client may not need to receive each data packet from the sender, so you can set the data packet communication protocol at this time Limit the data packets to be received and filter unnecessary data packets. This data packet communication protocol can be used to represent the sender of the data packet requested by the client, so as to realize the communication with the data packet in the historical data packet in the filter receiving buffer The protocol does not match the purpose of the packet, thus filtering out the first historical packet. For example, there are three senders and they are sender 1, sender 2, and sender 3. If the data packet communication protocol indicates that the client requests to receive data packets from sender 2, sender 1 and sender 2 will be filtered from historical data packets. The data packet sent by the sender 3 and the data packet sent by the sender 2 are retained, and the retained data packet sent by the sender 2 is the first historical data packet.
进一步的,针对该先进先出队列结构中的第一历史数据包,可以基于预设的数据包标识信息,可以从队尾向队头依次进行报头校验,数据包标识信息用于表征对应数据包的报头标识。比如,当发送端以UDP向服务器发送数据包,服务器也采用UDP向客户端逐帧转发数据包时,客户端接收到的每个数据包的报头可以至少表征对应数据包是否满足UDP,也即判断每个第一历史数据包的报头是否为UDP报头,并将UDP报头对应的第一历史数据包作为有效数据包保留,将非UDP报头的第一历史数据包作为无效数据包丢弃,以此从接收缓冲区的多个第一历史数据包中筛选出有效历史数据包。Further, for the first historical data packet in the first-in-first-out queue structure, based on the preset data packet identification information, the header verification can be performed sequentially from the end of the queue to the head of the queue, and the data packet identification information is used to represent the corresponding data The header identifier of the package. For example, when the sender uses UDP to send data packets to the server, and the server also uses UDP to forward data packets to the client frame by frame, the header of each data packet received by the client can at least indicate whether the corresponding data packet satisfies UDP, that is, Judge whether the header of each first historical data packet is a UDP header, and retain the first historical data packet corresponding to the UDP header as a valid data packet, and discard the first historical data packet of the non-UDP header as an invalid data packet, thereby Effective historical data packets are filtered out from multiple first historical data packets in the receiving buffer.
需要说明的是,从接收缓冲区中的第一历史数据包中筛选出有效历史数据包时,可以根据数据包标识信息的不同进行对应验证,比如发送端针对所要发送的每个数据包会预先在其报头中分别设置表明对应数据包是否有效的标识信息,使得客户端逐帧收到每个数据包时能够快速准确的筛选出有效数据包。此处不作具体限定。另外,当发送端的数量为一个时,客户端经由服务器接收到该发送端逐帧发送的历史数据包时,可以将接收的每个历史数据包依次插入到先进先出队列结构直至其队尾,再基于预设的数据包标识信息,从该先进先出队列结构的队尾向队头依次进行报头校验,其报头验证过程与前述过程类似,此处不再赘述。It should be noted that, when valid historical data packets are screened out from the first historical data packets in the receiving buffer, corresponding verification may be performed according to different data packet identification information. The identification information indicating whether the corresponding data packet is valid is respectively set in its header, so that the client can quickly and accurately filter out valid data packets when receiving each data packet frame by frame. No specific limitation is made here. In addition, when the number of senders is one, when the client receives the historical data packets sent by the sender frame by frame via the server, it can sequentially insert each received historical data packet into the first-in-first-out queue structure until the end of the queue, Based on the preset data packet identification information, header verification is performed sequentially from the tail of the FIFO queue structure to the queue head. The header verification process is similar to the aforementioned process, and will not be repeated here.
本发明提供的传输延迟时间测量方法,客户端将所接收的数据包写入接收缓冲区,先后基于数据包通信协议和数据包标识信息对接收缓冲区内的数据包进行过滤,以此结合客户端缓冲方式和客户端内存划分方式,减小数据包的丢失概率,提高传输数据包的稳定性和可靠性,从而也能降低传输延迟的发生概率。In the method for measuring transmission delay time provided by the present invention, the client writes the received data packets into the receiving buffer, and successively filters the data packets in the receiving buffer based on the data packet communication protocol and the data packet identification information, so as to combine the client The end-side buffering method and the client-side memory partitioning method reduce the probability of data packet loss, improve the stability and reliability of transmitted data packets, and thus reduce the probability of transmission delay.
可选的,在所述将所述第一历史数据包中与所述数据包标识信息不匹配的数据包进行过滤之后,所述方法还包括:Optionally, after filtering the data packets that do not match the data packet identification information in the first historical data packets, the method further includes:
首先,将过滤后所得的第二历史数据包依次写入播放缓冲区;再进一步指示所述播放缓冲区对所述第二历史数据包进行整序后播放。Firstly, the filtered second historical data packets are sequentially written into the play buffer; and further instructing the play buffer to sort the second historical data packets and then play them.
具体的,客户端针对接收缓冲区过滤后所得的第二历史数据包,先将该第二历史数据包依次写入播放缓冲区,然后针对写入播放缓冲区的各个第二历史数据包先指示其执行整序操作,这是由于客户端接收到的历史数据包经过滤后其次序已被打乱,因此为了确保播放效果,可先指示第二历史数据包进行整序之后再指示播放,以使得对整序后的第二历史数据包进行解码并渲染播放。Specifically, the client first writes the second historical data packets into the playback buffer sequentially for the second historical data packets obtained after filtering in the receiving buffer, and then first instructs each second historical data packet written into the playback buffer It performs sorting operation, because the order of historical data packets received by the client has been disrupted after filtering, so in order to ensure the playback effect, you can first instruct the second historical data packets to be sorted before instructing to play, so that So that the sequenced second historical data packets are decoded and rendered for playback.
本发明提供的传输延迟时间测量方法,客户端针对过滤所得的第二历史数据包,先依次写入播放缓冲区,再进一步指示对第二实际数据包进行整序后再播放,以此结合客户端缓冲方式和整序方式,提高客户端播放视频的流畅性和可靠性。In the method for measuring transmission delay time provided by the present invention, the client first writes the second historical data packets obtained by filtering into the playback buffer in sequence, and then further instructs the second actual data packets to be sorted and then played, so as to combine the client End-side buffering and sorting methods improve the fluency and reliability of video playback on the client side.
可选的,在所述将所述历史数据包依次写入接收缓冲区之后,所述方法还包括:Optionally, after the historical data packets are sequentially written into the receiving buffer, the method further includes:
获取所述历史数据包写入所述接收缓冲区的缓冲区容量;若确定所述缓冲区容量大于预设容量最大值,则指示发送端降低数据包发送速率;若确定所述缓冲区容量小于预设容量最小值,则指示所述发送端增加数据包发送速率。Obtain the buffer capacity of the historical data packet written into the receiving buffer; if it is determined that the buffer capacity is greater than the maximum preset capacity, then instruct the sending end to reduce the data packet sending rate; if it is determined that the buffer capacity is less than If the preset minimum capacity is used, the sending end is instructed to increase the data packet sending rate.
具体的,客户端针对接收缓冲区中的历史数据包,可以根据该历史数据包所占空间指示发送端调整发送数据包的速率,也即,客户端中预先设置接收缓冲区的两个容量临界值,包括预设容量最大值和预设容量最小值,然后判断接收缓冲区内所写入的历史数据包所占的缓冲区容量分别与两个容量临界值之间的大小关系,若该缓冲区容量大于预设容量最大值,则指示发送端降低数据包发送速率,以此避免上溢风险;若该缓冲区容量小于预设容量最小值,则指示发送端增加数据包发送速率,以此避免下溢风险;若该缓冲区容量大于等于预设容量最小值且小于等于预设容量最大值,则指示发送端维持原有速率发送数据包。Specifically, for the historical data packets in the receiving buffer, the client can instruct the sending end to adjust the rate of sending data packets according to the space occupied by the historical data packets, that is, the client pre-sets two capacity thresholds of the receiving buffer Values, including the maximum value of the preset capacity and the minimum value of the preset capacity, and then judge the size relationship between the buffer capacity occupied by the historical data packets written in the receiving buffer and the two capacity critical values, if the buffer If the buffer capacity is greater than the preset maximum capacity, it will instruct the sender to reduce the packet sending rate to avoid the risk of overflow; if the buffer capacity is smaller than the preset minimum capacity, it will instruct the sender to increase the packet sending rate to Avoid the risk of underflow; if the buffer capacity is greater than or equal to the preset minimum capacity and less than or equal to the preset maximum capacity, the sender is instructed to maintain the original rate to send data packets.
需要说明的是,服务器可以利用FLV文件的时间戳作为控制发送端的数据包发送速率的控制标准,在其缓冲区也预设两个临界值MAX和MIN,当服务器将接收到的历史数据包放入缓冲区时,可以监测缓冲区的容量与MAX和MIN之间的大小关系,当缓冲区的容量大于MAX时,为避免上溢风险,可以指示发送端减小数据包发送速率;当缓冲区的容量小于MIN时,为避免下溢风险,可以指示发送端增大数据包发送速率;当缓冲区的容量大于等于MIN且小于等于MAX时,可以指示发送端按照原有速率发送数据包,以此解决网络条件极端不理想情况下的数据丢包问题。It should be noted that the server can use the time stamp of the FLV file as a control standard for controlling the sending rate of data packets at the sending end, and also preset two critical values MAX and MIN in its buffer, when the server puts the received historical data packets into When entering the buffer, you can monitor the size relationship between the capacity of the buffer and MAX and MIN. When the capacity of the buffer is greater than MAX, in order to avoid the risk of overflow, you can instruct the sender to reduce the packet sending rate; when the buffer When the capacity of the buffer is less than MIN, in order to avoid the risk of underflow, you can instruct the sender to increase the data packet sending rate; when the capacity of the buffer is greater than or equal to MIN and less than or equal to MAX, you can instruct the sender to send data packets at the original rate. This solves the problem of data packet loss under extremely unfavorable network conditions.
本发明提供的传输延迟时间测量方法,客户端通过接收缓冲区内写入的历史数据包所占的缓冲区容量与预设容量最大值或容量最小值的大小关系,指示发送端降低或者增加数据包发送速率,以此提高发送端和客户端的灵活交互性,从而也有效降低了数据丢失率,确定数据传输的可靠性和稳定性。In the method for measuring transmission delay time provided by the present invention, the client instructs the sending end to reduce or increase data through the relationship between the buffer capacity occupied by the historical data packets written in the receiving buffer and the preset maximum or minimum capacity. The packet sending rate is used to improve the flexible interaction between the sender and the client, thereby effectively reducing the data loss rate and ensuring the reliability and stability of data transmission.
可选的,步骤130的实现过程可以包括:Optionally, the implementation process of step 130 may include:
将所述预测时间间隔与所述当前数据包对应的实际时间间隔进行大小比较;确定所述实际时间间隔大于所述预测时间间隔时,基于所述实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Comparing the predicted time interval with the actual time interval corresponding to the current data packet; when determining that the actual time interval is greater than the predicted time interval, based on the actual time interval and the data packet transmission time interval of the sending end, Output propagation delay time.
具体的,客户端针对服务器逐帧传输的当前数据包对应的预测时间间隔,可以先将该预测时间间隔与该当前数据包对应的实际时间间隔进行大小比较,当该实际时间间隔大于该预测时间间隔时,可以判定通信过程中发生较大传输延迟或链路中断、丢包等其它异常,此时可将该实际时间间隔与发送端的数据包发送时间间隔之差,确定为传输延迟时间并输出;反之,当该实际时间间隔小于等于该预测时间间隔时,表明当前没有传输延迟及异常,此时可针对下一正在传输的数据包进行预测并判断是否发生传输延迟。比如,当第n个实际时间间隔大于第n个预测时间间隔时,可以将第n个实际时间间隔与数据包发送时间间隔t之间的差值,作为传输延迟时间;反之,当第n个实际时间间隔小于等于第n个预测时间间隔时,可以将前n个历史时间间隔输入至预设延迟预测模型中,预测第n+1个预测时间间隔,再判断第n+1个预测时间间隔与第n+1个实际时间间隔的关系是否发生传输延迟,以此方式循环,直至测量传输延迟流程结束。Specifically, the client can first compare the predicted time interval with the actual time interval corresponding to the current data packet for the predicted time interval corresponding to the current data packet transmitted by the server frame by frame. When the actual time interval is greater than the predicted time interval interval, it can be determined that there is a large transmission delay or link interruption, packet loss and other abnormalities in the communication process. At this time, the difference between the actual time interval and the sending time interval of the data packet at the sender can be determined as the transmission delay time and output ; Conversely, when the actual time interval is less than or equal to the predicted time interval, it indicates that there is no transmission delay and abnormality at present, and at this time, the next data packet being transmitted can be predicted and judged whether transmission delay occurs. For example, when the nth actual time interval is greater than the nth predicted time interval, the difference between the nth actual time interval and the packet sending time interval t can be used as the transmission delay time; otherwise, when the nth When the actual time interval is less than or equal to the nth prediction time interval, you can input the first n historical time intervals into the preset delay prediction model, predict the n+1th prediction time interval, and then judge the n+1th prediction time interval Whether transmission delay occurs in relation to the n+1th actual time interval, loop in this way until the process of measuring transmission delay ends.
本发明提供的传输延迟时间测量方法,客户端判定服务器逐帧传输的当前数据包的实际时间间隔大于预测时间间隔时,通过将实际时间间隔与数据包发送时间间隔的差值确定为传输延迟时间的方式,提高了测量传输延迟时间的直观有效性和可靠稳定性。In the transmission delay measurement method provided by the present invention, when the client determines that the actual time interval of the current data packet transmitted by the server frame by frame is greater than the predicted time interval, the difference between the actual time interval and the data packet sending time interval is determined as the transmission delay time The method improves the intuitive effectiveness and reliable stability of measuring transmission delay time.
可选的,所述预设延迟预测模型的训练过程包括:Optionally, the training process of the preset delay prediction model includes:
首先,基于样本历史时间间隔,确定训练集和测试集;再进一步基于所述训练集对LSTM模型进行训练,得到训练后的LSTM模型;然后,基于所述测试集对所述训练后的LSTM模型进行测试,得到测试结果;最后,基于所述测试结果与所述测试集之间的误差,确定所述训练后的LSTM模型为预设延迟预测模型。First, determine the training set and the test set based on the sample historical time interval; then further train the LSTM model based on the training set to obtain the trained LSTM model; then, based on the test set, train the LSTM model Perform a test to obtain a test result; finally, based on the error between the test result and the test set, determine that the trained LSTM model is a preset delay prediction model.
具体的,客户端接收并记录已到达的N个数据包各自对应的到达时间,以此得到N个到达时间,并以每两个到达时间的时间间隔为样本历史时间间隔,得到N-1个样本历史时间间隔,再进一步对该N-1个样本历史时间间隔进行归一化处理,生成样本数据集,将该样本数据集以p:q的预设比例划分为训练集和测试集,N为大于1的整数,p、q均为大于0的正整数,且p>q。Specifically, the client receives and records the corresponding arrival times of the N data packets that have arrived, so as to obtain N arrival times, and takes the time interval between every two arrival times as the sample historical time interval to obtain N-1 Sample historical time interval, and then further normalize the N-1 sample historical time intervals to generate a sample data set, and divide the sample data set into a training set and a test set with a preset ratio of p:q, N is an integer greater than 1, p and q are both positive integers greater than 0, and p>q.
然后,设置初始训练次数M,并使用训练集对LSTM模型进行训练,直至训练次数达到M,得到本轮训练后的LSTM模型;再进一步使用测试集对本轮训练后的LSTM模型进行测试,得到本轮测试结果,计算本轮测试结果与测试集中样本历史时间间隔之间的均方根误差△t1,接着使用RMSProp一阶优化算法,通过梯度下降的方式对训练M次后的LSTM模型进行模型参数优化,比如学习率的更新和下轮迭代次数的更新,使用更新后的迭代次数对本轮训练后的LSTM模型再次进行训练,直至训练次数达到M,得到次轮训练后的LSTM模型,再使用测试集对次轮训练后的LSTM模型进行测试,得到次轮测试结果,并计算次轮测试结果与测试集中样本历史时间间隔之间的均方根误差△t2。以此方式不断进行训练和测试,直至所得到的均方根误差不再下降时停止训练和测试,并将均方根误差最小时对应的训练后的LSTM模型确定为预设延迟预测模型。Then, set the initial number of training times M, and use the training set to train the LSTM model until the number of training times reaches M, and obtain the LSTM model after this round of training; then further use the test set to test the LSTM model after this round of training, and get For the test results of this round, calculate the root mean square error Δt 1 between the test results of this round and the historical time interval of the samples in the test set, and then use the RMSProp first-order optimization algorithm to perform gradient descent on the LSTM model after training M times. Model parameter optimization, such as updating the learning rate and the number of iterations in the next round, uses the updated number of iterations to train the LSTM model after the current round of training until the number of training reaches M, and obtains the LSTM model after the second round of training. Then use the test set to test the LSTM model after the second round of training, get the second round of test results, and calculate the root mean square error Δt 2 between the second round of test results and the historical time interval of the samples in the test set. Continuously carry out training and testing in this way until the obtained root mean square error no longer decreases, stop training and testing, and determine the trained LSTM model corresponding to the minimum root mean square error as the preset delay prediction model.
需要说明的是,本发明实施例所提及的均方根误差可使用现有均方根误差的计算方法进行计算,此处不作具体限定。It should be noted that the root mean square error mentioned in the embodiment of the present invention may be calculated using an existing root mean square error calculation method, which is not specifically limited here.
本发明提供的传输延迟时间测量方法,通过先使用训练集训练LSTM模型、后使得测试集测试训练后的LSTM模型的方式,得到预设延迟预测模型,以此结合循环神经网络模型学习的方法,实现对下一时间间隔进行准确预测的目的,从而进一步提高了测量传输延迟时间的准确性。The method for measuring transmission delay time provided by the present invention obtains a preset delay prediction model by first using the training set to train the LSTM model, and then makes the test set test the trained LSTM model, in combination with the method of recurrent neural network model learning, The purpose of accurately predicting the next time interval is realized, thereby further improving the accuracy of measuring transmission delay time.
可选的,针对LSTM模型的训练,除了对每轮的迭代次数和模型参数进行更新外,还可以对每轮参与训练和测试的训练集和测试集进行更新。基于此,所述方法还可以包括:Optionally, for the training of the LSTM model, in addition to updating the number of iterations and model parameters of each round, the training set and test set participating in the training and testing of each round can also be updated. Based on this, the method may also include:
基于预设时长内接收到的新的历史时间间隔,对所述训练集和所述测试集分别进行更新。The training set and the test set are respectively updated based on new historical time intervals received within a preset time period.
具体的,由于发送端的数据包发送时间间隔为t,因此为了提高模型精度,可以对每轮的训练集和测试集也进行更新,也即,客户端记录预设时长T内到达的新的历史时间间隔,并对该新的历史时间间隔进行归一化处理后再按p:q的预设比例分别划分至当前的训练集和测试集中,以此实现对训练集和测试集的更新。其中,T=Ft,F为大于1的整数。Specifically, since the sending interval of data packets at the sending end is t, in order to improve the accuracy of the model, the training set and test set of each round can also be updated, that is, the client records the new history arriving within the preset time T Time interval, and the new historical time interval is normalized, and then divided into the current training set and test set according to the preset ratio of p:q, so as to update the training set and test set. Wherein, T=Ft, F is an integer greater than 1.
本发明提供的传输延迟时间测量方法,通过预设时长内到达客户端的新的历史时间间隔对训练集和测试集进行更新的方式,实现在更新每轮迭代次数和模型参数的同时也能够更新训练集和测试集的目的,相对于现有训练LSTM模型的过程与预测过程分离的方式,本发明的训练过程和测试过程均依赖于最新获取的新的历史时间间隔,从而更进一步提高了预设延迟预测模型的精准性和稳定性。The transmission delay time measurement method provided by the present invention updates the training set and the test set through the new historical time interval arriving at the client within the preset time period, so as to update the training set while updating the number of iterations and model parameters in each round. The purpose of set and test set, compared with the existing way of separating the process of training LSTM model from the prediction process, the training process and test process of the present invention all depend on the new historical time interval acquired recently, thereby further improving the preset Accuracy and stability of latency prediction models.
基于前述实施例,图3是本发明提供的传输延迟时间测量方法的总体流程图,如图3所示,该传输延迟时间测量方法包括:Based on the foregoing embodiments, FIG. 3 is an overall flow chart of the method for measuring transmission delay time provided by the present invention. As shown in FIG. 3 , the method for measuring transmission delay time includes:
步骤310、发送端采用数据包发送时间间隔t向服务器逐帧发送数据包。In step 310, the sending end sends data packets frame by frame to the server using the data packet sending time interval t.
步骤320、服务器逐帧接收数据包并进行缓冲,然后再逐帧转发至客户端。Step 320, the server receives and buffers the data packets frame by frame, and then forwards them to the client frame by frame.
步骤330、客户端的接收缓冲区逐帧接收历史数据包,并对接收到的历史数据包进行过滤。Step 330, the receiving buffer of the client receives historical data packets frame by frame, and filters the received historical data packets.
步骤340、客户端的接收缓冲区播放过滤后的历史数据包。其中,过滤后的历史数据包可以为前述实施例中的过滤后所得的第二历史数据包。Step 340, the receiving buffer of the client plays the filtered historical data packets. Wherein, the filtered historical data packet may be the second historical data packet obtained after filtering in the foregoing embodiment.
步骤350、客户端将过滤后的历史数据包以pscp格式存储为pacp文件。Step 350, the client stores the filtered historical data packets as a pacp file in pscp format.
步骤360、客户端解析存储的pacp文件,确定过滤后的历史数据包所对应的历史时间间隔。Step 360, the client parses the stored pacp file, and determines the historical time interval corresponding to the filtered historical data packets.
步骤370、客户端将该历史时间间隔输入至预设延迟预测模型中后,得到逐帧传输的当前数据包对应的预测时间间隔。Step 370: After the client inputs the historical time interval into the preset delay prediction model, the predicted time interval corresponding to the current data packet transmitted frame by frame is obtained.
步骤380、客户端基于预测时间间隔、当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Step 380, the client outputs the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sender.
本发明提供的传输延迟时间测量方法,通过将服务器逐帧传输的历史数据包所对应的历史时间间隔输入至预设延迟预测模型的方式,预测逐帧传输的当前数据包对应的预测时间间隔,并基于该预测时间间隔、当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。以此结合逐帧传输方式降低数据发送延时,实现兼顾整个传输系统的实时性和带宽利用率的目的,无需发送端和客户端时间同步,提高了传输延迟测量的适用性;进一步的,由于预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型,以此能够结合训练循环神经网络的方式,实现提高预测时间间隔的精准度的目的,从而也能有效提高传输延迟测量的精确性和可靠性。The transmission delay time measurement method provided by the present invention predicts the prediction time interval corresponding to the current data packet transmitted frame by frame by inputting the historical time interval corresponding to the historical data packet transmitted frame by frame by the server into the preset delay prediction model, And output the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval at the sending end. In this way, combined with the frame-by-frame transmission method, the data transmission delay is reduced, and the purpose of taking into account the real-time and bandwidth utilization of the entire transmission system is achieved, and the time synchronization between the sending end and the client is not required, which improves the applicability of transmission delay measurement; further, due to The preset delay prediction model is a model obtained by training the LSTM model based on the sample historical time interval, which can be combined with the training of the cyclic neural network to achieve the purpose of improving the accuracy of the prediction time interval, thereby effectively improving the accuracy of transmission delay measurement sex and reliability.
下面对本发明提供的传输延迟时间测量装置进行描述,下文描述的传输延迟时间测量装置与上文描述的传输延迟时间测量方法可相互对应参照。The device for measuring transmission delay time provided by the present invention is described below, and the device for measuring transmission delay time described below and the method for measuring transmission delay time described above can be referred to in correspondence with each other.
参照图4,为本发明实施例提供的传输延迟时间测量装置,在图4中,该传输延迟时间测量装置400,包括:Referring to FIG. 4, the transmission delay time measurement device provided for the embodiment of the present invention, in FIG. 4, the transmission delay time measurement device 400 includes:
获取模块410,用于获取服务器逐帧传输的历史数据包所对应的历史时间间隔。The obtaining module 410 is configured to obtain the historical time interval corresponding to the historical data packets transmitted frame by frame by the server.
预测模块420,用于将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔;其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型。The prediction module 420 is configured to input the historical time interval into a preset delay prediction model to obtain a prediction time interval corresponding to the current data packet transmitted frame by frame; wherein, the preset delay prediction model is based on sample historical time The model obtained by interval training the LSTM model.
输出模块430,用于基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。The output module 430 is configured to output the transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
可选的,获取模块410,具体可以用于接收服务器逐帧传输的历史数据包;基于预设的数据包接收协议,对所述历史数据包进行过滤;按照预设文件格式,将过滤后的剩余历史数据包存储为数据包文件;解析所述数据包文件中的时间戳信息并进行时间格式转化,得到所述剩余历史数据包所对应的历史时间间隔。Optionally, the obtaining module 410 can specifically be used to receive historical data packets transmitted frame by frame by the server; filter the historical data packets based on a preset data packet receiving protocol; and filter the filtered historical data packets according to a preset file format. The remaining historical data packets are stored as data packet files; the time stamp information in the data packet files is analyzed and time format conversion is performed to obtain the historical time intervals corresponding to the remaining historical data packets.
可选的,获取模块410,具体还可以用于建立与发送端之间的数据通信链路;向所述发送端发送数据传输请求,所述数据传输请求携带数据封装要求和数据发送方式;接收所述发送端针对所述数据传输请求发送的数据包。Optionally, the obtaining module 410 can also be specifically used to establish a data communication link with the sending end; send a data transmission request to the sending end, and the data transmission request carries a data encapsulation requirement and a data transmission method; A data packet sent by the sending end in response to the data transmission request.
可选的,获取模块410,具体还可以用于将所述历史数据包依次写入接收缓冲区;基于预设的数据包通信协议,从所述接收缓冲区中的所述历史数据包中筛选出第一历史数据包;基于预设的数据包标识信息,将所述第一历史数据包中与所述数据包标识信息不匹配的数据包进行过滤。Optionally, the acquisition module 410 can also be specifically configured to sequentially write the historical data packets into the receiving buffer; based on a preset data packet communication protocol, filter the historical data packets in the receiving buffer Outputting the first historical data packet; based on the preset data packet identification information, filtering the data packets in the first historical data packet that do not match the data packet identification information.
可选的,输出模块430,具体可以用于将所述预测时间间隔与所述当前数据包对应的实际时间间隔进行大小比较;确定所述实际时间间隔大于所述预测时间间隔时,基于所述实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Optionally, the output module 430 can be specifically configured to compare the predicted time interval with the actual time interval corresponding to the current data packet; when it is determined that the actual time interval is greater than the predicted time interval, based on the The actual time interval and the packet transmission time interval of the sender, output transmission delay time.
可选的,所述装置还可以包括播放模块,用于将过滤后所得的第二历史数据包依次写入播放缓冲区;指示所述播放缓冲区对所述第二历史数据包进行整序后播放。Optionally, the device may also include a playback module, configured to sequentially write the filtered second historical data packets into the playback buffer; instruct the playback buffer to sort the second historical data packets play.
可选的,所述装置还可以包括调整模块,用于获取所述历史数据包写入所述接收缓冲区的缓冲区容量;若确定所述缓冲区容量大于预设容量最大值,则指示发送端降低数据包发送速率;若确定所述缓冲区容量小于预设容量最小值,则指示所述发送端增加数据包发送速率。Optionally, the device may also include an adjustment module, configured to obtain the buffer capacity of the historical data packets written into the receiving buffer; if it is determined that the buffer capacity is greater than the preset maximum capacity, then instruct to The end reduces the data packet sending rate; if it is determined that the buffer capacity is smaller than the preset minimum capacity, instruct the sending end to increase the data packet sending rate.
可选的,所述装置还可以包括训练模块,用于基于样本历史时间间隔,确定训练集和测试集;基于所述训练集对LSTM模型进行训练,得到训练后的LSTM模型;基于所述测试集对所述训练后的LSTM模型进行测试,得到测试结果;基于所述测试结果与所述测试集之间的误差,确定所述训练后的LSTM模型为预设延迟预测模型。Optionally, the device may also include a training module, configured to determine a training set and a test set based on the sample history time interval; train the LSTM model based on the training set to obtain a trained LSTM model; based on the test The set tests the trained LSTM model to obtain a test result; based on the error between the test result and the test set, it is determined that the trained LSTM model is a preset delay prediction model.
可选的,所述装置还可以包括更新模块,用于基于预设时长内接收到的新的历史时间间隔,对所述训练集和所述测试集分别进行更新。Optionally, the device may further include an update module, configured to update the training set and the test set based on a new historical time interval received within a preset time period.
图5示例了一种电子设备的实体结构示意图,如图5所示,该电子设备500可以包括:处理器(processor)510、通信接口(CommunicationsInterface)520、存储器(memory)530和通信总线540,其中,处理器510,通信接口520,存储器530通过通信总线540完成相互间的通信。处理器510可以调用存储器530中的逻辑指令,以执行传输延迟时间测量方法,该方法包括:FIG. 5 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG. 5, the electronic device 500 may include: a processor (processor) 510, a communication interface (CommunicationsInterface) 520, a memory (memory) 530 and a communication bus 540, Wherein, the processor 510 , the communication interface 520 , and the memory 530 communicate with each other through the communication bus 540 . The processor 510 can call the logic instructions in the memory 530 to execute the method for measuring transmission delay time, and the method includes:
获取服务器逐帧传输的历史数据包所对应的历史时间间隔;Obtain the historical time interval corresponding to the historical data packets transmitted frame by frame by the server;
将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔;其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型;The historical time interval is input into a preset delay prediction model to obtain a prediction time interval corresponding to the current data packet transmitted frame by frame; wherein, the preset delay prediction model is obtained by training an LSTM model based on a sample historical time interval Model;
基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Outputting a transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
此外,上述的存储器530中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-OnlyMemory)、随机存取存储器(RAM,RandomAccessMemory)、磁碟或者光盘等各种可以存储程序代码的介质。In addition, the above logic instructions in the memory 530 may be implemented in the form of software function units and be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, RandomAccessMemory), magnetic disk or optical disk and other media that can store program codes.
另一方面,本发明还提供一种计算机程序产品,所述计算机程序产品包括计算机程序,计算机程序可存储在非暂态计算机可读存储介质上,所述计算机程序被处理器执行时,计算机能够执行上述各方法所提供的传输延迟时间测量方法,该方法包括:On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can Perform the transmission delay time measurement method provided by each of the above methods, the method includes:
获取服务器逐帧传输的历史数据包所对应的历史时间间隔;Obtain the historical time interval corresponding to the historical data packets transmitted frame by frame by the server;
将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔;其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型;The historical time interval is input into a preset delay prediction model to obtain a prediction time interval corresponding to the current data packet transmitted frame by frame; wherein, the preset delay prediction model is obtained by training an LSTM model based on a sample historical time interval Model;
基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Outputting a transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
又一方面,本发明还提供一种非暂态计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现以执行上述各方法提供的传输延迟时间测量方法,该方法包括:In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to perform the transmission delay time measurement method provided by the above methods, the method include:
获取服务器逐帧传输的历史数据包所对应的历史时间间隔;Obtain the historical time interval corresponding to the historical data packets transmitted frame by frame by the server;
将所述历史时间间隔输入至预设延迟预测模型,得到所述逐帧传输的当前数据包对应的预测时间间隔;其中,所述预设延迟预测模型是基于样本历史时间间隔训练LSTM模型得到的模型;The historical time interval is input into a preset delay prediction model to obtain a prediction time interval corresponding to the current data packet transmitted frame by frame; wherein, the preset delay prediction model is obtained by training an LSTM model based on a sample historical time interval Model;
基于所述预测时间间隔、所述当前数据包对应的实际时间间隔和发送端的数据包发送时间间隔,输出传输延迟时间。Outputting a transmission delay time based on the predicted time interval, the actual time interval corresponding to the current data packet, and the data packet sending time interval of the sending end.
以上所描述的装置实施例仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下,即可以理解并实施。The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative efforts.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件。基于这样的理解,上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品可以存储在计算机可读存储介质中,如ROM/RAM、磁碟、光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.
最后应说明的是:以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims (12)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210716495.2A CN115277497B (en) | 2022-06-22 | 2022-06-22 | Transmission delay time measurement method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210716495.2A CN115277497B (en) | 2022-06-22 | 2022-06-22 | Transmission delay time measurement method, device, electronic equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115277497A CN115277497A (en) | 2022-11-01 |
CN115277497B true CN115277497B (en) | 2023-09-01 |
Family
ID=83762101
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210716495.2A Active CN115277497B (en) | 2022-06-22 | 2022-06-22 | Transmission delay time measurement method, device, electronic equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115277497B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116366628A (en) * | 2022-12-12 | 2023-06-30 | 中国建设银行股份有限公司 | File transmission prediction method, device, equipment, storage medium and product |
CN116170348A (en) * | 2023-03-09 | 2023-05-26 | 上海奥朋医疗科技有限公司 | Method, system, medium and equipment for predicting communication delay of remote surgical robot |
CN116668475B (en) * | 2023-05-18 | 2023-12-26 | 尚学仕教育科技(北京)有限公司 | Online education operation system |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101022369A (en) * | 2007-03-23 | 2007-08-22 | 中山大学 | End-to-end quening time delay measuring method |
JP2012182747A (en) * | 2011-03-02 | 2012-09-20 | Ntt Communications Kk | Network quality measurement method, delay fluctuation estimation apparatus, network monitoring system, and program |
CN107454017A (en) * | 2017-06-05 | 2017-12-08 | 上海交通大学 | A Cooperative Scheduling Method for Mixed Data Flows in Cloud Data Center Network |
CN111539780A (en) * | 2019-09-04 | 2020-08-14 | 拉扎斯网络科技(上海)有限公司 | A task processing method, device, storage medium and electronic device |
CN113992599A (en) * | 2021-11-18 | 2022-01-28 | 北京达佳互联信息技术有限公司 | Training method and device of time delay prediction model and congestion control method and device |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11551182B2 (en) * | 2020-09-22 | 2023-01-10 | Convey, Llc | Systems and methods for AI-based detection of delays in a shipping network |
US11353700B2 (en) * | 2020-10-07 | 2022-06-07 | Industrial Technology Research Institute | Orientation predicting method, virtual reality headset and non-transitory computer-readable medium |
-
2022
- 2022-06-22 CN CN202210716495.2A patent/CN115277497B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101022369A (en) * | 2007-03-23 | 2007-08-22 | 中山大学 | End-to-end quening time delay measuring method |
JP2012182747A (en) * | 2011-03-02 | 2012-09-20 | Ntt Communications Kk | Network quality measurement method, delay fluctuation estimation apparatus, network monitoring system, and program |
CN107454017A (en) * | 2017-06-05 | 2017-12-08 | 上海交通大学 | A Cooperative Scheduling Method for Mixed Data Flows in Cloud Data Center Network |
CN111539780A (en) * | 2019-09-04 | 2020-08-14 | 拉扎斯网络科技(上海)有限公司 | A task processing method, device, storage medium and electronic device |
CN113992599A (en) * | 2021-11-18 | 2022-01-28 | 北京达佳互联信息技术有限公司 | Training method and device of time delay prediction model and congestion control method and device |
Non-Patent Citations (1)
Title |
---|
基于延迟预测的自适应媒体播放算法;陈瑞;焦良葆;;计算机工程(24);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN115277497A (en) | 2022-11-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN115277497B (en) | Transmission delay time measurement method, device, electronic equipment and storage medium | |
US9237108B2 (en) | Jitter buffer emulation for RTP streams in passive network monitoring systems | |
US10659380B2 (en) | Media buffering | |
US9231992B2 (en) | Informative data streaming server | |
US9585062B2 (en) | System and method for implementation of dynamic encoding rates for mobile devices | |
US9565482B1 (en) | Adaptive profile switching system and method for media streaming over IP networks | |
CN102932676B (en) | Self-adaptive bandwidth transmitting and playing method based on audio and video frequency synchronization | |
EP2898637B1 (en) | Exporting real time network traffic latency and buffer occupancy | |
WO2013163465A1 (en) | Impairment simulation for network communication to enable voice quality degradation estimation | |
US20130091528A1 (en) | Video reproduction system, receive terminal, home gateway device, and quality control system | |
EP3125497B1 (en) | Estimating processor load | |
WO2011102317A1 (en) | Monitoring system, device, and method | |
US20200274910A1 (en) | Network status of stream push terminal-based optimizaiton method, system and stream push terminal | |
JP2020507235A (en) | Data buffering method, network device, and storage medium | |
CN101631030B (en) | Prediction method of self-adapting digital home network flow media transmission band width | |
CN113473185B (en) | Method and device for detecting available bandwidth based on video stream key frame burst characteristics | |
WO2024244879A1 (en) | Network transmission optimization method and apparatus, storage medium, electronic device, and program product | |
US9697328B2 (en) | Transmission apparatus, transmission method, reception apparatus, reception method, and computer program | |
EP2613541A1 (en) | Server device, video image quality measurement system, video image quality measurement method and program | |
US11368389B2 (en) | Data transfer method, data transfer device and program | |
US10587518B2 (en) | Identifying network conditions | |
KR101098826B1 (en) | Method and apparatus for providing video streaming scheduling | |
JP6904600B2 (en) | Data acquisition device, client-server system, data acquisition method, and program | |
JP2008244894A (en) | Packet receiving apparatus, method and program for determining necessary and sufficient reception buffer size | |
JP2004080102A (en) | Packet processing device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |