CN102868666A - Implementation method of streaming media quality monitoring report based on user experience interaction - Google Patents

Abstract

A method for implementing a streaming media quality monitoring report based on user experience interaction. First, a custom two-way transmission quality of experience QoE interaction protocol and related network environment parameters and service performance parameters are set. The server monitors the sending status of streaming media, and the client Monitor the receiving status of streaming media and evaluate its user experience quality, generate SC report and CS report respectively and then transmit them via the network, realize real-time interaction of streaming media quality monitoring reports between the client and server, so Real-time monitoring of streaming media service quality. The present invention is proposed based on the existing network technology basis, extensively absorbing the latest achievements of various current streaming media quality monitoring and monitoring technologies, and performing comprehensive optimization. The operation steps are simple, less bandwidth resources are occupied, the calculation difficulty is low, and the existing network does not need to be changed or upgraded, and the implementation is easy.

Description

Implementation method of streaming media quality monitoring report based on user experience interaction

technical field

The invention relates to a method for realizing a streaming media quality monitoring report based on user experience interaction, and belongs to the technical field of multimedia communication.

Background technique

Streaming media is a technology that enables audio, video and other multimedia elements to be played in real time on the Internet and wireless networks without downloading and waiting. Streaming media technology can provide users with a continuous stream of audio and video data to meet the needs of users to watch dynamic audio and video online.

With the rapid development of computer networks, and the rapid development of wireless networks such as the third-generation mobile communication technology 3G (3rd-generation) and wireless local area network WLAN (Wireless Local Area Network), the application of streaming media services is becoming more and more extensive. The important field of application in the wired network is interactive network television IPTV (Interactive Personal TV), while in the mobile network field it is mainly mobile streaming media.

However, in existing networks, especially when streaming media services are provided on wireless networks with very tight spectrum resources, harsh network environments such as network delays and congestion often cause delays, packet loss, and jitter in the transmission of multimedia data packets. or out-of-sequence, etc., thereby deteriorating the service quality of streaming media and adversely affecting user experience. For network operators and service providers, it is necessary to use an effective and accurate streaming media quality evaluation solution to monitor their business transmission performance, thereby optimizing network performance and improving service quality.

However, the current streaming media quality evaluation methods and the actual use of streaming media transmission and control protocols are not ideal. They do not truly reflect the service quality, nor can they achieve real-time evaluation and monitoring of streaming media services. The main reason is that the client and server Failure to exchange information in a timely manner: On the one hand, the client cannot know the network environment conditions in a timely manner, let alone grasp the impact of the network environment on the quality of user experience; Quality of user experience, in order to adjust network parameters in time, are two major problems in current streaming media service quality monitoring.

Here is a brief overview of the state of the art for mobile streaming quality reporting:

(1) Use network parameters to predict streaming media quality directly on the network side. For example, the document "Research on Monitoring Quality of 3G Streaming Media Service" [C] (published in "International Conference on Communication Systems, Networks and Applications" 2010), using the number of packet loss, packet loss rate, packet delay, packet jitter, etc. Parameters to evaluate streaming quality. The document "No-Reference Quality Assessment for Networked Video via Primary Analysis of Bit Stream" [J] (published in "IEEE Transactions on circuits and video technology" 2010) considers the encoding-side quantization step size, frame rate and network-side packet loss, etc. The impact of the parameters is weighted in the time domain, and finally the objective evaluation of the quality of the video stream is realized. IETF RFC4445 also proposes a method for measuring IPTV transmission quality: MDI (Media Delivery Index), and only uses two network parameters: DF (DelayFactor) and MLR (Media Loss Rate) for measurement.

(2) Report the service capability of the client to the streaming media server in the form of a report. For example, the Chinese invention patent application "A Method of Reporting Capabilities and Related Processing Devices" (Application No. 200810056281.7) introduces a reporting method, but this method is limited to reporting the client's business capability identification to the server and is not interactive; Moreover, the judgment of the client's service capability is not based on the user's perception. The judgment result of this capability can only provide a reference for the server to adjust the network status, and cannot be used as a complete basis. Chinese invention patent application "Method and device for reporting streaming media quality by cross-referencing related applications" (Application No. 200780048832.9) introduces another method for cross-referencing reporting streaming media quality, which mainly focuses on the method of reporting, without Covers what to report and how to get it. An IPTV service management method, system and service quality management node involved in the Chinese invention patent application "Internet TV service management method and system, service quality management node" (application number 201010174116.9), which can overcome the defect that IPTV service indicators cannot be automatically measured , but the parameters reported by IPTV terminals are only ratings, request success rate, packet loss rate, frame loss rate, and jitter, etc., and do not involve the transmission and viewing experience quality of audio and video services themselves.

(3) In the traditional RTP/RTCP/RTSP protocol cluster, the RTCP protocol plays a role in controlling the quality of service transmission. The RTCP protocol uses network parameters to judge and evaluate service quality, and adjust parameters such as network bandwidth. However, the RTCP protocol does not monitor and analyze the user's service quality of experience, so it cannot fully and truly reflect the streaming media service quality of the client.

From the above analysis, it can be seen that on the basis of the client independently evaluating the streaming media experience quality, combined with network parameters to implement a comprehensive evaluation, and reporting the relevant parameters of the client to the streaming media server in real time, it has not been seen at home and abroad so far. There are relevant documents published, and at the same time, the existing protocols also have many deficiencies in the aspect of service quality judgment and control based on user experience.

Contents of the invention

In view of this, the purpose of the present invention is to provide a method for implementing a streaming media quality monitoring report based on user experience interaction. The present invention is aimed at the current streaming media quality monitoring requirements, and fully considers the technical foundation that can be implemented in the existing network, and then extensively absorbs the latest achievements of various current streaming media quality monitoring and monitoring technologies, and carries out comprehensive optimization. Compared with various existing quality monitoring technical solutions, the method of the present invention is based on the existing network technology, making full use of relevant advanced and mature technologies that have been deployed in the network, and the quality of experience QoE (Quality of Experience) protocol transmits the quality of user experience and related parameters of streaming media, realizes real-time interaction between the client and the server, and solves the above two problems in the quality monitoring of streaming media services.

In order to achieve the above object, the present invention proposes a method for realizing a streaming media quality monitoring report based on user experience interaction. Environment Parameters) and service performance parameters (Service Performance Parameters), the server monitors the sending status of streaming media, the client monitors the receiving status of streaming media and evaluates its user experience quality, and generates S-C reports (Server-Client Report) from the server to the client respectively And the C-S report (Client-Server Report) from the client to the server, and then transmitted through the network, realizes the real-time interaction of the streaming media quality monitoring report between the client and the server, so as to realize the quality of the streaming media service transmitted between the two real-time monitoring.

The implementation method of the streaming media quality monitoring report based on user experience interaction in the present invention uses a self-defined QoE interactive protocol for two-way transmission and related parameters, so that the streaming media client can obtain network environment parameters during service transmission in time, and analyze wired and wireless The impact of network conditions on the transmission of streaming media services, and combined with the set streaming media service quality evaluation method to obtain the quality of experience of service users. Through real-time interaction with the client, the streaming media server understands and learns the quality of experience of the client user on the streaming media service, so as to make timely adjustments to network conditions or streaming media service parameters to improve the service quality of the service.

From the perspective of the network operator, the bandwidth resource occupied by the method of the present invention is very small, and can be directly applied to the existing network without additional changes and upgrades to the existing network; from the perspective of the network customer, the transmission of the QoE interactive report It will not affect the transmission and playback of streaming media, but is conducive to the evaluation and improvement of service quality.

Furthermore, the method of the present invention has simple operation steps, low calculation difficulty, and is easy to implement. It is of great significance to the actual operation and management of streaming media services, and has good promotion and application prospects and value in the application of improving the quality of streaming media services.

Description of drawings

FIG. 1 is a flow chart of the method for monitoring quality reporting of streaming media based on user experience interaction in the present invention.

Fig. 2 is a schematic diagram of implementing the QoE protocol of the present invention.

Fig. 3 is a schematic diagram of the structure of the S-C report of the present invention.

Fig. 4 is a schematic diagram of the C-S report structure of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

The present invention is a method for monitoring and reporting streaming media quality based on user experience interaction: first, a self-defined two-way transmission quality of experience QoE interaction protocol and related network environment parameters and service performance parameters are set, the server monitors the sending status of streaming media, The client monitors the receiving status of the streaming media and evaluates its user experience quality, generates the S-C report from the server to the client and the C-S report from the client to the server, and then transmits the streaming media quality monitoring report between the client and the server through network transmission Real-time interaction between the two, so as to realize real-time monitoring of the streaming media service quality transmitted between the two.

Referring to Fig. 1, introduce each specific operation step of the inventive method:

Step 1. Set up a custom QoE interaction protocol for two-way transmission, and define the network environment parameters and service performance parameters. The QoE interaction protocol for two-way transmission stipulates that: the S-C report from the server to the client carries network environment parameters, The C-S report carries service performance parameters.

The transmission situation of the QoE interactive protocol of the self-defined two-way transmission of the present invention in the network, as shown in Figure 2, requires respectively the S-C report and the structure composition of the C-S report that server and client send are as follows:

The S-C report from the server to the client consists of a header and a data block, where the header includes:

Protocol version number (Protocol Version): the version number of the current QoE protocol, field length: 8bit, the version number in the embodiment of the present invention is 1;

Packet Type: Type of current report: 1 is S-C report, 2 is C-S report, field length: 8bit;

Packet Length: the current reported packet length, unit: byte, field length: 16bit;

The data block is used to carry one or more of the following network environment parameters:

Multimedia coding type (Code Type): the coding format information of audio and/or video data, that is, the method of converting a file of an audio and video format into another audio and video format file by using the set compression technology, field length: 16bit; common coding types of audio data are G.711, G.723, common coding types of video data are MPEG-2, H.263, H.264;

Multimedia file length (Length): the length of the audio and/or video file, unit: second, field length: 16bit;

Send Timestamp (Send Timestamp): Send the clock information of the current streaming media packet, field length: 32bit;

Packets Sent by the server (Packets Sent): the cumulative number of streaming media packets sent by the server to the client, field length: 32bit;

Sent Bytes (Bytes Sent): the cumulative number of streaming media bytes sent by the server to the client, field length: 32bit;

Send Rate: The average bit rate of streaming media service data sent by the server, unit: bit/s; field length: 32bit;

Two-way delay (Roundtrip Delay): the transmission time of the streaming media test packet from the server to the client, and then back to the server, unit: second, precision is microseconds; field length: 32bit;

Server extension field (Expand): a spare field for future protocol expansion, field length: 256bit;

The C-S report from the client to the server is composed of a header and a data block. Among them, the protocol version number included in the header, the data packet type, the field bit length and the meaning of the length are the same as those of the S-C report, and will not be described again; the data block is used to carry Describe one or more business performance parameters:

Packets Received: the cumulative number of streaming media packets received by the client, field length: 32bit;

Bytes Received: the cumulative number of stream media bytes received by the client, field length: 32bit;

Packets Loss Rate: the ratio of the number of packets lost to the number of packets sent by the server, where the number of packets lost is the difference between the number of packets sent by the server and the number of packets received, field length: 8bit;

Jitter (Jitter): The difference between the relative transmission time of two streaming media packets, that is, the difference between the receiving timestamp and sending timestamp of the two streaming media packets, unit: second, precision is microseconds; field length : 24bit;

Receive Timestamp: Receive the clock information of the current streaming media packet, field length: 32bit;

Receive Rate (Receive Rate): The average bit rate of receiving streaming media service data, unit: bit/s, field length: 32bit;

Audio quality QoA (Quality ofAudio): Use the improved E-Model algorithm based on jitter extension to evaluate the audio quality on the client side, field length: 32bit;

Video quality QoV (Quality of Video): Use the correction method based on the Chinese invention patent application "No-reference video quality objective evaluation method based on key frame image quality weighting" (application number: 201110021600.2) to evaluate video quality on the client side, field length: 32bit;

Audio and video time difference (Time Difference): the time stamp difference between a pair of originally synchronized video packets and audio packets arriving at the client in the same streaming media file, used to measure the synchronization of audio and video, unit: second, precision is microseconds; field Length: 32bit;

Connection waiting time (Time Waiting): the delay time from the user service request to the client starting to present the service, unit: second, precision is microseconds; field length: 32bit;

Client extension field (Expand): a spare field for future protocol expansion, field length: 256bit.

Step 2, the server tests, records and counts network environment parameters, generates an S-C report according to the QoE protocol, and sends it to the client. This step 2 includes the following operations:

(21) The server tests and records the multimedia encoding type, the length of the multimedia file and the sending time stamp, and at the same time counts the number of packets sent by the server P _send and the number of bytes sent B _send , and calculates the network environment including the sending rate and two-way delay according to the following formula parameters, where

式中，B″_send和B′_send分别为服务器在T″_{server，send}和T″_{server，send}两个不同时刻统计的发送字节数；sending rate

In the formula, B″ _send and B′ _send are respectively the number of bytes sent by the server at T″ _{server, send} and T″ _{server, and send} statistics at two different times;

Two-way time delay T _RD =T _{server, recv} -T _{server, send} , where, T _{server, recv} and T _{server, send} are the time when the server receives and sends the test packet respectively;

(22) The server writes the obtained network environment parameters of the streaming media service into the corresponding fields of the QoE protocol message, generates an S-C report, and sends it to the client through the network.

Step 3, the client side tests, records and counts the received streaming media information, and uses the S-C report to calculate the service performance parameters of the streaming media, and at the same time, evaluates the user experience quality of the streaming media service; then generates a C-S report according to the QoE protocol, Then send it to the server.

This step 3 includes the following operations:

(31) The client tests and records the receiving timestamp and connection waiting time, and counts the number of received packets P _recv and the number of bytes received B _recv at the same time, and uses the SC report information to calculate the packet loss rate, jitter, and receiving rate according to the following formula and the business performance parameters of audio and video time difference, among them,

式中，P_loss＝P_send-P_recv；Packet loss rate

In the formula, P _loss = P _send - P _recv ;

式中，自然数上标m、n分别为发送流媒体的两个不同数据包的序号；jitter

In the formula, the natural number superscripts m and n are respectively the sequence numbers of two different data packets for sending streaming media;

式中，B″_recv和B′_recv分别为T″_{client，recv}和T′_{client，recv}两个不同时刻统计的接收字节数；Receive rate

In the formula, B " _recv and B' _recv are respectively T" _{client, recv} and T' _{client, the number of received bytes counted by recv} at two different times;

客户端接收同一流媒体文件中原来同步的一对视频包的时刻与音频包的时刻

之差，式中，自然数上标k为流媒体文件中的数据包序号；Audio and video time difference

The moment when the client receives the original synchronized pair of video packets in the same streaming media file Moment with Audio Pack

The difference, in the formula, the natural number superscript k is the packet sequence number in the streaming media file;

(32) The client evaluates the user experience quality of the audio and/or video of the streaming media service; wherein,

The audio quality evaluation method adopts the E-Model algorithm based on jitter extension correction: QoA=93.2-I _d -I _e -I _j , that is, the jitter factor I _j is introduced to describe the impact of jitter on voice quality: where, I _d and I _e is the voice quality damage factor caused by voice signal transmission delay and packet loss in the original E-Model algorithm, and I _j is the newly added jitter influence factor for extended correction, and I _j = I _jd + I _jl ; where ,

当R_loss＞i时，a₃＝0；式中，i为丢包率的阈值，此处设i＝5％；函数f为阶跃函数，其数学表达式为：J≤a₃，f(J-a₃)＝0，J＞a₃，，f(J-a₃)＝1；该式表示：若抖动J大于a₃，会对语音质量造成严重影响；I _jd is the jitter impact factor caused by network transmission delay: I _jd =a ₁ ×J+a ₂ ×(Ja ₃ )×f(Ja ₃ ), where the parameters a ₁ and a ₂ respectively reflect network performance The empirical parameters, the embodiment is calculated according to the various parameters of the quality damage factor caused by the transmission time delay in the original E-Model algorithm: echo loudness, background noise, and loudness evaluation value: a ₁ =0.029, a ₂ = 0.194; parameter a ₃ is the threshold value for describing the severity of jitter with two-way delay, and its value depends on the packet loss rate R _loss : when R _loss ≤ i,

When R _loss >i, a ₃ =0; where i is the threshold of the packet loss rate, where i=5%; the function f is a step function, and its mathematical expression is: J≤a ₃ , f (Ja ₃ )=0, J>a ₃ , f(Ja ₃ )=1; this formula means: if the jitter J is greater than a ₃ , it will seriously affect the voice quality;

式中，b为根据不同音频编码器选取不同数值的系数，因实施例使用G.723.1.B-6.3编码器，故b＝10.3；T为缓冲区的字节容量，该实施例中的T＝1MB；c为该音频编码方式下占用缓冲区的比例系数，c＝10.81％；R_recv为接收速率；I _jl is the jitter impact factor due to buffer overflow:

In the formula, b is to select the coefficient of different numerical values according to different audio encoders, because the embodiment uses the G.723.1.B-6.3 encoder, so b=10.3; T is the byte capacity of the buffer zone, T in this embodiment =1MB; c is the proportional coefficient of the occupied buffer zone under the audio coding mode, c=10.81%; R _recv is the receiving rate;

Video quality evaluation method, using the method of Chinese patent application "No-Reference Video Quality Objective Evaluation Method Based on Key Frame Image Quality Weighting" (Application No.: 201110021600.2), and using SC report with packet loss rate, jitter and/or reception Rate correction evaluation results. Assume that the video quality score using the original patent application method is qov, and then use the packet loss rate R _loss to correct, that is, QoV=qov×(1+d×R _loss ). The video that the embodiment of the present invention tests is Foreman sequence (total number of frames is 250 frames), and this video resolution is the commonly used standardized image format QCIF (Quarter Common Intermediate Format, in the H.323 protocol cluster, stipulates the standard of video capture equipment The size of the acquisition resolution is 176×144 pixels), which belongs to the video type in which both the foreground and the background move and the movement is relatively slow, and the parameter d=10 is set.

In this step, both the audio quality evaluation method and the video quality evaluation method use a 5-point system to evaluate quality, wherein 5 points are the best and 1 point is the worst.

(33) The client writes the obtained parameters into the corresponding fields of the QoE protocol message, generates a C-S report, and then transmits it to the server through the network.

In step 4, the server correspondingly stores a pair of QoE protocol packets generated in steps 2 and 3, for subsequent analysis of the relationship between the user experience quality of the streaming media and the network environment, and providing a basis for later monitoring of the streaming media service quality. This step 4 includes the following operations:

(41) The server stores the S-C report generated in step 2 and the C-S report received in step 3 as a pair of QoE protocol messages in the database;

(42) Use the fuzzy evaluation algorithm to analyze the relationship between the quality of experience of streaming media users and the network environment, and provide a basis for monitoring the quality of mobile streaming media services in the later stage: firstly, determine various evaluation indicators including network environment parameters and service performance parameters, and each evaluation Then, the weight of each evaluation index is determined by the AHP, so as to reasonably quantify the relevant factors of the evaluation object; finally, the fuzzy membership function is constructed, and the fuzzy mathematical method is used to make classification decisions, and the user experience quality of streaming media is obtained. relationship with the network environment.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (7)

1. An implementation method based on user experience interactive streaming media quality monitoring report, characterized in that: at first the quality of experience QoE interactive protocol of self-defined two-way transmission and its related network environment parameters and business performance parameters are set, and the server monitors the streaming media The client monitors the receiving status of the streaming media and evaluates its user experience quality, generates the S-C report (Server-Client Report) from the server to the client and the C-S report (Client-Server Report) from the client to the server respectively, and then passes Network transmission realizes the real-time interaction of the streaming media quality monitoring report between the client and the server, thereby realizing real-time monitoring of the streaming media service quality transmitted between the two. 2. The method according to claim 1, characterized in that the method comprises the following steps: (1) Set up a custom QoE interaction protocol for two-way transmission, and define the network environment parameters and service performance parameters. The QoE interaction protocol for two-way transmission stipulates that: the S-C report from the server to the client carries network environment parameters, and The C-S report carries business performance parameters; (2) The server tests, records and counts the network environment parameters, and generates an S-C report according to the QoE protocol, and then sends it to the client; (3) The client side tests, records and counts the received streaming media information, and uses the S-C report to calculate the service performance parameters of the streaming media, and at the same time, evaluates the user experience quality of the streaming media service; then generates a C-S report according to the QoE protocol, and send it to the server; (4) The server correspondingly stores a pair of QoE protocol messages generated in steps (2) and (3) for subsequent analysis of the relationship between the user experience quality of streaming media and the network environment, and provides a basis for later monitoring of streaming media service quality . 3. The method according to claim 2, characterized in that: the QoE interaction protocol of the self-defined two-way transmission stipulates: (11) The S-C report from the server to the client consists of a header and a data block, wherein the header includes: the protocol version number (Protocol Version), the data packet type (Packet Type) and its length (PacketLength); the data block is used to carry the following One or more network environment parameters: Multimedia encoding type (Code Type): the encoding format information of audio and/or video data, that is, the method of converting a file of an audio and video format into another audio and video format file by using a set compression technology; The commonly used encoding types are G.711, G.723, and the common encoding types of video data are MPEG-2, H.263, H.264; Multimedia file length (Length): the length of the audio and/or video file; Send Timestamp: Send the clock information of the current streaming media packet; Packets sent by the server (Packets Sent): the cumulative number of streaming media packets sent by the server to the client; Bytes Sent: the total number of streaming media bytes sent by the server to the client; Send Rate (Send Rate): The average bit rate at which the server sends streaming media service data; Two-way delay (Roundtrip Delay): the transmission time of the streaming media test packet from the server to the client, and then back to the server; Server extension field (Expand): a spare field for future protocol extensions; (12) The C-S report from the client to the server consists of a header and a data block, wherein the header includes: protocol version number, data packet type and its length; the data block is used to carry one or more of the following business performance parameters: Packets Received: the cumulative number of streaming media packets received by the client; Bytes Received: the cumulative number of streaming media bytes received by the client; Packets Loss Rate: the ratio of the number of packets lost to the number of packets sent by the server, where the number of packets lost is the difference between the number of packets sent by the server and the number of packets received by the server; Receive Timestamp: Receive the clock information of the current streaming media packet; Jitter (Jitter): The difference between the relative transmission time of two streaming media packets, that is, the difference between the receiving timestamp and sending timestamp of the two streaming media packets; Receive Rate: the average bit rate for receiving streaming media service data; Audio quality QoA (Quality of Audio): use the E-Model algorithm based on jitter extension correction to evaluate the audio quality at the client; Video quality QoV (Quality of Video): Use the correction method based on the Chinese invention patent application "No-reference video quality objective evaluation method based on key frame image quality weighting" (application number: 201110021600.2) to evaluate the video quality at the client; Audio and video time difference (Time Difference): The time stamp difference between a pair of originally synchronized video packets and audio packets arriving at the client in the same streaming media file is used to measure the synchronization of audio and video; Connection waiting time (Time Waiting): the delay time from the user service request to the client starting to present the service; Client extension field (Expand): a spare field for future protocol extensions. 4. The method according to claim 2 or 3, characterized in that: said step (2) comprises the following operations: (21) The server tests and records the multimedia encoding type, the length of the multimedia file, and the sending time stamp, and counts the number of packets P _send and the number of bytes sent by the server B _send at the same time, and calculates the network environment including the sending rate and two-way delay according to the following formula parameters, where sending rate

In the formula, B″ _send and B′ _send are respectively the number of bytes sent by the server at T″ _{server, send} and T′ _{server, and send} statistics at two different times; Two-way time delay T _RD =T _{server, recv} -T _{server, send} , where, T _{server, recv} and T _{server, send} are the time when the server receives and sends the test packet respectively; (22) The server writes the obtained network environment parameters of the streaming media service into the corresponding fields of the QoE protocol message, generates an S-C report, and sends it to the client through the network. 5. according to the described method of claim 2 or 3 or 4, it is characterized in that: described step (3) comprises following operation content: (31) The client tests and records the receiving timestamp and connection waiting time, and counts the number of received packets P _recv and the number of bytes received B _recv at the same time, and uses the SC report information to calculate the packet loss rate, jitter, and receiving rate according to the following formula and the business performance parameters of audio and video time difference, among them, Packet loss rate

In the formula, P _loss = P _send - P _recv ; jitter

In the formula, the natural number superscripts m and n are respectively the sequence numbers of two different data packets for sending streaming media; Receive rate

In the formula, B " _recv and B' _recv are the number of received bytes counted by the client at T" _{client, recv} and T' _{client, recv} respectively at two different times; Audio and video time difference The moment when the client receives the original synchronized pair of video packets in the same streaming media file

Moment with Audio Pack The difference, in the formula, the natural number superscript k is the packet sequence number in the streaming media file; (32) The client evaluates the user experience quality of the audio and/or video of the streaming media service; wherein, The audio quality evaluation method adopts the E-Model algorithm based on jitter extension correction: QoA=93.2-I _d -I _e -I _j , that is, the jitter factor I _j is introduced to describe the impact of jitter on voice quality: where, I _d and I _e is the voice quality damage factor caused by the voice signal transmission delay and packet loss in the original E-Model algorithm, and I _j is the newly added jitter influence factor for the expansion correction, and I _j = I _jd + I _jl ; Wherein, I _jd is the jitter impact factor caused by network transmission delay: I _jd =a ₁ ×J+a ₂ ×(Ja ₃ )×f(Ja ₃ ), where the parameters a ₁ and a ₂ respectively reflect network performance The empirical parameter of , the parameter a ₃ is the threshold value that describes the severity of jitter by two-way delay, and its value depends on the packet loss rate R _loss : when R _loss ≤ i,

When R _loss >i, a ₃ =0; where i is the threshold of packet loss rate; function f is a step function, and its mathematical expression is: J≤a ₃ , f(Ja ₃ )=0, J >a ₃ ,, f(Ja ₃ )=1; this formula means: if the jitter J is greater than a ₃ , it will seriously affect the voice quality; I _jl is the jitter impact factor due to buffer overflow:

In the formula, b is a coefficient that selects different values according to different audio encoders, T is the byte capacity of the buffer, c is the proportional coefficient of the buffer occupied by different audio encoding methods, and R _recv is the receiving rate; Video quality evaluation method, using the method of Chinese patent application "No-reference video quality objective evaluation method based on key frame image quality weighting" (application number: 201110021600.2) for evaluation, and using S-C report to correlate with packet loss rate, jitter and/or reception Rate correction evaluation results; (33) The client writes the obtained parameters into the corresponding fields of the QoE protocol message, generates a C-S report, and then transmits it to the server through the network. 6. The method according to claim 5, characterized in that: in the step (31), the audio quality evaluation method and the video quality evaluation method all adopt a 5-point system to evaluate the quality, wherein 5 points are the best, and 1 point is the best. worst. 7. The method according to claim 5, characterized in that: said step (4) comprises the following operations: (41) The server stores the S-C report generated in step (2) and the C-S report received in step (3) in the database as a pair of QoE protocol messages; (42) Use the fuzzy evaluation algorithm to analyze the relationship between the quality of experience of streaming media users and the network environment, and provide a basis for monitoring the quality of mobile streaming media services in the later stage: firstly, determine various evaluation indicators including network environment parameters and service performance parameters, and each evaluation Then, the weight of each evaluation index is determined by the AHP, so as to reasonably quantify the relevant factors of the evaluation object; finally, the fuzzy membership function is constructed, and the fuzzy mathematical method is used to make classification decisions, and the user experience quality of streaming media is obtained. relationship with the network environment.

Images