CN100515068C - Static frame loss method in video playing - Google Patents

Abstract

The invention discloses a method for static frame loss in video playback, which is used for selectively discarding video frames received by a video terminal during video playback, including: predicting the video decoding capability of the video terminal by using a benchmark test code stream; Analyze the input video frame sequence received by the terminal to estimate the decoding complexity of the video frame sequence; calculate the target playback frame rate; determine whether static frame loss is required, and if necessary, perform the next step; otherwise, end the operation; calculate the frame loss percentage ; Determine the frame loss frequency M in the static frame loss filter in the video terminal by using the frame loss percentage; implement the static frame loss process. The invention has the advantages of: overall reducing the complexity of video playing; balancing the decoding pressure of the whole playing process; avoiding the discontinuity of video playing and reducing the pause of video display; ensuring the smoothness of video playing and display.

Description

Static drop frame method in video playback

technical field

The present invention relates to a video playing terminal, in particular to a video playing terminal with limited multimedia processing capability.

Background technique

Currently, the video frames played by the video playback system are divided into I frame, P frame and B frame, wherein the I frame and the P frame are reference frames, and the B frame is a non-reference frame. Among the reference frames, the I frame is a key frame, and the P frame is an ordinary reference frame. When decoding a video frame, the I frame can be decoded relying on the information contained in itself, the P frame needs the information of an I frame or another P frame to realize its decoding, and the B frame needs an I frame and a P frame, or two P frames to achieve its decoding.

In the video playback system, such a problem is often encountered: the average decoding complexity of the video sequence input to the playback terminal exceeds the processing capability of the terminal. When the decoder decodes a few frames of video images, the current time of the system lags behind. Due to the normal display time of the video frame, if the frame-by-frame decoding continues, the lag of the video image display will be further aggravated, causing the user to see a slow-play video.

At present, there are generally two methods to deal with this problem: one is to perform best-effort decoding on the input video sequence, if the current system time has lagged behind the input video frame to be decoded, then discard the video frame without decoding; the other is On the basis of method 1, the consideration of frame type is added, that is, if the discarded frame is a B frame, the subsequent frames will continue to be decoded; if the discarded I frame or B frame is discarded, the subsequent frames will be continuously discarded until the next I frame arrives . These two methods can try to keep the display time of the decoded video image close to the display time required by the video image, and have been widely used in currently commonly used players, such as Windows Media Player, Real Player, etc. However, if method 1 is used, when the discarded frame is a reference frame, errors will occur in the decoding of subsequent frames, and the video quality after decoding will be reduced; if method 2 is used, multiple video frames need to be discarded continuously, which may easily cause playback The discontinuity and pause will affect the user's viewing effect. Therefore, how to reduce the pressure of video decoding and avoid display pauses caused by uneven frame loss when the terminal processing capability is limited has practical application value.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings of the prior art on terminals with insufficient multimedia processing capabilities, such as continuous frame loss during video playback, which causes frequent pauses in the picture and affects users' viewing, reduces the video decoding pressure of the terminal, and improves the fluency of video playback , to improve the user's viewing effect; thereby providing a method for selectively discarding video frames received by a video terminal during video playback.

In order to achieve the above object, the present invention provides a static frame loss method in video playback, which is used for selectively discarding video frames received by a video terminal during video playback, comprising the following steps:

1), using the benchmark test code stream to predict the video decoding capability of the video terminal, and obtain the three parameters of reference frame length, reference frame width and reference frame frequency;

2), analyzing the input video frame sequence received by the video terminal, estimating the decoding complexity of the video frame sequence, and obtaining three parameters of input frame length, input frame width, and input frame frequency;

3), according to step 1) the video decoding ability that obtains and step 2) the decoding complexity of the video frame sequence that obtains, calculates target playback frame rate;

4), compare the target playback frame rate and the input frame rate to determine whether static frame loss is required, if the input frame rate is greater than the target playback frame rate, perform a static frame loss operation and execute the next step, otherwise, end the operation;

5), calculate the frame loss percentage according to the target playback frame rate obtained in step 3);

6), using the frame loss percentage to determine the frame loss frequency M in the static frame loss filter;

7), a counter is set at the video terminal, and the initial value of the counter is 0;

8), the video terminal receives the video frame sequence, and compares the value of the counter with the frame loss frequency M in the static frame loss filter;

If the value of the counter is less than M, then the current input frame is not discarded, and the input video frame is directly sent to the decoder for decoding;

If the value of the counter is equal to M, then judge the type of the current input frame, if the input frame is a B frame, then discard the frame, and clear the value of the counter, if it is an I frame or a P frame, then do not discard the frame, and Increment the counter by one;

If the value of the counter is greater than M, judge the type of the input frame, if the input frame is a B frame, then discard the frame, and set the value of the counter as the difference between the value of the counter and M, if it is an I frame or a P frame, then The frame is not discarded, and the counter is incremented by one.

In above-mentioned technical scheme, in described step 3) in, the calculation of described target playback frame rate is:

Target playback frame rate=reference frame length×reference frame width×reference frame rate/input frame length/input frame width.

In said step 5), the calculation of said frame loss percentage is:

Frame loss percentage = 1—target playback frame rate/input frame rate.

In the step 6), the frame loss frequency M is an integer, and should satisfy the formula at the same time:

|Frame loss percentage×M—1|<|Frame loss percentage×(M-1)—1|

|Frame loss percentage×M—1|<=|Frame loss percentage×(M+1)—1|.

The static frame dropping method in the video playback of the present invention has the following advantages:

(1) The present invention reduces the complexity of video playback as a whole;

(2) The present invention fully considers the decoding capability of the terminal and the decoding complexity of the input video, and balances the decoding pressure of the entire playback process;

(3) The present invention avoids the discontinuity of video playback, and reduces video display pauses;

(4) The frame loss implementation carried out by the present invention according to the frame loss filter and video frame type not only maintains the frame loss percentage as a whole, but also ensures the independence of the discarded frames, does not affect the decoding of other frames, and ensures the smoothness of video playback and display. fluency.

Description of drawings

Fig. 1 is a schematic diagram of a set-top box application scenario of the present invention;

Fig. 2 is the position of the static drop frame filter in the video player;

Fig. 3 is the determination process of the static drop frame filter;

Fig. 4 is the flowchart of the implementation process of static frame loss in the method of static frame loss in video playback of the present invention;

FIG. 5 is a flow chart of the static frame dropping method in video playback according to the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

The static frame dropping method in video playback of the present invention can be applied in the scene shown in FIG. 1 . The set-top box sends a video service request to the streaming media server through the broadband network. The streaming media server responds to the request and sends video data to the set-top box through the network. After receiving the video data, the set-top box decodes it and sends it to the TV for display. In this scenario, the network bandwidth can meet the transmission requirements of the video stream, and the network receiving module of the set-top box is also capable of receiving data, but the video decoding capability of the set-top box is insufficient to meet the real-time decoding of all data of the video stream. The static frame dropping method of the present invention is suitable for the situation where the decoding capability of the video terminal is insufficient.

In order to achieve the effect of smooth video playback, the static frame dropping method proposed by the present invention is adopted in the video player of the set-top box. Described static drop frame method is mainly realized by static drop frame filter, and the position of static drop frame filter in video player is as shown in Figure 2: Compressed video sequence is first processed by static drop frame filter, then sent to The video decoder obtains a decoded video sequence, and the decoded video sequence is rendered to form a display video sequence.

As shown in Figure 5, the static frame loss method in this embodiment specifically includes the following steps:

1. Predict the decoding capability of the video terminal, use the benchmark test code stream to test the video playback capability of the set-top box, and set the video frame length, video frame width and frame rate of the test code stream when the set-top box is playing at full capacity as: benchmark frame length, base frame width, and base frame frequency. The decoding capability of the video terminal can be reflected by the above three parameters. For example, through the test of the reference code stream, the decoding capability parameters of the set-top box used in this embodiment are: 352×288×30, that is, the reference frame length is 352, the reference frame width is 288, and the reference frame frequency is 30.

2. Analyzing the input video frame sequence received by the set-top box, and estimating the decoding complexity of the video frame sequence. By analyzing the video frame sequence, the input frame length, input frame width, and input frame frequency can be known. For example, the parameters of the video sequence played in this embodiment are: 512×384×25, that is, the input frame length is 512, the input frame width is 384, and the input frame frequency is 25.

3. Using the decoding capability of the video terminal obtained in step 1 and the decoding complexity of video sequence frames obtained in step 2, calculate the target playback frame rate of the input sequence. The calculation of the target playback frame rate is shown in formula (1):

Target playback frame rate = base frame length x base frame width x base frame rate/input frame length/input frame width(1)

Substituting each parameter in this embodiment, then

Target playback frame rate = 352×288×30/512/384=15.5

4. Compare the target playback frame rate with the input frame rate to determine whether static frame loss is required. In this embodiment, since the target playback frame rate (15.5) is lower than the input frame rate (25), static frame dropping is required.

5. Calculate the frame loss percentage by the target playback frame rate obtained in step 3, and the calculation formula of the frame loss percentage is as shown in formula (2):

Frame loss percentage = 1—target playback frame rate/input frame rate (2)

In this embodiment, frame loss percentage=1—15.5/25=38%

6. Using the frame loss percentage to determine the frame loss frequency M in the static frame loss filter, M means that one frame needs to be discarded for every M frames input. In order to facilitate the implementation of frame loss, M must be an integer, and should satisfy both formula (3) and formula (4):

|Frame loss percentage×M—1|<|Frame loss percentage×(M-1)—1| (3)

|Frame loss percentage×M—1|<=|Frame loss percentage×(M+1)—1| (4)

The M value can be determined by enumeration method, starting from the value 1, and substituting the values from 1 to (input frame rate - 1) into the above formula one by one, and encountering the values satisfying the formulas (3) and (4) or the values are (input frame rate-1) and stop, and determine the value as M.

In this embodiment, the frame loss percentage (38%) is substituted into (3) (4), and the satisfying M value of 3 can be obtained by using the above enumeration method.

7. Set a counter, the initial value of the counter is 0, and the value of the counter increases with the input of the video frame.

8. Compare the value of the counter with the M value 3 in this embodiment, as shown in Figure 4, there are three situations:

1), if the value of the counter is less than 3, the input frame is not discarded, and the input video frame is directly sent to the decoder for decoding;

2), if the value of the counter is equal to 3, then judge the type of the input frame, if the input frame is a B frame, then discard the frame, and clear the value of the counter, if it is an I frame or a P frame, then do not discard the frame , and add one to the counter;

3), if the value of the counter is greater than 3, judge the type of the input frame, if the input frame is a B frame, then discard the frame, and set the value of the counter as the difference between the value of the counter and M, if it is an I frame or a P frame, the frame is not discarded, and the counter is incremented by one.

Claims (1)

1, the static frame loss method in a kind of video playback may further comprise the steps:

1), utilize the video decoding capability of benchmark test code stream prediction video terminal, obtain the benchmark frame length, reference frame is wide and three parameters of benchmark frame frequency, and calculate decoding capability parameter=benchmark frame length * reference frame wide * the benchmark frame frequency;

2), the input video frame sequence that video terminal received is analyzed, estimate the decoding complex degree of sequence of frames of video, obtain importing frame length, incoming frame is wide and input frame frequency three parameters, and calculate decoding complex degree=input frame length * incoming frame wide * import frame frequency;

3), the video decoding capability and the step 2 that obtain according to step 1)) decoding complex degree of the sequence of frames of video that obtains, calculate target and play frame frequency;

Described target is play being calculated as of frame frequency:

That target is play frame frequency=benchmark frame length * reference frame is wide * and benchmark frame frequency/input frame length/incoming frame is wide;

4), comparison object plays frame frequency and input frame frequency, judges whether to carry out static frame loss, if the input frame frequency is play frame frequency greater than target, then does the static frame loss operation, carries out next step, otherwise, end operation;

5), the target that obtains according to step 3) is play frame frequency and is calculated frame losing percentage;

Frame losing percentage=1-target is play frame frequency/input frame frequency;

6), utilize frame losing percentage to determine frame losing frequency M in the static frame loss filter in the video terminal, described frame losing frequency M round numbers, and should satisfy following formula simultaneously:

| frame losing percentage * M-1|＜| frame losing percentage * (M-1)-1|

| frame losing percentage * M-1|＜=| frame losing percentage * (M+1)-1|;

7), at video terminal a counter is set, the initial value of this counter is 0, and the value of counter increases along with the input of frame of video;

8), video terminal receiver, video frame sequence, and the value of described counter and the frame losing frequency M in the described static frame loss filter compared;

If the value of counter does not then abandon current incoming frame less than M, directly input video frame is sent in the decoder and decode;

If the value of counter equals M, then judge the type of current incoming frame, if incoming frame is the B frame, then abandons this frame, and,, and the value of counter is added one if I frame or P frame then do not abandon this frame the value zero clearing of counter;

If the value of counter greater than M, is judged the type of incoming frame, if incoming frame is the B frame, then abandons this frame, and the value of counter is changed to the value of counter and the difference of M, if I frame or P frame then do not abandon this frame, and the value of counter is added one.

Images