CN103313054A - Transferring and scheduling method of scalable video coding (SVC) video - Google Patents

Abstract

The invention relates to a transmission scheduling method of SVC video, which utilizes the adaptability of SVC's scalability to time-varying channels, and optimizes SVC video in cognitive radio by coordinating factors that comprehensively affect subjective experience of information sources, channels and receiving ends The scheduling of transmissions in this time-varying environment of the channel optimizes the subjective quality of the video at the receiving end. According to the characteristics of the CR network, the present invention considers the status of the receiving end buffer in the SVC video transmission scheduling, and in the case of limited resources, sends those data that contribute more to improving the subjective video experience first, reducing the receiving end buffer Probability of underflow: at the receiving end, the buffer underflow probability at the receiving end is reduced through an adaptive video playback mechanism. At the same time, due to the consideration of the content characteristics of the video, the user hardly notices the adjustment of the playback rate. The present invention optimizes the subjective experience of the SVC video by comprehensively considering and coordinating factors affecting the subjective experience of the video at the sending end and the receiving end.

Description

Transmission Scheduling Method for Scalable Video Coding SVC Video

technical field

The invention relates to the field of video coding and transmission, in particular to a transmission scheduling method of SVC video in a cognitive radio environment.

Background technique

With the wide application of wireless technology in mobile communications, public safety, broadcasting and television, etc., modern society relies more and more on radio spectrum resources. At present, the demand for wireless spectrum resources is getting higher and higher, but its utilization rate is very low, and its allocation is also extremely unbalanced. Specifically, the current spectrum allocation policy allocates most of the spectrum resources to licensed band applications (such as television and broadcasting), and correspondingly, the spectrum resources in unlicensed bands are much less. In addition, the utilization rate of spectrum resources in many licensed frequency bands is very low, while most emerging wireless applications work in unlicensed frequency bands, thus making the spectrum occupancy in unlicensed frequency bands extremely crowded. Wireless spectrum is a non-renewable resource. If you want to improve the utilization rate of spectrum resources, you can only reuse spectrum resources. Therefore, cognitive radio technology (Cognitive Radio, CR) came into being in this environment.

CR is a technology that realizes spectrum dynamic allocation and spectrum sharing through spectrum sensing and system intelligent learning. Through this technology, the vacant spectrum in the licensed frequency band can be found and rationally used without affecting the normal communication in the licensed frequency band. To a certain extent, CR has improved the current low utilization rate of spectrum resources caused by the fixed spectrum allocation policy.

There are primary users and secondary users in the CR network. The secondary user detects the status of multiple channels, and can only access the channel and transmit data when the primary user is idle. This "opportunistic" access method The available bandwidth of the secondary users varies greatly. The instability of this bandwidth will cause the application on the CR channel to have high packet loss rate, delay, jitter, etc., so that the quality of service (Quality of Service, QoS) of the secondary users in the CR network is difficult. Guaranteed. Such problems are particularly prominent in video services that are sensitive to delay and jitter.

Compared with other non-scalable video coding (Non Scalable Video Coding) videos, scalable video coding (Scalable Video Coding, SVC) video is more adaptable to the extremely unstable CR network. SVC with a layered coding structure is scalable in three dimensions of time, space, and quality, and it can encode a video source into an SVC video stream containing multiple subsets, each of which is the source video A reconstruction of SVC, each subset is only different in the resolution of time, space or quality. This characteristic makes SVC have outstanding anti-packet loss ability under poor network conditions.

As far as the transmission scheduling method in the wireless environment is concerned, some current SVC videos mostly use the method that the SVC video source actively adapts to the channel conditions at the sending end or actively allocates corresponding available channel resources according to the video content. However, these methods only consider the adaptation of the video source and the channel at the sending end, but do not consider the influencing factors of the subjective video experience from a global perspective. The subjective experience of video is affected by many factors, including buffer status at the receiving end and video playback strategy.

Contents of the invention

The purpose of the present invention is to provide a transmission scheduling method of SVC video in a cognitive radio environment that can overcome the above-mentioned defects. It utilizes the adaptability of the scalable characteristics of SVC to time-varying channels, and coordinates the information source, channel and receiving end. The factors that affect the subjective experience are integrated to optimize the transmission scheduling of SVC video in the time-varying environment of the cognitive radio channel, so as to optimize the subjective quality of the video at the receiving end.

In a first aspect of the present invention, a transmission scheduling method of Scalable Video Coding SVC video is provided. The SVC video is composed of a plurality of network abstraction layer units NALU, and each NALU has a unique quality dimension and time dimension. Quality level and unique time level, the quality level is divided into a quality base layer and a plurality of quality enhancement layers, the time level is divided into a time base layer and a plurality of time enhancement layers, the method includes: according to the SVC video The state of the buffer at the receiving end determines the length of the transmission scheduling window, and the transmission scheduling window is used to schedule the NALUs to be transmitted; select the first part of the NALUs from the SVC video and place them in the transmission scheduling window, and the first part The quality level of the NALU is the quality basic level and its time level is less than or equal to the first time level threshold; if the data volume of the first part of the NALU does not reach the currently available cognitive radio channel bandwidth, select the first time level from the SVC video Two parts of NALU are placed in the transmission scheduling window, the quality level of the second part of NALU is equal to the quality basic layer and its time level is less than or equal to the second time level threshold; if the first part of NALU and the second part The amount of data of the NALU does not reach the currently available cognitive radio channel bandwidth, then select a third part of the NALU from the SVC video and place it in the transmission scheduling window, the quality level of the third part of the NALU is equal to the quality basic layer, Wherein, the span of the first part of NALU, the second part of NALU and the third part of NALU in the time dimension does not exceed the length of the transmission scheduling window; the transmission is scheduled through the currently available cognitive radio channel The NALU in the window is transmitted to the receiver of the SVC video.

In a second aspect of the present invention, a transmission scheduling controller for Scalable Video Coding SVC video is provided, the SVC video is composed of a plurality of network abstraction layer units NALU, and each NALU has a unique quality dimension and time dimension. The quality level and the unique time level, the quality level is divided into a quality basic layer and a plurality of quality enhancement layers, the time level is divided into a time basic layer and a plurality of time enhancement layers, the controller is configured to In: determining the length of the transmission scheduling window according to the state of the receiver buffer of the SVC video, the transmission scheduling window is used to schedule the NALUs to be transmitted; selecting the first part of the NALUs from the SVC video and placing them in the transmission scheduling window , the quality level of the first part of the NALU is the quality basic level and its time level is less than or equal to the first time level threshold; if the data volume of the first part of the NALU does not reach the currently available cognitive radio channel bandwidth, then from the Select the second part of NALUs in the SVC video to be placed in the transmission scheduling window, the quality level of the second part of the NALUs is equal to the quality basic layer and its time level is less than or equal to the second time level threshold; if the first part of the NALU and The data volume of the second part of NALU does not reach the currently available cognitive radio channel bandwidth, then select a third part of NALU from the SVC video and place it in the transmission scheduling window, and the quality level of the third part of NALU Equal to the quality base layer, wherein the span of the first partial NALU, the second partial NALU and the third partial NALU in the time dimension does not exceed the length of the transmission scheduling window; through the currently available cognitive radio channel Transmitting the NALUs in the transmission scheduling window to the receiving end of the SVC video.

In a third aspect of the present invention, a method for adaptive playback of scalable video coding SVC video is provided, including: determining the set of frame rates when the data in the playback buffer cannot support playing for a predetermined time at the current playback rate , each frame rate in the frame rate set is not lower than a predetermined frame rate and the absolute value of the difference with the frame rate of the previous image group does not exceed a predetermined threshold; calculate the motion intensity of the current image group and the frame rate The absolute value of the difference between the ratio of each frame rate in the set and the ratio of the motion intensity of the previous image group and its playback frame rate; in the frame rate set, select the frame rate corresponding to the minimum absolute value as the playback of the current image group frame rate.

In a fourth aspect of the present invention, there is provided an adaptive player for Scalable Video Coding SVC video, the player is configured to: the data in the playback buffer cannot support playing the video for a predetermined time at the current playback rate In this case, determine the frame rate set, each frame rate in the frame rate set is not lower than the predetermined frame rate and the absolute value of the difference with the frame rate of the previous image group does not exceed the predetermined threshold; calculate the current image group The absolute value of the difference between the ratio of the motion intensity to each frame rate in the frame rate set and the ratio of the motion intensity of the previous image group to its playback frame rate; select the frame corresponding to the minimum absolute value in the frame rate set rate as the playback frame rate of the current image group.

Aiming at the characteristics of CR network, the present invention will consider the state of buffer at the receiving end in the scheduling of SVC video transmission, and give priority to sending those data that contribute more to improving the subjective experience of video in the case of limited resources, which also reduces the buffering at the receiving end Area underflow probability; at the receiving end, we reduce the buffer underflow probability at the receiving end by adopting an adaptive video playback mechanism. At the same time, due to the consideration of the content characteristics of the video, the user hardly notices the adjustment of the playback rate. Through the comprehensive consideration and coordination of the factors affecting the subjective experience of video at the sending end and receiving end, the subjective experience of video is optimized.

Description of drawings

1 is a schematic diagram of transmission and playback of SVC video in a cognitive radio environment according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of a comparison between traditional and SVC transmission scheduling methods according to an embodiment of the present invention.

Detailed ways

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

In order to understand the present invention more thoroughly, the characteristics of the receiver buffer, the cognitive radio environment and the SVC video will be described below.

First, the function, status and related precautions of the buffer at the receiving end are introduced.

At the receiving end, the setting of the buffer helps to smooth out the jitter of packet arrival time caused by the time-varying channel. For the buffer, try to avoid too much data accumulated in the buffer, so as not to cause a large playback delay; at the same time, avoid buffer underflow, otherwise the video playback will be interrupted. Since the data from the sending end is continuously received and the video player reads video data from the playback buffer for playing, the amount of data in the buffer of the receiving end also changes accordingly. When the buffered data is enough to support the video player to play continuously for a certain period of time (such as 1 second) at the normal playback speed (such as 30fps), it can be said that the current receiving end buffer is "full"; when the buffered data cannot support the video player If the continuous playback at normal playback speed reaches this time (such as the above 1 second), it can be said that the current receiving end buffer is "hungry"; when there is no data in the receiving end buffer, this situation is called Buffer underflow.

Next, the basic environment of cognitive radio and its user relationship are introduced.

In a cognitive radio environment, there are primary users and secondary users. Primary users have their own specific channels and have access priority to their specific channels. Only when the primary user is idle, its specific channel can be accessed and transmitted by the secondary user in an opportunistic manner. When the primary user regains the channel and transmits data, the maximum time that the primary user can tolerate interference from the secondary user is T _ts , where T _ts is, for example, 50 ms. Therefore, the system time can be divided into time slots with T _ts as a unit, and the secondary user will scan the channel once in each time slot, so as to find available channels.

A secondary user may monitor N channels in the cognitive radio network, for example N=10, and access to transmit data when it finds a free channel. Suppose a time period spans Q slots, and any channel n (n=1, 2, ..., N) can provide its users in the qth (q = 1, 2, ..., Q) slot The available channel bandwidth is R(n,q). Then, the available channel bandwidth R _c that a secondary user can obtain in this period of time can be expressed as shown in formula (1), where R ^* (n,q) represents the The contribution of the bandwidth available to level users.

$R_c=\underset{q=1}{\overset{Q}{\mathrm{\Σ}}}\underset{\mathrm{no}=1}{\overset{N}{\mathrm{\Σ}}}{R}^{*}(\mathrm{no},q)$ (1)

Next, the characteristics of SVC and its transmission scheduling principle are introduced.

SVC provides scalability in three dimensions of time (Temporal), space (Spatial) and quality (Fidelity). SVC video has a basic layer and several enhancement layers in each scalable dimension. Preferably, only the scalability characteristics of the SVC in the two dimensions of time (Temporal) and quality (Fidelity) can be used in sending scheduling. For example, an SVC video source has 4 layers in the time scalable dimension (T0, T1, T2, T3, T0 are base layers, and the other 3 layers are enhancement layers), and has 3 layers in the quality scalable dimension (F0, F1, F2, F0 are base layers, and the other 2 are enhancement layers).

The encoded SVC video is organized in the form of a Network Abstraction Layer Unit (NALU), in which data blocks with the same temporal layer identifier, the same spatial layer identifier, and the same quality layer identifier are organized into a NALU. Therefore, the sending scheduling of the SVC video is actually the sending scheduling of the NALU in it. The scheduling time unit is set to T _DSU , which is numerically equal to the time required to play a Group of Pictures (GOP) at a normal frame rate. For example, if the size of a GOP (Size of GOP, SG) is set to 8 frames, the normal frame rate of its playback is 30 frames per second, and T _DSU is equal to 8/30 seconds. In the case of dividing time slots by T _ts =50 ms, the number Q of time slots spanned by T _DSU is rounded down to 5. A secondary user in a radio cognitive environment monitors a total of 10 channels simultaneously in one time slot. Under the above conditions, by substituting Q=5 and N=10 into formula (1), the available channel bandwidth that a secondary user in this T _DSU can obtain can be calculated.

The invention provides a method for scheduling transmission of SVC video in a cognitive radio environment considering user subjective experience. According to the direction of the video stream, the method specifically involves: scheduling the transmission of the SVC video at the sending end, transmitting the scheduled video to the buffer of the receiving end through the cognitive radio channel, and finally playing it in the adaptive playback module. The improvement of the present invention is mainly to avoid buffer underflow and reduce the possibility of video quality mutation on the premise of meeting the basic quality requirements of video playback. The improvement of the present invention does not consider adjusting the spatial resolution of the SVC video, but only considers its temporal scalability and spatial scalability, and the improvement is to model the cognitive radio channel from the application layer of the network protocol layer, without specifically considering the MAC layer and physical layer operations. To sum up, the scheduling principle for SVC video according to the embodiment of the present invention has three points, which are specifically described as follows.

1) The video playback meets the basic quality requirements.

In the dimension of quality scalability, the quality base layer can provide a rough video reconstruction that meets the basic quality requirements. That is, enhancement layers in the quality scalability dimension can be selectively transmitted but not necessarily transmitted. Therefore, the transmission data should be selected from the data in the quality base layer to meet the basic quality requirements of SVC video. In order to meet the basic quality requirements of video playback, the time scalable dimension of data in the quality basic layer can be selected according to the content characteristics of the video scene. In other words, a corresponding time level threshold is selected for a specific GOP, and data whose time level is greater than the time level threshold will be selectively sent, rather than necessarily sent.

Therefore, in order to select the time level threshold of a GOP, it is necessary to determine the content characteristics of the GOP. For example, Motion Intensity (MI) can be used to measure video content features, and the calculation of motion intensity MI is shown in formula (2), where _NF represents the number of frames in the current GOP and will follow the active frame number in the GOP. Discarded and changed, S _PIC represents the number of pixels in a frame, and lume(f,p) represents the brightness value of the p-th pixel of the f-th frame in the current GOP. For example, the resolution of SVC video is 352×288, which can give S _PIC =101376.

$\mathrm{<span\; class="notranslate">\; MI</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}}\underset{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; PIC</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; lume</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; lume</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

The motion of the video scene where the MI is larger is severe, while the motion of the video scene where the MI is smaller is slow. If some video frames with intense motion are actively discarded, the discontinuity of the received video content is easy to be noticed during playback; and if some video frames with slow motion are actively discarded, the discontinuity of the video content will be caused. Sex is not particularly obvious, and even the user is not aware of it. Based on this, some video frames can be selectively discarded when necessary. Dropping of video frames is achieved through the temporal scalability of SVC. In other words, by discarding video frames whose temporal levels are above a temporal level threshold without causing unacceptable degradation of the quality of the received video.

The calculation of the time-level threshold TL _ti of the i-th (i=1, 2, ..., I) GOP in a video sequence is shown in formula (3), where MI _i (TL) is TL selected as the i-th The time level threshold of a GOP is the motion intensity of the GOP, which can be calculated according to formula (2). MI _U is a sequence-dependent exercise intensity threshold that can be set by prior testing. Once the GOP motion intensity caused by actively discarding video frames is greater than the _MIU , the video continuity at the receiving end is easily perceived by the user, resulting in a decline in the subjective experience of the video. In addition, TL _H is the highest temporal level in the video, and TL _ti should not exceed TL _H .

2) Reduce the probability of buffer underflow at the receiving end, and try to avoid interruption of video playback.

3) Reduce the possibility of sudden changes in video quality caused by video transmission scheduling.

The principles to be satisfied by principles 2) and 3) are reflected in the SVC video transmission scheduling scheme, and the scheduling scheme is divided into ①, ②, and ③ to describe separately.

FIG. 1 is a schematic diagram of transmission and playback of SVC video in a cognitive radio environment according to an embodiment of the present invention.

In the following, a method for scheduling transmission of SVC video in a cognitive radio environment according to an embodiment of the present invention will be described with reference to FIG. 1 . In this scheduling scheme, the concept of sending scheduling window is adopted, and the data in the sending scheduling window will be sent in the scheduling time unit.

① Determine the length of the sending scheduling window.

The initialization value of the sending scheduling window in each scheduling time unit is equal to the size of the group of pictures (Size of GOP, SG). The SVC video transmission scheduling controller receives feedback from the receiving end to know whether the receiving buffer is currently in a full state or a hungry state. When the receiving end buffer is in a full state, the SVC video transmission scheduling module can set the upper boundary of the sending scheduling window length to be SG to keep the receiving end buffer full; when the receiving end buffer is already in a hungry state, it can be set The upper boundary of the length of the sending scheduling window is set as 2*SG, so that as many frames as possible can be sent in the scheduling time unit, so as to avoid underflow of the buffer at the receiving end. For example, if SG=8, then the upper boundary of the sending scheduling window can be set to 16.

After the upper boundary of the length of the sending scheduling window is determined, the length of the sending scheduling window should be maximized as much as possible. The longer the sending scheduling window, the more frames are sent, which means the longer the continuous playback time can be maintained at the receiving end. In this way, the purpose of reducing the probability of buffer underflow at the receiving end is achieved, thereby effectively avoiding playback interruption.

The maximum sending scheduling window is obtained under the condition of considering bandwidth limitation and basic video quality requirement. The available channel bandwidth R _c (k) in the kth scheduling time unit can be obtained by formula (1). The data sent in the scheduling time unit should not exceed the available bandwidth, otherwise serious packet loss will occur, resulting in a decline in the subjective experience of the video. In order to maximize the length of the sending scheduling window and meet the basic video quality requirements, first consider preparing the data that can meet the basic quality requirements in the sending scheduling window. The data that can meet the basic quality requirements are as described in 1).

If the total data volume of these data has not exceeded the available bandwidth in the scheduling time unit, it is further considered to prepare some enhancement layer data of time scalable dimension and enhancement layer data of quality scalable dimension in the sending scheduling window for preparation Send after the decision is made, as shown in steps ② and ③. If the total amount of data that meets the basic quality requirements exceeds the available bandwidth, the data sent earlier will be prioritized in the sending scheduling window, and steps ② and ③ will be skipped. The decision-making phase ends and the data starts to be sent.

② Increase the enhancement layer of time scalable dimension.

In order to meet the user's demand for smoother video playback, more data can be sent with the remaining available bandwidth to make the video playback smoother and the video definition higher. Smooth video playback can be achieved by sending more enhancement layers with temporal scalability dimensions.

First, an enhancement layer of temporal scalability is added to the quality base layer. In the process of preparing those quality base layer data that have not been placed in the sending scheduling window in the sending scheduling window, the NALU with a lower temporal hierarchical level has a higher priority. If there are multiple NALUs with the same temporal hierarchical level, the NALU sent earlier has higher priority. If all the quality basic layer data have been placed in the sending scheduling window and there is still remaining available bandwidth, then continue to increase the enhancement layer of the quality scalable dimension in the sending scheduling window as shown in ③, otherwise skip step ③, The decision phase ends and data is sent.

③ Increase the enhancement layer of quality scalable dimension.

If the quality base layer data has been placed in the sending scheduling window and there is still available bandwidth remaining, consider further adding the enhancement layer data of the quality scalability dimension. Since there is a dependency relationship between layers in the quality scalability dimension, the lower the level of the quality enhancement layer, the higher its selection priority. In a certain quality enhancement layer, it is still possible that the data of the enhancement layer cannot be placed in the sending scheduling window due to the limitation of the available bandwidth, so the NALU with the lower time hierarchical level in this layer is selected with a higher priority . If there are multiple NALUs with the same temporal hierarchical level, the NALU sent earlier has a higher priority. In order to avoid sudden changes in the quality of video playback, the difference in the number of quality layers between adjacent GOPs can be limited to no more than 2, so that users are less likely to perceive obvious quality changes in video playback.

Through steps ①, ② and ③, those data that can best optimize the subjective experience of video under the limitation of available bandwidth are all selected in the sending scheduling window, and the decision-making phase is completed. Subsequently, the data in the sending scheduling window will be sent through the available channels 1-10 according to their respective sending order.

When the data in the receiving buffer is full, the normal playback rate is of course the best choice. However, when the data in the receiving buffer is in a hungry state, if the normal playback rate is still used, the buffer at the receiving end may underflow due to excessive data consumption, resulting in interruption of video playback. In the latter case, an appropriate method of reducing the playback rate can be adopted, so that video playback at this rate will neither cause underflow of the buffer at the receiving end nor cause a significant decrease in the subjective experience of the video.

Reducing the playback rate at places where the video scene moves violently will cause obvious picture lag, resulting in a decline in the subjective experience of the video; reducing the playback rate at places where the video scene moves slowly, the user's subjective experience of the video will not be significantly reduced. Therefore, when the data in the buffer at the receiving end is in a hungry state, the content-based adaptive playback module needs to properly and timely adjust the video playback rate according to the motion intensity of the video scene.

For example, when the playback rate is adjusted in units of GOP, the playback rate of each GOP corresponds to the frame rate (Frame Rate, FR) of video playback. In order to reflect that the motion intensity MI and frame rate FR of the video scene analyzed in the previous paragraph should show a positive correlation, when selecting the playback frame rate corresponding to each GOP, the MI/FR value between two adjacent GOPs can be used The absolute value of the difference should be as small as possible.

In addition, in order to avoid the degradation of the subjective experience of the video due to sudden changes in the frame rate of the video playback or the frame rate is too small, it can be set that the difference between the playback frame rates of two adjacent GOPs should not exceed the threshold FD _bench , and the playback of each GOP None of the frame rates should be less than the threshold FR _low . In summary, the selection of the corresponding frame rate FR _c for each GOP is shown in formula (4). In formula (4), FR _SET represents the frame rate set (15, 16, ..., 30) available for the current GOP; MI _c and MI _p represent the motion intensity of the current GOP and the motion intensity of the previous GOP, and the motion intensity The value of MI can be calculated by publicity (2); FR _p represents the playback frame rate adopted by the previous GOP.

${\mathrm{FR}}_c=\underset{\mathrm{FR}\&Element;{\mathrm{FR}}_{\mathrm{SET}}}{\arg \min }|\frac{{\mathrm{MI}}_c}{\mathrm{FR}}-\frac{{\mathrm{MI}}_p}{{\mathrm{FR}}_p}|$ (5)

$\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; \{</span>\begin{array}{c}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; FR</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; FR</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&le;</span>{\mathrm{<span\; class="notranslate">\; FD</span>}}_{\mathrm{<span\; class="notranslate">\; bench</span>}}\\ \mathrm{<span\; class="notranslate">\; FR</span>}<span\; class="notranslate">\; \&Greater\; Equal;</span>{\mathrm{<span\; class="notranslate">\; FR</span>}}_{\mathrm{<span\; class="notranslate">\; low</span>}}\end{array}$

The present invention comprehensively considers multiple influencing factors on the subjective experience of the video, and optimizes the subjective experience of the user of the video at the receiving end. The solution of the present invention is mainly applied to non-live video applications, and all information of video sources can be prepared in advance, so the calculation complexity of the scheduling solution is low.

Fig. 2 is a schematic diagram of a comparison between traditional and SVC transmission scheduling methods according to an embodiment of the present invention.

First, it is assumed that the receiving buffer is known to be in a "starved" state through the feedback of the receiving end before the sending end schedules the transmission, and the user's available channel bandwidth in one time slot is sufficient to support sending 24 NALUs in the SVC encoded video. Next, the SVC transmission scheduling method according to the present invention will be described through the comparison and specific implementation of FIG. 2 .

Part (a) in Figure 2 shows that the traditional method is still used for sending scheduling when the buffer is in a "starved" state. As shown in part (a), the sender puts the data in the sending scheduling window (black box) for sending, and the data in the sending scheduling window is the i-th GOP. In this GOP to be sent, the NALU data belongs to the quality basic layer F0 and the quality enhancement layers F1 and F2 in the quality scalable dimension, and belongs to the time basic layer T0 and the time enhancement layers T1, T2 and F2 in the time scalable dimension. T3. The levels of the quality scalability dimension include F0, F1, and F2, and the higher quality levels, namely F1 and F2, will make the played video clearer; and the levels of the time scalability dimension include T0, T1, T2, T3, T3, T2, T3, T3, that is, the sent data only spans 8 frames in the time dimension, and there is no guarantee that the receiving end buffer can be supplemented with data to avoid buffer underflow.

Parts (b) to (e) in FIG. 2 adopt the SVC video transmission scheduling method according to the embodiment of the present invention.

In part (b), consider sending only those data that guarantee basic quality. As mentioned above, it can be known through calculation that the time level threshold of the i-th GOP is TL2. As shown in part (b), the sender puts the basic layer data that can guarantee the basic quality in the sending scheduling window, that is, puts all the basic layer data whose time dimension level does not exceed TL2 in the sending scheduling window. Since the available bandwidth of the user in the current time slot is sufficient to support 24 NALUs, after the above-mentioned basic layer data is placed in the sending scheduling window, the current available channel bandwidth can still support 20 NALUs.

In part (c), the saved available channel bandwidth can be used to increase the length of the sending scheduling window. The increased sending scheduling window is twice the length before the increasing, and the span of data to be sent in the time dimension can reach 16 frames. In addition, as mentioned above, it can be known through calculation that the time level threshold of the i+1th GOP is TL1, so all NALUs whose time level does not exceed TL1 in the i+1th GOP are placed in the sending scheduling window. Sending the data of the scheduling window can only guarantee the most basic clarity and continuity requirements of the video at the receiving end. At this time, the total amount of data in the sending scheduling window does not exceed the available channel bandwidth in the current scheduling time unit.

The numbers 1-10 between (c) and (d) indicate the priority order in which the NALUs belonging to F0 are selected into the sending scheduling window, and the smaller the number, the higher the selected priority. It can be seen that NALUs with lower time levels have higher priorities, and NALUs with the same time level have higher priorities in sending order.

In part (d), multiple NALUs with their own time levels belonging to F0 are selected to join the sending scheduling window. It can be seen that the NALUs with lower time levels and higher sending order are selected according to the sending order until all NALUs of F0 are added. until the dispatch window is sent. Undoubtedly, this increases the number of temporal layers of the SVC video, thereby optimizing the visual continuity of the video. So far, all NALU data of the basic quality layer have been prepared in the sending scheduling window, and the available channel bandwidth of the current scheduling time unit can accommodate 8 NALUs.

In part (e), the enhancement layer data is further added to the sending scheduling window. As mentioned above, there is a dependency relationship between layers in the quality scalability dimension, and the quality enhancement layer with a lower layer has a higher priority for selection. Therefore, the NALUs of the F1 layer that have higher priority than the F2 layer and the F3 layer are added to the sending scheduling window. Also as mentioned above, due to the limitation of the available channel bandwidth, the NALU with the lower temporal level in the quality enhancement layer is selected with a higher priority. Therefore, in the case that there are still 8 optional NALUs, 8 NALUs of layers T0, T1 and T2 are sequentially selected from the data in the quality enhancement layer F1 to add to the sending scheduling window. By increasing the vertical quality level in the sending scheduling window, the available channel bandwidth in the current time scheduling unit is fully utilized, and the purpose of optimizing video clarity is also achieved.

After (b) to (e) in Figure 2, the entire sending scheduling ends, and the data in the sending scheduling window will be sent to the receiving end according to its own sending order.

Professionals should further realize that the units and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the relationship between hardware and software Interchangeability. In the above description, the composition and steps of each example have been generally described according to their functions. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, within the spirit and principles of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.

Claims (10)

1. the transmission dispatching method of a scalable video SVC video, described SVC video is comprised of a plurality of network abstraction layer unit NALU, each NALU has unique quality level and unique time level in quality dimensions and time dimension, described quality layers level is divided into the basic layer of a quality and a plurality of quality enhancement layer, described time horizon level is divided into basic layer of a time and a plurality of time enhancement layer, and described method comprises:

Determine the length of transmitting and scheduling window according to the state in the rx-side buffering district of SVC video, described transmitting and scheduling window is used for dispatching NALU waiting for transmission;

Select the NALU of first to place described transmitting and scheduling window from described SVC video, the quality level of the described NALU of first is that the basic layer of quality and its time horizon level are less than or equal to very first time level threshold value;

If the data volume of the described NALU of first does not reach current available cognitive radio channel bandwidth, then select second portion NALU to place described transmitting and scheduling window from described SVC video, the quality level of described second portion NALU equals the basic layer of quality and its time horizon level is less than or equal to the second time level threshold value;

If the data volume of the described NALU of first and described second portion NALU does not reach current available cognitive radio channel bandwidth, then from described SVC video, select third part NALU to place described transmitting and scheduling window, the quality level of described third part NALU equals the basic layer of quality, wherein, the described NALU of first, described second portion NALU and the span of described third part NALU on time dimension are no more than the length of described transmitting and scheduling window;

The NALU in the described transmitting and scheduling window is transferred to the receiving terminal of SVC video by current available cognitive radio channel.

2. method according to claim 1 also comprises:

After the described NALU of first, described second portion NALU and described third part NALU are placed described transmitting and scheduling window, if also have remaining bandwidth in the current available cognitive radio channel, then select the 4th part NALU to place described transmitting and scheduling window from the SVC video, the quality level of described the 4th part NALU is quality enhancement layer and near the basic layer of described quality.

3. method according to claim 1, wherein, described very first time level threshold value and described the second time level threshold value are that the video content features of the present image group that adheres to separately by the described NALU of first and described second portion NALU calculates in advance.

4. method according to claim 1 and 2, wherein, in the process of selecting described second portion NALU, third part NALU and the 4th part NALU, the preferential relatively low NALU of select time level, the preferential forward NALU of sending order that selects in the identical situation of the time of NALU level.

5. the transmitting and scheduling controller of a scalable video SVC video, described SVC video is comprised of a plurality of network abstraction layer unit NALU, each NALU has unique quality level and unique time level in quality dimensions and time dimension, described quality layers level is divided into the basic layer of a quality and a plurality of quality enhancement layer, described time horizon level is divided into basic layer of a time and a plurality of time enhancement layer, and described controller is arranged to:

Determine the length of transmitting and scheduling window according to the state in the rx-side buffering district of SVC video, described transmitting and scheduling window is used for dispatching NALU waiting for transmission;

Select the NALU of first to place described transmitting and scheduling window from described SVC video, the quality level of the described NALU of first is that the basic layer of quality and its time horizon level are less than or equal to very first time level threshold value;

If the data volume of the described NALU of first does not reach current available cognitive radio channel bandwidth, then select second portion NALU to place described transmitting and scheduling window from described SVC video, the quality level of described second portion NALU equals the basic layer of quality and its time horizon level is less than or equal to the second time level threshold value;

If the data volume of the described NALU of first and described second portion NALU does not reach current available cognitive radio channel bandwidth, then from described SVC video, select third part NALU to place described transmitting and scheduling window, the quality level of described third part NALU equals the basic layer of quality, wherein, the described NALU of first, described second portion NALU and the span of described third part NALU on time dimension are no more than the length of described transmitting and scheduling window;

The NALU in the described transmitting and scheduling window is transferred to the receiving terminal of SVC video by current available cognitive radio channel.

6. controller according to claim 5 also is arranged to:

After the described NALU of first, described second portion NALU and described third part NALU are placed described transmitting and scheduling window, if also have remaining bandwidth in the current available cognitive radio channel, then select the 4th part NALU to place described transmitting and scheduling window from the SVC video, the quality level of described the 4th part NALU is quality enhancement layer and near the basic layer of described quality.

7. controller according to claim 5, wherein, described very first time level threshold value and described the second time level threshold value are that the video content features of the present image group that adheres to separately by the described NALU of first and described second portion NALU calculates in advance.

8. according to claim 5 or 6 described controllers, wherein, in the process of selecting described second portion NALU, third part NALU and the 4th part NALU, the preferential relatively low NALU of select time level, the preferential forward NALU of sending order that selects in the identical situation of the time of NALU level.

9. the self adaptation player method of a scalable video SVC video comprises:

Data in play buffer can't support by current playback rate and play in the situation of the scheduled time,

Determine frame per second set, each frame per second in the described frame per second set is not less than predetermined frame rate and is no more than predetermined threshold with absolute value that the previous image multicast is put the difference of frame per second;

The absolute value of the difference of the ratio of the exercise intensity of the ratio of each frame per second and previous image group and play frame rate thereof in the exercise intensity of calculating present image group and the set of described frame per second;

In described frame per second set, select frame per second corresponding to least absolute value as the play frame rate of present image group.

10. the self adaptation player of a scalable video SVC video, described player is arranged to:

Data in play buffer can't support by current playback rate and play in the situation of the scheduled time,

Determine frame per second set, each frame per second in the described frame per second set is not less than predetermined frame rate and is no more than predetermined threshold with absolute value that the previous image multicast is put the difference of frame per second;

The absolute value of the difference of the ratio of the exercise intensity of the ratio of each frame per second and previous image group and play frame rate thereof in the exercise intensity of calculating present image group and the set of described frame per second;

In described frame per second set, select frame per second corresponding to least absolute value as the play frame rate of present image group.

Images