JP2002290974A - Transmission rate control method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] In a video distribution system represented by video-on-demand, motion priority according to a user's instruction,
A transmission rate control such as image quality priority and a transmission rate control according to a change in usable transmission band are realized. SOLUTION: A video file including two or more video data encoded from the viewpoints of encoding rate variation, motion priority and image quality priority is prepared in a transmission terminal 10. Then, video data to be transmitted is dynamically switched in accordance with the user's movement / image quality priority instruction given from the receiving terminal 11 and in accordance with a change in the available transmission band.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a transmission rate control method capable of adapting to changes in usable transmission bands, transmission errors, failures, and the like.

[0002]

2. Description of the Related Art When encoding video data at a limited encoding rate, "motion (frame rate)" and "image quality (quality per frame determined by resolution and quantization step)" Are in a trade-off relationship. In the case where encoding is performed at a high frame rate so that the smoothness of motion is not impaired, that is, in the case of “motion priority”, the quality of each frame is reduced because the amount of information allocated to one frame is reduced. Will be done. Conversely, when encoding one frame at a high quality, that is, in the case of "image quality priority", the amount of information allocated to one frame increases, so the frame rate must be reduced.

[0003] Therefore, in the video data recording apparatus disclosed in Japanese Patent Application Laid-Open No. 2000-287173, an appropriate encoding parameter is transmitted to the encoder on the basis of information about the content, for example, information indicating whether the content is sports or news. The setting is automatically made, and the encoded video data is recorded. This makes it possible to achieve coding control such as motion priority and image quality priority according to the type of content.

[0004]

On an IP (Internet Protocol) network such as an intranet or the Internet, usable transmission bands differ greatly depending on the connection mode. In addition, the available transmission band varies with time due to the influence of other flows. Here, the “usable transmission band” refers to a transmission band that can be used without causing congestion between transmitting and receiving terminals. In other words,
A transmission band excluding a band used by a packet lost due to a transmission error, a failure, or the like. For example, 100k
When 10% of the packets are lost due to transmission errors in the transmission band of bps, the available transmission band is 90 kbp.
s.

[0005] In order to provide stable communication quality using such a network, the maximum value of a transmission band that can be secured in a transmission path is estimated (referred to as band estimation), and transmission is performed according to the temporal fluctuation of the band. It is necessary to change the transmission rate of data from the terminal (referred to as transmission rate control).

[0006] Even in the transmission rate control of video data in an environment where a fixed transmission band is allocated, "motion" and "image quality" have a trade-off relationship. However, conventionally, in streaming distribution of audio / video data (AV data), transmission rate control such as motion priority and image quality priority according to a user (content viewer) instruction,
In fact, transmission rates cannot be controlled in accordance with changes in usable transmission bands. Therefore, if the usable transmission band becomes narrower than the coding rate of the video data to be transmitted, the video data cannot be transmitted completely and a packet loss occurs.

An object of the present invention is to realize, in a video distribution system, transmission rate control such as motion priority and image quality priority according to a user's instruction, and transmission rate control according to a change in an available transmission band. is there.

[0008]

SUMMARY OF THE INVENTION According to the present invention, there is provided a method for converting two or more video data encoded from at least one of the following: encoding rate variation, motion priority, and image quality priority. In this case, the transmission rate of data from the transmitting terminal is controlled by dynamically switching the transmission rate in accordance with the user's instruction.

As a specific switching method, as a first example, 64 kbps, 56 kbps, 48 kbps
An AV stream encoded at a plurality of encoding rates is prepared in advance, an available transmission band is estimated, and the AV stream is switched according to the estimation result.

[0010] As a second example, when the user notifies the transmitting terminal of an instruction of motion priority or image quality priority from the receiving terminal (this may be notified in advance or may be performed during transmission of the AV stream. In the case of motion priority, the coding rate of the AV stream to be transmitted is changed. For example, if the available transmission bandwidth decreases, 64 kbps to 5
Change to 6 kbps. Further, when the available transmission band is reduced in the case of priority on image quality, the transmission rate is adjusted by the number of frames without changing the encoded AV stream.

As a third example, two types are prepared, namely, a case where an AV stream having a coding rate of 64 kbps is encoded with priority on motion and a case where encoding is performed with priority on image quality.
An AV stream to be transmitted on the transmitting terminal side is determined according to a user's instruction from the receiving terminal.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. Here, VoD (Video on Deman
d) An example of application of the present invention to streaming distribution of stored content, typified by IP broadcasting, will be described. In the embodiments of the present invention, “video data” refers to a plurality of videos encoded from at least one of the following: coding rate variation, motion priority, and image quality priority based on the same content. Refers to each of the data. Further, the “video file” refers to a file in which the plurality of video data are combined into one.

FIG. 1 shows a configuration example of a video distribution system for implementing a transmission rate control method according to the present invention. In FIG. 1, 10 is a transmitting terminal, 11 is a receiving terminal,
Reference numeral 12 denotes a video file creation device. The video file creation device 12 is a means for creating a video file containing video data to be transmitted to the receiving terminal 11. The created video file is passed to the transmission terminal 10 and stored. It should be noted that a plurality of receiving terminals are actually connected to the transmitting terminal 10, but FIG.
Only one receiving terminal 11 is shown.

In the transmission terminal 10, 100 is a transmission control unit, 101 is a transmission band estimation unit, 102 is a video data selection unit, 103 is a video data storage unit, 104 is a video data reading unit, and 105 is a video data transmission unit. .

The transmission control unit 100 receives transmission control information including instructions such as reproduction, stop, and motion / image quality priority from the receiving terminal 11, and notifies the video data selecting unit 102 of this information. Means. As a protocol to be applied, RFC 23
RTSP (Real Time Streaming Prot)
ocol), and methods such as Setup, Play, and Describing can be used (H. Schulzrinne et al., "Rea
l Time Streaming Protocol ", RFC 2326, Internet Eng
ineering Taskforce, Apr. 1998). As for the motion / image quality priority instruction, the above-mentioned video transmission control protocol may be extended, or a protocol dedicated to the motion / image quality priority instruction may be used separately. For example, each W3C
(P3P standardized by the World Wide Web Consortium)
(Platform for Privacy Preferences), CC / PP
An extension of a standard protocol such as (Composite Capabilities / Preference Profile) can be used as a protocol dedicated to motion / image quality priority instruction.

The transmission band estimation unit 101 detects congestion and transmission errors based on transmission state notifications from intermediate nodes (routers and the like) on the transmission path, the receiving terminal 11 and the like, and estimates an available transmission band. This is a means for notifying the video data selection unit 102 of the result as a transmission rate instruction. The method of estimating the usable transmission band is arbitrary, for example, LD
A (Loss-Delay Based Adjustment Algorithm) method (D. Sisalem et al., "The Loss-De
lay Based Adjustment Algorithm: A TCP-Friendly Ada
ptation Scheme ", in the proceedings of NOSSDAV'98,
July, Cambridge, UK). According to the LDA method, the data loss rate is fed back from the receiving terminal 11 to the transmitting terminal 10, and the transmission rate of the transmitting terminal 10 is controlled based on the packet loss rate, the receiving rate of the receiving terminal 11, and the like.

At the time of data transmission, the video data selecting section 102 transmits the video data to be transmitted in accordance with the usable transmission band determined by the transmission band estimating section 101 and the motion / image quality priority instruction from the transmission control section 100. And a means for notifying the video data reading unit 104 of the selection result. Further, the video data selection unit 102 includes the transmission control unit 1
It is also a means for starting / ending the video data reading unit 104 according to the video data reproduction / stop instruction from 00.

The video data storage unit 103 is a means for storing a video file containing video data to be transmitted to the receiving terminal 11, such as a hard disk drive or a removable medium. The video file is created in advance by the video file creation device 12.

The video data reading unit 104 reads the video data selected by the video data selection unit 102 from the video data storage unit 103, and sends the video data to the video data transmission unit 105.
This is means for passing the read video data to the.

The video data transmitting unit 105 is a means for receiving the video data from the video data reading unit 104, packetizing the data if necessary, and transmitting it to the receiving terminal 11. The applicable protocol is RTP (Realtime T
A protocol for data transmission typified by a transport protocol is assumed.

In the receiving terminal 11, 110 is an instruction input unit, 111 is a transmission control unit, 112 is a transmission status notifying unit, 1
13 is a video data receiving unit, 114 is a video data decoding unit,
Reference numeral 115 denotes a video display unit.

The instruction input unit 110 reproduces / displays video data.
This is an interface for the user to input a stop instruction and a motion / image quality priority instruction, and is a unit for notifying the transmission control unit 111 of the input instruction.

The transmission control unit 111 is a means for transmitting transmission control information to the transmitting terminal 10 based on a notification from the instruction input unit 110.

The transmission status notifying section 112 calculates statistical information (packet loss rate, propagation delay time, jitter, etc.) of the received video data, and notifies the transmitting terminal 10 of the calculation result as a transmission status notification. Means. The applicable protocol is RTCP (RTP Control Protocol)
It is assumed that a protocol for transmitting statistical information represented by l). Note that the transmission band estimation unit 10
If the statistical information of the receiving terminal 11 is not required for the band estimation in step 1, the transmission state notifying unit 112 is not required.

[0025] The video data receiving unit 113 is connected to the transmitting terminal 10.
Video data from the
This is a means for passing to the video data decoding unit 114.

The video data decoding section 114 is means for decoding the video data received from the video data receiving section 113 and passing the decoding result to the video display section 115.

The video display section 115 is a means for displaying data decoded by the video data decoding section 114 to a user, such as a liquid crystal display.

FIG. 2 shows an example of a video file created by the video file creation device 12 and prepared in the video data storage section 103. The video file shown in FIG. 2 has six video data 1 to 1 created based on the same content.
6 is comprised. These video data have three coding rate variations (128 kbps and 96 kbps).
ps, 64 kbps) are encoded from the viewpoint of motion priority (fixed at “frame rate = 10”) and from the viewpoint of image quality priority (fixed at “quantization step = 9”). The video data 1, 2, and 3 encoded from the viewpoint of motion priority have a smaller amount of information to be transmitted in this order. The video data 4, 5, and 6 encoded from the viewpoint of image quality priority have a smaller amount of information to be transmitted in this order. However, three or more motion-priority or image-quality-priority video data may be prepared for each encoding rate.

With respect to the encoding parameters (eg, quantization step, image size, and number of frames) of each video data, the video data after switching is selected based on the encoding parameters of the video data before switching, so that the image quality can be improved. Can be prevented from changing greatly. For example, the video data 4 to 6 in which image quality is prioritized in FIG. 2 are all coded at “quantization step = 9” and have the same image size. Therefore, when switching from the 128 kbps video data 4, the 96 kbps video data 5
Alternatively, a large change in image quality can be prevented by selecting the 64 kbps video data 6 as video data after switching and controlling the transmission rate by the number of frames per unit time.

A plurality of video data having a variation in error resilience strength may be prepared in addition to or instead of the variation in the coding rate in FIG. This is a wireless LAN (Wireless Local Area)
Network), W-CDMA (Wideband Code Division Mu)
It is assumed that video data is distributed via a wireless network such as ltiple Access) or FWA (Fixed Wireless Access), and if the available transmission bandwidth decreases due to transmission errors or failures in the wireless network, By transmitting video data having a higher error resilience strength, it is possible to prevent video degradation at the receiving terminal 11. As a method of adding error resilience, there is a method of adding redundant data described in RFC 2733 (J. Rosenberg et al., "An RTP Payload For
mat forGeneric Forward Error Correction ", RFC 273
3, Internet Engineering Taskforce, Dec. 1999). As an encoding method, MPEG (Moving Picture Coding) is used.
If Experts Group 4 is used, HEC (Hea
AIR (Adaptive I) with der Extension Code
For example, shortening the cycle of the ntra refresh), shortening the packet length, and shortening the insertion cycle of the I-VOP (Intra-Video Object Plane). When the error tolerance is provided, the frame rate and the image quality of the video data must be reduced by the amount of the error tolerance. Therefore,
Even when the error resilience strength is changed, it is necessary to determine which of the motion and the image quality is prioritized in consideration of a user instruction, a type of content, an available transmission band, and the like.

FIG. 3 is a sequence diagram showing the operation of the video distribution system of FIG. First, the video file creation device 12 creates the video file of FIG. 2 from a certain content, and stores it in the transmission terminal 10 before transmitting / receiving a video (Step 300). Subsequently, the receiving terminal 11 transmits a motion / image quality priority instruction before transmitting the video data (step 301). In the illustrated example, it is assumed that the user has selected the movement priority. Subsequently, the receiving terminal 11 is the transmitting terminal 1
0, a reproduction instruction (video data transmission request) is transmitted, and the transmitting terminal 10 starts transmitting the video data specified by the motion / image quality priority instruction (step 302). In the example shown in the figure, video data 1 of motion priority and 128 kbps is transmitted. The receiving terminal 11 periodically notifies the transmission state, and the transmitting terminal 10 estimates an available transmission band based on the notification (step 303). The thick broken line in FIG. 3 indicates that the usable transmission band is from 128 kbps to 96 kb.
It represents that it has changed to ps. The transmitting terminal 10
When it is detected that the usable transmission band is narrowed, the video data to be transmitted can be transmitted within the usable transmission band.
kbps video data 2 (step 30)
4). At this time, the motion / image quality priority instruction from the user is followed. In the illustrated example, the motion / image quality priority instruction is issued before the transmission of the video data. However, the motion / image quality priority instruction may be transmitted during the transmission of the video data.

By the way, when a format for collecting each video data as an individual access unit is applied to the storage format of the video data, the transmission time must be checked in order from the top of the video data every time the video data to be transmitted is switched. Have to be. This is because it is necessary to start transmission of the video data after switching from the continuation of the video data before switching. Such an operation makes smooth file switching difficult. For example, in the case of long-time content, if video data switching occurs near the end time of the content, the transmission time of all data from the beginning of the video data to the vicinity of the end of the video data is checked. The time becomes longer, and the video stops for the processing time.

Therefore, two or more pieces of video data to be transmitted (or received / reproduced) at the same time
By recording in a video file as one access unit, it is possible to smoothly switch video data generated by a change in usable transmission band or a user's instruction.

FIG. 4 illustrates a specific format of the video file of FIG. According to the format of FIG. 4, the video file includes a header 400 and areas 401 and 40 of data Tn (n = 1, 2,...) For each transmission time.
And 2.

The header 400 contains the number N of video data stored in the video file (N = 6 in the example of FIG. 2).
Is stored. Next, properties of each video data are stored. The property in this case includes information indicating whether each piece of video data has been encoded in the motion priority or the image quality priority, and the encoding rate.

In the area 401 of the data T1, a transmission time t1, a transmission flag F1, and a total data length L1 are sequentially stored. The transmission time t1 is a time at which any video data portion included in the area 401 is to be transmitted. The transmission flag F1 is a flag indicating the number of video data to be transmitted at the transmission time t1. For example, if N = 6 and six video data are stored in the video file, and any one of the video data 1, video data 2, and video data 5 is transmitted at time t1, F
1 = “110010”. That is, the transmission flag F
The i-th bit from the beginning of 1 (i = 1 to 6) is
It indicates whether the video data i is data transmitted at time t1. If the value is “1”, it indicates that the data is transmitted. If the value is “0”, the data to be transmitted is transmitted. Indicates that it is not. The total data length L1 indicates the length of the remaining portion in the area 401. Following this total data length L1, the video data j transmitted at time t1
Are sequentially stored (e.g., j = 1, 2, 5). The video data j stored here is video data in which the corresponding bit of the transmission flag F1 is 1. Area 402 of data T2
The subsequent file structure is the same as that of the area 401.

Using the video file format shown in FIG. 4, how the video data selecting unit 102 and the video data reading unit 104 in FIG. 1 process video data will be described below.

FIG. 5 is a block diagram showing the video data selection unit 102 shown in FIG.
The operation of FIG. The operation in FIG. 5 is performed when the video data selection unit 102 receives a notification from the transmission control unit 100 or the transmission band estimation unit 101. First, if the received instruction is a reproduction instruction, the video file is opened, and the number of video data N
Based on the number N, information Pd (I) indicating whether each of the video data I is encoded with motion priority or image quality priority for I = 1 to N, and The coding rate Rd (I) is read and stored (step 501). Subsequently, the initial value of the transmission rate R is appropriately set, and the video data reading unit 104 is activated (Step 502). For example, the encoding rate of the video data 1, that is, Rd (1) is set as the initial value of the transmission rate R. Subsequently, video data to be transmitted is selected (step 503). Here, the number of the selected video data is Dt. The selection method will be described later. Finally,
The change of the video data is notified to the video data reading unit 104, and the operation ends (step 504).

On the other hand, in the flow of FIG. 5, if the received instruction is a motion / image quality priority instruction,
It is stored in ri (step 505). When the transmission rate instruction is received from the transmission band estimation unit 101, the instructed transmission rate is stored in R (step 506). After performing the processes of steps 505 and 506, it is determined whether or not the video data reading unit 104 is being activated. If the video data reading unit 104 is being activated, steps 503 and 504 are performed. 507). If the received instruction is a stop instruction, the video data reading unit 104 is stopped, the video file is closed, and the operation ends (step 508).

FIG. 6 shows the details of step 503 for determining the number of the video data to be transmitted in FIG. First, the reference coding rate Rs is initialized to 0. And
It is checked whether the information Pd (I) of the video data I matches the instruction content Pri for motion / image quality priority. If not, the process proceeds to the next video data check. Proceed to check (Step 60)
1). If the coding rate Rd (I) of the video data I is not higher than the specified transmission rate R and higher than the coding rate of the video data checked in the past, that is, the reference coding rate Rs, the video The number of the data I is stored as the number Dt of the video data to be transmitted, the reference encoding rate Rs is updated, and the process returns to the step 601 (step 602). By performing this for all the video data in the video file, the video data having the maximum transmission rate within a range not higher than the specified transmission rate R can be selected as the video data to be transmitted. it can.

FIG. 7 shows the video data reading unit 1 in FIG.
04 is shown. Video data reading unit 104
Reads the required video data by reading the continuation of the video file opened by the video data selection unit 102 in the following steps. First, the video data reading unit 1
04 stores the number D of the video data to be read when the video data change notification is received from the video data selection unit 102, and transmits the transmission time tn from the video file when the video data change notification is not received. , The transmission flag Fn and the total data length Ln are read (step 701).
Then, based on the transmission flag Fn, the video data D is changed to the time tn.
Is determined if there is no data to be transmitted, that is, D from the beginning of the transmission flag Fn.
If the bit is “0”, the total data length Ln
The data is skipped until the next transmission time based on the value of (step 702). If there is data to be transmitted, the steps of reading the data length Lg and skipping the data Data are repeated until the data to be transmitted is reached, and then the target data is read (step 70).
3). Then, it waits until the transmission time tn is reached (step 704). At the transmission time tn, the data is passed to the video data transmission unit 105, and if the stop instruction is received or the end of the video file is reached, the operation is terminated; otherwise, the process returns to step 701 (step 701). 705).

As described above, by using the video file format of FIG. 4, even when the video data to be transmitted is switched, a process of searching for the video data portion corresponding to the next transmission time from the beginning of the video file is performed. Eliminates the need.

In the embodiment of the present invention, two types of video data of motion priority and image quality priority are prepared for each coding rate as described in the description of FIG.
Only one type of video data may be prepared for each encoding rate. For example, only the motion priority video data 1 to 3 in FIG. 2 are prepared as video files. Then, in the case of the motion priority, the video data to be transmitted is switched when the available transmission band decreases or increases. Further, when the available transmission band decreases or increases in the case of priority on image quality, the transmission rate is adjusted by thinning out frames of the video data without switching the video data to be transmitted.

Further, instead of determining the motion / image quality priority based on the user's instruction, video data encoded in either the motion / image quality priority may be prepared based on the type of the content. For example, if the content is sports, only motion-priority video data is prepared, and if the content is news or movie, only image-priority video data is prepared, and transmitted according to a change in available transmission band. The video data to be switched. This makes it possible to switch between motion / image quality priority according to the type of content when the usable transmission band is narrowed.

Further, the present invention is applicable to both wired and wireless networks.
Further, the present invention can be applied not only to the unicast but also to a multicast video distribution system. Further, the present invention can be applied to a video distribution system using a transmission path of not only a narrow band but also a broad band. Even in the case of a wide band, it is expected that the transmission content will have high quality and the amount of information to be transmitted will increase accordingly, so that transmission rate control is still necessary.

[0046]

As described above, according to the present invention, in a video distribution system, regardless of a wired network or a wireless network, transmission rate control such as motion priority and image quality priority in accordance with a user's instruction, and use of a video signal can be performed. Transmission rate control according to a possible change in the transmission band becomes possible.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration example of a video distribution system for implementing a transmission rate control method according to the present invention.

FIG. 2 is an explanatory diagram illustrating a video file prepared in a video data storage unit in FIG. 1;

FIG. 3 is a sequence diagram showing an operation of the video distribution system of FIG. 1;

FIG. 4 is an explanatory diagram illustrating a specific format of the video file of FIG. 2;

FIG. 5 is a flowchart illustrating an operation of a video data selection unit in FIG. 1;

FIG. 6 is a flowchart showing details of steps for determining the number of video data to be transmitted in FIG.

FIG. 7 is a flowchart illustrating an operation of a video data reading unit in FIG. 1;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Transmission terminal 11 Receiving terminal 12 Video file creation device 100 Transmission control unit 101 Transmission band estimation unit 102 Video data selection unit 103 Video data storage unit 104 Video data reading unit 105 Video data transmission unit 110 Instruction input unit 111 Transmission control unit 112 Transmission Status notification unit 113 Video data receiving unit 114 Video data decoding unit 115 Video display unit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04L 29/08 H04N 7/13 Z H04N 5/76 H04L 13/00 307C (72) Inventor Junichi Sato Osaka 1006 Kadoma Kadoma, Matsushita Electric Industrial Co., Ltd. (72) Inventor Hiroshi Arakawa 1006 Kadoma Kadoma, Osaka Prefecture F term (reference) 5C052 AA01 CC11 CC12 DD04 EE10 5C059 KK34 PP04 RB02 RF04 SS08 TA17 TC21 TC45 TD13 UA02 5D044 AB07 CC04 DE11 DE17 DE49 GK07 HL06 HL11 5K014 FA12 HA05 5K032 AA05 CC02

Claims (7)

[Claims] 1. A method according to claim 1, wherein two or more pieces of video data coded from at least one viewpoint among coding rate variation, motion priority, and image quality priority are changed according to a change in an available transmission band or specified by a user. A transmission rate control method characterized in that the transmission rate is controlled by dynamically switching according to the transmission rate. 2. The transmission rate control method according to claim 1, further comprising a step of preparing a video file including the two or more video data created in advance based on the same content. Transmission rate control method. 3. The transmission rate control method according to claim 2, wherein the preparing step includes, as one access unit, two or more video data portions to be transmitted at the same time in the video file. A transmission rate control method, comprising a step of recording. 4. The transmission rate control method according to claim 2, wherein the preparing step includes a step of recording a plurality of video data having different error resilience strengths from each other in the video file. Transmission rate control method. 5. The transmission rate control method according to claim 1, wherein an available transmission band is estimated, and one of the two or more video data is selected according to the available transmission band. And transmitting the selected video data. 6. The transmission rate control method according to claim 1, further comprising: receiving a motion / image quality priority instruction from a user; and selecting one of the two or more pieces of video data according to the motion / image quality priority instruction. A transmission rate control method, comprising: selecting one of the video data; and transmitting the selected video data. 7. The transmission rate control method according to claim 1, wherein, based on an encoding parameter of video data before switching,
A transmission rate control method comprising a step of selecting video data after switching.