JP7565733B2 - STREAM TRANSMISSION DEVICE, STREAM GENERATION DEVICE, AND PROGRAM - Google Patents

Description

The present invention relates to a stream sending device, a stream generating device, and a program, and in particular to a stream sending device, a stream generating device, and a program that can accommodate increases and decreases in network bandwidth (transmittable bit rate).

In recent years, content including ultra-high definition video images such as 4K and 8K has been put into practical use. This content is encoded and stored in advance, and is then sent out over a network or other means by a stream sending device. Note that content includes multiple media such as video and audio, but since this invention is directed to video encoding technology, the following description will only focus on video.

Various compression coding technologies have been put into practical use for recording and transmitting video. For example, a coding method is used that realizes highly accurate prediction by simultaneously referring to past and future pictures by rearranging the coding order (Non-Patent Document 1). Also, depending on the reference method, each picture may be called an I (Intra Coded) picture, which is a picture that can be decoded independently, a P (Predictive Coded) picture, which is a picture that uses forward motion compensation prediction, or a B (Bi-directional Predictive Coded) picture, which is a picture that uses bi-directional motion compensation prediction.

In order to explain the technology in this specification, some terms and functions are defined. An image (picture, frame, etc.) is called an "AU (Access Unit)". An "IRAP (Intra Random Access Point) AU" refers to an AU that can be decoded normally when the decoding process is started from that AU, as well as the AUs following that AU in the display order. A fragment of an encoded stream including an IRAP AU to the AU immediately preceding the next IRAP AU is called a "GOP (Group of Pictures)". This GOP is the same as the concept generally called GOP in the field of video coding. In addition, the "GOP structure" refers to the reference modes such as I, B, P of each AU in the GOP and the structure of the AUs that can be referenced. This is also the same as the concept generally called "GOP structure" in the field of video coding. In this specification, the GOP length of a certain GOP is expressed functionally as "len(GOP)" and the GOP structure as "struct(GOP)". Additionally, the bit length of a certain AU is expressed as "strlen(AU)".

"Two streams have the same GOP structure" means that when the multiple GOPs that make up one stream are GOP A[0], GOP A[1], GOP A[2], ..., and the multiple GOPs that make up the other stream are GOP B[0], GOP B[1], GOP B[2], ..., the GOP structures of GOP A[x] and GOP B[x] are the same for all x. Also, AU[x] refers to the xth AU in the coding order of the GOP of interest (0<=x< len(GOP), "<=" has the above meaning).

In general, a stream sending device encodes and stores the same content at multiple bit rates in preparation for a reduction in the network bandwidth used for transmission, and normally sends out an encoded stream encoded at a high bit rate. When congestion occurs in the network, for example, and the available network bandwidth is reduced, the device switches to sending out an encoded stream encoded at a low bit rate.

Figure 9 is a conceptual diagram of an encoded stream stored in a conventional stream sending device 1. "Stream output" is the stream sent by device 1. Stream sending device 1 is provided with a stream encoded at a high bit rate (hereinafter referred to as a "high bit rate stream") and a stream encoded at a low bit rate (hereinafter referred to as a "low bit rate stream"). Note that the thickness of the stream in the figure corresponds to the bit amount (amount of information). The GOPs that make up the high bit rate stream are, from the beginning, GOP H[0], GOP H[1], GOP H[2], ..., and the GOPs that make up the low bit rate stream are, from the beginning, GOP L[0], GOP L[1], GOP L[2], ....

Figure 10 is a conceptual diagram of bit rate switching in a conventional stream transmission device 1. A "bit rate change instruction" is a signal given to the device from outside, instructing it to change the bit rate. If network congestion occurs during transmission of a high bit rate stream (for example, during transmission of GOP H[1]) and switching to a low bit rate becomes necessary (when a bit rate change instruction is received), after transmission of GOP H[1] ends, the low bit rate stream is transmitted from GOP L[2]. This is because, in general, it is impossible to correctly decode an encoded stream from any AU other than the IRAP AU, which is the first AU of a GOP.

Sakae Ohkubo (editor), Teruhiko Suzuki, et al., "H.265/HEVC Textbook," published by Impress Japan Co., Ltd., October 21, 2013, pp. 236-241

When network bandwidth increases or decreases, it is desirable to switch bit rates as soon as possible. However, because it is impossible to correctly decode an encoded stream from anywhere other than an IRAP AU, switching in the middle of a GOP poses the problem that video distortion occurs on the receiving side immediately after switching. To prevent this distortion, it is necessary to switch the stream to be sent by IRAP AU (i.e., by GOP), but in this case, there is the problem that switching to a stream with a different bit rate is delayed until the next IRAP AU appears. Alternatively, if all AUs are encoded with IRAP AUs (i.e., all-intra encoding), it is possible to switch streams at any AU without causing video distortion on the receiving side, but in this case, there is the problem that the bit rate of the entire stream becomes high.

Therefore, in consideration of the above problems, the object of the present invention is to provide a stream sending device that can switch between streams encoded at different bit rates at any AU and send them without causing any disturbance in the video on the receiving side, as well as a stream generating device that generates the stream for this purpose, and a program.

In order to solve the above problems, the stream sending device according to the present invention is a stream sending device having a function of switching and sending out a plurality of encoded streams encoded at different bit rates, the stream sending device comprising: a first bit rate stream encoded at a first bit rate; and a second bit rate stream having the same GOP (Group of Pictures) structure as the first bit rate stream and encoded at a second bit rate, and a stream sending device for switching and sending out an AU (Access Unit) of the i-th (0<i<GOP length) GOP of the X-th (X is arbitrary) GOP of the first bit rate stream. When switching from the Xth GOP, the first bitrate stream is switched to the second bitrate stream from the Xth GOP (Unit), using a transition GOP that has the same GOP structure as the Xth GOP, and the first AU to the i-1th AU are encoded to obtain the same decoded result as the first AU to the i-1th AU of the Xth GOP of the first bitrate stream, and the i-th and subsequent AUs are encoded at a bitrate closer to the second bitrate than the first bitrate, and the first bitrate stream is sent up to the i-1th AU of the Xth GOP, and then the i-th and subsequent AUs of the transition GOP are sent, and the X+1th GOP is switched to the second bitrate stream.

In order to solve the above problem, the stream sending device according to the present invention includes a storage unit that stores multiple encoded streams encoded at different bit rates, and a sending stream selection unit that selects, switches, and sends out the encoded streams stored in the storage unit, and the storage unit includes a first bit rate stream encoded at a first bit rate, a second bit rate stream that has the same GOP structure as the first bit rate stream and is encoded at a second bit rate, and a transition stream used to switch from the first bit rate stream to the second bit rate stream, and the GOPs constituting the transition stream have GOP length -1 types for each GOP of the first bit rate stream, and the AUs from the first AU to the i-1th AU (0 < i < GOP length) are encoded so that the same decoding result is obtained as the i-1th AU from the first AU of the corresponding GOP of the first bit rate stream, and the i-th and subsequent AUs are encoded at a bit rate closer to the second bit rate than the first bit rate.

In addition, when the transmission stream selection unit switches from the i-th AU of the specified GOP of the first bitrate stream based on the bitrate change instruction, the stream transmission device preferably selects a GOP corresponding to the specified GOP of the transition stream, in which the first AU to the i-1-th AU are encoded so as to obtain the same decoding result as the first AU to the i-1-th AU of the specified GOP of the first bitrate stream, and the i-th and subsequent AUs are encoded at a bitrate closer to the second bitrate than the first bitrate, transmits the first bitrate stream up to the i-1-th AU of the specified GOP, then transmits the i-th and subsequent AUs of the selected GOP of the transition stream, and switches to the second bitrate stream from the next GOP of the specified GOP.

Furthermore, it is preferable that the storage unit of the stream transmission device is provided with first through nth (n is an integer equal to or greater than 2) encoded streams encoded at different bit rates, and a transition stream used to switch from any of the first through nth encoded streams to any other encoded stream.

Furthermore, it is preferable that the storage unit of the stream transmission device includes first through n-th (n is an integer equal to or greater than 2) encoded streams encoded at different bit rates, and a transition stream used to switch from any of the first through n-1-th encoded streams to the n-th encoded stream.

In order to solve the above problem, the stream generating device according to the present invention is a stream generating device that generates an encoded stream for switching between a plurality of encoded streams encoded at different bit rates, and includes a first bit rate encoding unit that encodes an input video at a first bit rate and generates a first bit rate stream, a second bit rate encoding unit that encodes the input video at a second bit rate and generates a second bit rate stream having the same GOP structure as the first bit rate stream, and a first second transition encoding unit that generates a first second transition stream used for switching from the first bit rate stream to the second bit rate stream, and the first second transition encoding unit generates GOP length-1 types of GOPs constituting the first and second transition streams for each GOP of the first bit rate stream, and encodes the AUs from the first AU to the i-1th AU (0<i<GOP length) so that the same decoding result as the i-1th AU from the first AU of the corresponding GOP of the first bit rate stream is obtained, and encodes the i-th and subsequent AUs at a bit rate closer to the second bit rate than the first bit rate.

The stream generating device further includes a second-first transition encoding unit that generates a second-first transition stream used for switching from the second bit rate stream to the first bit rate stream, and the second-first transition encoding unit generates GOPs constituting the second-first transition stream (GOP length -1) for each GOP of the second bit rate stream, and encodes the AUs from the first AU to the i-1th AU (0 < i < GOP length) so that the same decoded result is obtained as the AU from the first AU to the i-1th AU of the corresponding GOP of the second bit rate stream, and desirably encodes the i-th and subsequent AUs at a bit rate closer to the first bit rate than the second bit rate.

To solve the above problem, the program of the present invention is characterized by causing a computer to function as the stream sending device.

To solve the above problem, the program of the present invention is characterized by causing a computer to function as the stream generating device.

The stream sending device, stream generating device, and program of the present invention make it possible to switch between streams encoded at different bit rates at any AU and send them without causing any disturbance in the video on the receiving side.

1 is a conceptual diagram illustrating a case where a bit rate is switched in a stream sending device according to an embodiment. 1 illustrates an example of the configuration of a stream sending device according to an embodiment; 2 illustrates an example of the configuration of a storage unit of the stream sending device according to an embodiment; FIG. 1 is an image diagram of a high bit rate stream and a high-low transition stream. FIG. 2 illustrates a transition from a high bit rate stream to a low bit rate stream. FIG. 1 illustrates an example of a configuration of a stream generating device according to an embodiment. FIG. 1 is a diagram showing an example of the configuration of a stream sending device that supports n (n>2) different bit rates. FIG. 13 is a diagram showing another example of the configuration of a stream sending device that supports n (n>2) different bit rates. FIG. 1 is a conceptual diagram of an encoded stream stored in a conventional stream transmission device. FIG. 1 is a conceptual diagram showing bit rate switching in a conventional stream transmission device.

The following describes an embodiment of the present invention.

(Embodiment 1)
Fig. 1 is a conceptual diagram of bit rate switching in a stream sending device 10 according to an embodiment of the present invention. The stream sending device 10 in Fig. 1 can send two bit rate streams, a predetermined high bit rate and a predetermined low bit rate, and can also switch from a high bit rate stream to a low bit rate stream. The GOPs constituting the high bit rate stream are, from the beginning, GOP H[0], GOP H[1], GOP H[2], ..., and the GOPs constituting the low bit rate stream are, from the beginning, GOP L[0], GOP L[1], GOP L[2], ....

When it becomes necessary to switch to a low bitrate while transmitting a high bitrate stream (for example, when a bitrate change instruction is received while transmitting AU[f-1] of GOP H[1]), the system immediately switches to transmitting the corresponding GOP for high-low transition (hereinafter referred to as high-low transition GOP, represented by GOP HL[1]) from the AU immediately following the AU being transmitted in the high bitrate stream (GOP H[1]), and transmits the low bitrate stream from GOP L[2] after transmission of the high-low transition GOP (GOP HL[1]) is completed.

Here, the high-low transition GOP (GOP HL[1]) has the same GOP structure as the GOP (GOP H[1]) of the high bitrate stream at the time of switching. When switching from sending AU[f], AU[0] to AU[f-1] before switching are AUs that provide the same decoded results as AU[0] to AU[f-1] of GOP H[1] of the high bitrate stream, and may be the same AUs as AU[0] to AU[f-1] of GOP H[1]. AU[f] to AU[len(GOP H[1])-1] of GOP HL[1] are AUs that are coded at a bitrate closer to the low bitrate than the high bitrate (preferably at a bitrate equivalent to the specified low bitrate).

In this way, by switching the encoding stream using a transition GOP, the bit rate can be lowered immediately after a bit rate change instruction is issued, even in the middle of a GOP.

In this embodiment, switching from a high bit rate stream to a low bit rate stream has been described, but by using a corresponding GOP for low-high transition, switching from a low bit rate stream to a high bit rate stream is also possible in a similar manner. In other words, switching between a first bit rate stream and a second bit rate stream encoded at any different bit rate is possible.

(Embodiment 2)
Fig. 2 shows an example of the configuration of a stream sending device 10 according to one embodiment of the present invention. The stream sending device 10 in Fig. 2 can send out two bit rate streams, a high bit rate and a low bit rate, and can switch between the two. In other words, it supports switching from a high bit rate stream to a low bit rate stream, and from a low bit rate stream to a high bit rate stream. The stream sending device 10 includes a storage unit 11 and a sending stream selection unit 12. A bit rate change instruction is input to the sending stream selection unit 12.

Figure 3 shows an example of the configuration of the storage unit 11 of the stream transmission device 10. The storage unit 11 includes a database 111 that stores a stream encoded at a high bit rate (high bit rate stream), a database 112 that stores a stream used when switching from a high bit rate to a low bit rate (high-low transition stream), a database 113 that stores a stream used when switching from a low bit rate to a high bit rate (low-high transition stream), and a database 114 that stores a stream encoded at a low bit rate (low bit rate stream).

Here, the high bit rate stream is a stream that is encoded at a specific high bit rate. The GOPs that make up the high bit rate stream are GOP H[0], GOP H[1], GOP H[2], ... from the beginning.

On the other hand, a low bit rate stream is a stream that is encoded at a specified low bit rate. The GOPs that make up a low bit rate stream are, starting from the beginning, GOP L[0], GOP L[1], GOP L[2], ... Here, struct(GOP H[0]) = struct(GOP L[0]), and similarly, struct(GOP H[1]) = struct(GOP L[1]), struct(GOP H[2]) = struct(GOP L[2]), ... In other words, the GOP structures of corresponding GOPs are the same.

A high-low transition stream is a stream consisting of multiple GOPs for high-low transitions (transitions from a high bit rate to a low bit rate). These GOPs are denoted as GOP HL[X][y]. X indicates the order from the beginning of the GOPs that make up the high-low transition stream. y indicates the position of the boundary between the AUs encoded at a high bit rate and the AUs encoded at a low bit rate in each GOP, and is in the range 1 <= y < len(GOP H[X]). And, struct(GOP HL[X][y]) = struct(GOP H[X]). Also, AU[0] to AU[y-1] in GOP HL[X][y] are encoded streams that provide the same decoded results as AU[0] to AU[y-1] in GOP H[X], and AU[y] to AU[len(GOP H[X])-1] in GOP HL[X][y] are streams that are encoded at a low bit rate (which may be a bit rate lower than a predetermined high bit rate). Note that low bit rate AUs simply have a lower bit rate, and do not provide the same decoded results as AU[y] to AU[len(GOP H[X])-1] in GOP L[X].

A low-high transition stream is a stream consisting of multiple GOPs for low-high transition (transition from a low bit rate to a high bit rate). These GOPs are denoted as GOP LH[X][y]. X indicates the order from the beginning of the GOPs that make up the low-high transition stream. y indicates the position of the boundary between the AUs encoded at a low bit rate and the AUs encoded at a high bit rate in each GOP, and is in the range 1 <= y < len(GOP L[X]). And, struct(GOP LH[X][y]) = struct(GOP L[X]). Also, AU[0] to AU[y-1] in GOP LH[X][y] are encoded streams that provide the same decoded results as AU[0] to AU[y-1] in GOP L[X], and AU[y] to AU[len(GOP L[X])-1] in GOP LH[X][y] are streams that are encoded at a high bit rate (which may be a bit rate higher than a predetermined low bit rate). Note that high bit rate AUs simply have a high bit rate, and do not provide the same decoded results as AU[y] to AU[len(GOP H[X])-1] in GOP H[X].

Figure 4 shows an image of a high bit-rate stream and a high-low transition stream. The thickness (height) of each stream (GOP) corresponds to the bit rate, i.e., the amount of encoded bits. The GOPs that make up a high bit-rate stream are, from the beginning, GOP H[0], GOP H[1], GOP H[2], ..., and each contains a large amount of information.

GOP HL[0][1] in the upper left corner of the high-low transition stream is the GOP that corresponds to GOP H[0] and has the same GOP structure as GOP H[0], with the first AU[0] coded at a high bitrate (i.e., coded so that the decoded result is the same as AU[0] in GOP H[0]) and the AUs after AU[1] coded at a low bitrate.

GOP HL[1][1] to the right of GOP HL[0][1] is a GOP corresponding to GOP H[1], has the same GOP structure as GOP H[1], and the first AU[0] is coded at a high bit rate (i.e., coded so that the same decoded result as AU[0] of GOP H[1] is obtained), and the AUs after AU[1] are coded at a low bit rate. Similarly, GOP HL[2][1] to the right of GOP HL[1][1] is a GOP corresponding to GOP H[2], has the same GOP structure as GOP H[2], has the same GOP structure as GOP H[2], has the first AU[0] coded at a high bit rate (i.e., coded so that the same decoded result as AU[0] of GOP H[2] is obtained), and the AUs after AU[1] are coded at a low bit rate.

Also, looking vertically, GOP HL[0][2], which is one level below GOP HL[0][1], is a GOP corresponding to GOP H[0], has the same GOP structure as GOP H[0], with AU[0] and AU[1] coded at a high bit rate (i.e., coded so that the same decoded result is obtained as AU[0] and AU[1] of GOP H[0]), and AUs from AU[2] onwards are coded at a low bit rate. Similarly, GOP HL[0][3], which is one level below GOP HL[0][2], is a GOP corresponding to GOP H[0], has the same GOP structure as GOP H[0], with AU[0], AU[1] and AU[2] coded at a high bit rate (i.e., coded so that the same decoded result is obtained as AU[0], AU[1] and AU[2] of GOP H[0]), and AUs from AU[3] onwards are coded at a low bit rate.

In this way, the GOP of the high-low transition stream (GOP LH[X][y]) is configured so that it is possible to switch from any AU[y] of any GOP H[X] of the high bit rate stream to a low bit rate. In other words, a GOP corresponding to any switching point is prepared.

Returning to FIG. 2, the output stream selection unit 12 typically selects either a high bitrate stream or a low bitrate stream, and outputs them as stream output in order starting from the first GOP of each stream.

Now, suppose that an instruction to change the bit rate to a low bit rate arrives at a certain timing while the transmission stream selection unit 12 is transmitting a high bit rate stream. Let us suppose that this timing is immediately before the transmission of AU[f] of GOP H[G] (during the transmission of AU[f-1], or during the period from the completion of the transmission of AU[f-1] to the start of the transmission of AU[f]). At this time, the transmission stream selection unit 12 selects GOP HL[G][f] from the storage unit 11, and after the completion of the transmission of AU[f-1] of GOP H[G], instead of transmitting AU[f] of GOP H[G], transmits the AUs of GOP HL[G][f] in order starting from AU[f] of GOP HL[G][f]. After completing the transmission of GOP HL[G][f], the system moves to the low bitrate stream and transmits GOP L[G+1], and thereafter transmits the GOPs of the low bitrate stream in order.

In addition, if an instruction to change the bit rate to a high bit rate arrives at a certain timing while the transmission stream selection unit 12 is transmitting a low bit rate stream, assume that the timing is immediately before the transmission of AU[e] of GOP L[S] (during the transmission of AU[e-1], or during the period from the completion of the transmission of AU[e-1] to the start of the transmission of AU[e]). In this case, the transmission stream selection unit 12 selects GOP LH[S][e] from the storage unit 11, and after the completion of the transmission of AU[e-1] of GOP H[S], instead of transmitting AU[e] of GOP L[S], transmits the AUs of GOP LH[S][e] in order starting from AU[e] of GOP LH[S][e]. After completing the transmission of GOP LH[S][e], the system moves to the high bitrate stream and transmits GOP H[S+1], and then transmits the GOPs of the high bitrate stream in order.

Note that if the "certain timing" during the transmission of the above-mentioned high (or low) bit rate stream is immediately before the first AU of a GOP is transmitted, the high-low transition stream (or low-high transition stream) is not used, and the low bit stream (or high bit stream) can be transmitted from the first AU of the GOP as in the conventional manner.

In this embodiment, it is possible to switch from a high bit rate stream to a low bit rate stream and from a low bit rate stream to a high bit rate stream, but if switching is to be performed in only one direction, it is sufficient to have one transition stream. That is, it is sufficient to have a first bit rate stream encoded at a first bit rate, a second bit rate stream encoded at a second bit rate, and a transition stream used for switching from the first bit rate stream to the second bit rate stream.

The transition from a high bitrate stream to a low bitrate stream will be explained in more detail using Figure 5. Figure 5 shows an example of a case where an instruction to change the bitrate to a low bitrate is received while a high bitrate stream is being transmitted and AU[3] of GOP H[X] is being transmitted. The height of the rectangle indicating each AU roughly indicates the bit amount of each AU. The AUs displayed in gray indicate the AUs transmitted by the present invention.

When a bitrate change command arrives, GOP HL[X][4] is selected from the high-low transition stream. AU[0] to AU[3] are encoded in the same way, so they are the same AUs in GOP H[X] and GOP HL[X][4], or AUs that produce the same decoded results. After AU[3] of GOP H[X] of the high bitrate stream is sent, AU[4] to AU[N] are sent from GOP HL[X][4]. Here, for example, the sum of the bit amount from AU[4] onwards in GOP HL[X][4] is encoded to be approximately the same as the sum of the bit amount from AU[4] onwards in GOP L[X]. Then, from the next GOP, they are sent from GOP L[X+1].

In this embodiment, the sum of the bit amounts of AU[4] to AU[N] in GOP HL[X][4] is approximately equal to the sum of the bit amounts of AU[4] to AU[N] in GOP L[X], so there is a bitrate reduction effect equivalent to that when AU[4] is immediately switched to sending GOP L[X].

Also, AU[4] to AU[N] of GOP HL[X][4] may refer to AU[0] to AU[3] of GOP HL[X][4], but because AU[0] to AU[3] of GOP HL[X][4] will have the same decoded result as AU[0] to AU[3] of GOP H[X][4], switching streams will not cause any distortion in the video on the receiving side.

In this way, the stream transmission device 10 of the present invention makes it possible to switch between streams encoded at multiple bit rates at any AU and transmit them without causing any image disturbance on the receiving side. Furthermore, according to the present invention, transitions can be made without any problems even when inter-frame reference is performed, so there is no increase in bit rate as occurs with all-intra encoding.

A computer can be suitably used to function as the above-mentioned stream transmission device 10, and such a computer can be realized by storing a program describing the processing contents for realizing each function of the stream transmission device 10 in the memory unit of the computer, and having the CPU of the computer read and execute this program. Note that this program can be recorded on a computer-readable recording medium.

(Embodiment 3)
6 shows an example of the configuration of a stream generating device according to an embodiment of the present invention. A stream generating device 20 receives instructions of content (video) and output bit rates (predetermined high and low bit rates) and outputs streams of the respective bit rates and various transition streams. The stream generating device 20 includes a high bit rate encoding unit 21 that generates a high bit rate stream, a high-low transition encoding unit 22 that generates a high-low transition stream, a low-high transition encoding unit 23 that generates a low-high transition stream, and a low bit rate encoding unit 24 that generates a low bit rate stream.

Note that this embodiment supports two types of bit rates, a high bit rate and a low bit rate, but as described below, it may be configured to generate multiple bit rates and generate transition streams between multiple bit rates.

An example of the process of creating each stream in the stream generating device 20 is shown below. First, the high bit rate encoding unit 21 encodes the input video to a specified high bit rate as specified, creating a high bit rate stream. The low bit rate encoding unit 24 encodes the input video to a specified low bit rate as specified, creating a low bit rate stream.

Next, a high/low transition stream is created in the high/low transition encoding unit 22. GOP HL[X][y] (0<=y<len(GOP H[X]), which transitions from GOP H[X] of the high bit rate stream to GOP L[X] of the low bit rate stream, is generated by performing encoding under the following conditions (a) and (b).
(a) For i, where 0<=i<y, AU[i] in GOP HL[X][y] is coded so that the same decoded result is obtained as AU[i] in GOP H[X]. Alternatively, the coded stream of AU[i] in GOP H[X] is reused to make both the AU[i] the same.
(b) For i>=y, AU[i] in GOP HL[X][y] is coded to have a lower bit rate than AU[i] in GOP H[X].

Under the above conditions, for all y (0<=y<len(GOP H[X]) and all X (0<=X), generate GOP HL[X][y] and create a high-low transition stream.

Similarly, the low-high transition stream is also created by the low-high transition encoding unit 23. Specifically, the GOP LH[X][y] (0<=y<len(GOP L[X]) that transitions from the GOP L[X] of the low bit rate stream to the GOP H[X] of the high bit rate stream is generated by performing encoding under the following conditions (c) and (d).
(c) For i where 0<=i<y, AU[i] in GOP LH[X][y] is coded so that the same decoded result is obtained as AU[i] in GOP L[X], or the coded stream of AU[i] in GOP L[X] is reused to make both the AU[i] the same.
(d) For i >= y, AU[i] in GOP LH[X][y] is coded to have a higher bit rate than AU[i] in GOP L[X].

Under the above conditions, for all y (0<=y<len(GOP L[X])) and all X (0<=X), generate GOP LH[X][y] and create a low-high transition stream.

Each stream generated by the stream generating device 20 can be output to and stored in the storage unit 11 of the stream sending device 10.

In this embodiment, both a transition stream from a high bit rate stream to a low bit rate stream and a transition stream from a low bit rate stream to a high bit rate stream are created, but if switching is to be performed in only one direction, it is sufficient to create only one transition stream. That is, it is sufficient to have a first bit rate encoding unit that generates a first bit rate stream, a second bit rate encoding unit that generates a second bit rate stream, and a first-second transition encoding unit that generates a transition stream used for switching from the first bit rate stream to the second bit rate stream.

(Modification of the third embodiment)
In this modification, a target amount of information after encoding (target code amount) is set more strictly than in the third embodiment, and encoding is performed to generate a transition stream.

In this modified example, the generation of the high bit rate stream and the low bit rate stream is the same as in the third embodiment, and the conditions for creating the high-low transition stream and the low-high transition stream are also the same.

When encoding AU[i] (i>=y) of GOP HL[X] of the high-low transition stream, for i>=y, AU[i] of GOP HL[X][y] is encoded at a lower bit rate than AU[i] of GOP H[X]. In this case, the target amount of information is set and encoded to satisfy the following formula (1).

Here, strlen() means the bit length of a certain AU. And Σ indicates the sum of the bit lengths of AU[i] in a given range of i.

When encoding AU[i] (i>=y) of GOP LH[X] of the low-high transition stream, AU[i] of GOP HL[X][y] is encoded at a higher bit rate than AU[i] of GOP L[X] for i>=y. At this time, the target amount of information is set and encoded to satisfy the following formula (2).

In other words, when generating a transition stream that transitions from the first bit rate to the second bit rate, the i-th and subsequent AUs are encoded at a bit rate closer to the second bit rate than the first bit rate.

(Another Modification of the Third Embodiment)
This modification is an embodiment in which a target amount of information after encoding (target code amount) is set and encoding is performed to generate a transition stream so that a sufficient bit rate transition effect occurs even in the middle of a GOP.

When encoding AU[i] (i>=y) of GOP HL[X] of the high-low transition stream, for i>=y, AU[i] of GOP HL[X][y] is encoded at a lower bit rate than AU[i] of GOP H[X]. At that time, the target amount of information is set and encoded to satisfy the following formula (3).

According to formula (3), the sum of the bit amount from AU[y] onwards in GOP HL[X] is coded so that it is equal to or less than the sum of the bit amount from AU[y] onwards in GOP L[X]. Then, from the next GOP onwards, GOP L[X+1] is transmitted. Therefore, even in the middle of a GOP, from AU[y] onwards, there is the same bitrate reduction effect as if the transmission was immediately switched to GOP L[X].

When encoding AU[i] (i>=y) of GOP LH[X] of the low-high transition stream, AU[i] of GOP HL[X][y] is encoded at a higher bit rate than AU[i] of GOP L[X] for i>=y. At that time, the target amount of information is set and encoded to satisfy the following formula (4).

According to formula (4), the sum of the bit amount from AU[y] onwards in GOP LH[X] is coded to be equal to or greater than the sum of the bit amount from AU[y] onwards in GOP H[X]. Then, from the next GOP onwards, GOP H[X+1] is transmitted. Therefore, even in the middle of a GOP, from AU[y] onwards, there is the same bandwidth expansion effect as if the transmission was immediately switched to GOP H[X].

A computer can be suitably used to function as the above-mentioned stream generating device 20, and such a computer can be realized by storing a program describing the processing contents for realizing each function of the stream generating device 20 in the memory unit of the computer, and having the CPU of the computer read and execute this program. Note that this program can be recorded on a computer-readable recording medium.

(Embodiment 4)
In the embodiments described above, an example has been shown in which a high bit rate stream and a low bit rate stream are switched between, but in this embodiment, it is possible to switch between streams with even more bit rates, n types (n>2).

Figure 7 shows an example of the configuration of a stream sending device 10 that supports n different bit rates. The stream sending device 10 in Figure 7 includes a storage unit 13 and a sending stream selection unit 12. Note that although the explanation in Figure 7 assumes n>2, when n=2, it is the same as in embodiment 2, so n can be an integer equal to or greater than 2.

The storage unit 13 has a first bit rate stream, a second bit rate stream, ..., an n-th bit rate stream, which correspond to the first to n-th bit rates. It also stores transition streams for transitioning from each bit rate stream to any other bit rate stream.

That is, a first second transition stream, a first third transition stream, ..., a first nth transition stream for transitioning from a first bit rate stream to a second to nth bit rate stream; a second first transition stream, a second third transition stream, ..., a second nth transition stream for transitioning from a second bit rate stream to a first, third to nth bit rate stream; ... and an nth first transition stream, an nth second transition stream, ..., an nth n-1th transition stream for transitioning from an nth bit rate stream to a first to n-1th bit rate stream. These transition streams are provided.

In this way, in order to transition between any bit rates, a stream that transitions between all bit rates is prepared. The structure of each transition stream is the same as the transition stream described in the second and third embodiments.

The transmission stream selection unit 12 selects a transition stream based on the bit rate change instruction and switches the bit rate stream. Specifically, when a bit rate change instruction is issued to change to another specified bit rate while a certain bit rate stream is being transmitted, a transition stream for transitioning between the two bit rates is selected from among the stored ones, the transition stream is switched to once, and the corresponding GOP of the transition stream is transmitted. After that, the stream with the specified bit rate after the change can be transmitted.

(Embodiment 5)
Fig. 8 shows another example of the configuration of the stream sending device 10 that supports n (n>2) different bit rates. The stream sending device 10 in Fig. 8 includes a storage unit 14 and a sending stream selection unit 12. Note that, although the explanation in Fig. 8 is given assuming n>2, when n=2, it is the same as part of the second embodiment, so n can be an integer of 2 or more.

The storage unit 14 has a first bit rate stream, a second bit rate stream, ..., an nth bit rate stream, which correspond to the first to nth bit rates. It also stores a transition stream for transitioning from each bit rate stream to a specific bit rate stream (here, the nth bit rate stream). This specific bit rate stream is preferably a bit rate stream corresponding to the lowest bit rate.

That is, it has a first n-th transition stream for transitioning from a first bit rate stream to an n-th bit rate stream, a second n-th transition stream for transitioning from a second bit rate stream to an n-th bit rate stream, ... and an n-1-th n-th transition stream for transitioning from an n-1-th bit rate stream to an n-th bit rate stream.

In this way, we provide a stream that transitions from all bit rates to a specific bit rate stream (e.g., the lowest bit rate). The structure of each transition stream is the same as the transition stream described in the second and third embodiments.

The transmission stream selection unit 12 selects a transition stream based on a bit rate change instruction and switches the bit rate stream. Specifically, when a bit rate change instruction is issued to change to another specified bit rate while a certain bit rate stream is being transmitted, a transition stream for transitioning from that certain bit rate to the nth (e.g., lowest) bit rate is selected from among the stored ones, and the transition stream is switched to once, and the corresponding GOP of the transition stream is transmitted. After that, the stream with the specified bit rate after the change can be transmitted.

In this embodiment, when switching, a transition to a specific bit rate (corresponding to the lowest bit rate) occurs, so that no matter what bit rate is switched to, it is possible to switch to another bit rate stream without exceeding the allowable bit rate range. In addition, the amount of streams stored in the storage unit 14 can be significantly reduced compared to embodiment 4.

In the above first, second, fourth and fifth embodiments, the configuration and operation of the stream sending device 10 have been described, but the present invention is not limited to this, and may be configured as a stream sending method in which bit rate streams encoded at different bit rates are switched based on a bit rate change instruction. In other words, the present invention may be configured as a stream sending method that includes a step of sending a first bit rate stream, a step of sending a transition stream, and a step of sending a second bit rate stream according to the data flow in each figure.

Furthermore, in the above third embodiment, the configuration and operation of the stream generating device 20 have been described, but the present invention is not limited to this, and may be configured as a stream generating method that generates a bit rate stream and a transition stream encoded at different bit rates. In other words, the method may be configured as a stream generating method that includes a step of generating a first bit rate stream, a step of generating a second bit rate stream, and a step of generating a transition stream according to the data flow of each figure.

The above-mentioned embodiment has been described as a representative example, but it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple component blocks described in the embodiment into one, or to divide one component block.

1, 10 Stream sending device 11 Storage unit 111 High bit rate stream storage database 112 High-low transition stream storage database 113 Low-high transition stream storage database 114 Low bit rate stream storage database 12 Sending stream selection unit 13 Storage unit 14 Storage unit 20 Stream generating device 21 High bit rate coding unit 22 High-low transition coding unit 23 Low-high transition coding unit 24 Low bit rate coding unit

Claims (9)

A stream transmission device having a function of switching between and transmitting a plurality of encoded streams encoded at different bit rates,
a first bit rate stream encoded at a first bit rate; and a second bit rate stream having the same GOP (Group of Pictures) structure as the first bit rate stream and encoded at a second bit rate,
when switching is performed from the i-th AU (Access Unit) (0<i<GOP length) of the X-th GOP (X is arbitrary) of the first bit-rate stream, a transition GOP is used which has the same GOP structure as the X-th GOP, and from the first AU to the i-1-th AU is encoded to obtain the same decoded result as the first AU to the i-1-th AU of the X-th GOP of the first bit-rate stream, and the i-th and subsequent AUs are encoded at a bit rate closer to the second bit rate than the first bit rate,
Sending the first bit rate stream up to the i-1th AU of the Xth GOP;
Next, transmit the i-th and subsequent AUs of the transition GOP;
A stream sending device which switches to a second bit rate stream from the (X+1)th GOP. A stream transmission device comprising: a storage unit that stores a plurality of coded streams coded at different bit rates; and a transmission stream selection unit that selects, switches, and transmits a coded stream stored in the storage unit,
the storage unit includes a first bit rate stream encoded at a first bit rate, a second bit rate stream having the same GOP structure as the first bit rate stream and encoded at a second bit rate, and a transition stream used for switching from the first bit rate stream to the second bit rate stream;
A stream transmission device characterized in that the GOPs constituting the transition stream have a GOP length minus 1 type for each GOP of the first bit-rate stream, and the AUs from the first AU to the i-1th AU (0<i<GOP length) are encoded so as to obtain the same decoding result as the AU from the first AU to the i-1th AU of the corresponding GOP of the first bit-rate stream, and the AUs from the i-th AU onwards are encoded at a bit rate closer to the second bit rate than the first bit rate. 3. The stream sending device according to claim 2,
When the delivery stream selection unit performs switching from the i-th AU of the specified GOP of the first bit rate stream based on a bit rate change instruction,
selecting a GOP corresponding to the indicated GOP of the transition stream, the GOP being encoded so that the first AU to the (i-1)th AU are encoded so as to obtain the same decoding result as the first AU to the (i-1)th AU of the indicated GOP of the first bit rate stream, and the i-th and subsequent AUs are encoded at a bit rate closer to a second bit rate than the first bit rate;
Sending the first bit rate stream up to the i-1th AU of the indicated GOP;
Then, sending the i-th and subsequent AUs of the selected GOP of the transition stream;
A stream sending device, characterized in that the device switches to a second bit rate stream from the GOP next to the specified GOP. 4. The stream sending device according to claim 2,
The storage unit is characterized in that it has first to nth (n is an integer greater than or equal to 2) encoded streams encoded at different bit rates, and a transition stream used to switch from any of the first to nth encoded streams to any other encoded stream. 4. The stream sending device according to claim 2,
The storage unit is characterized in that it has first to nth (n is an integer equal to or greater than 2) encoded streams encoded at different bit rates, and a transition stream used to switch from any of the first to n-1th encoded streams to the nth encoded stream. 1. A stream generating device for generating an encoded stream for switching between a plurality of encoded streams encoded at different bit rates, comprising:
a first bit rate encoding unit that encodes an input video at a first bit rate to generate a first bit rate stream;
a second bit rate encoding unit that encodes an input video at a second bit rate to generate a second bit rate stream having the same GOP structure as the first bit rate stream;
a first/second transition encoding unit configured to generate a first/second transition stream used for switching from the first bit-rate stream to the second bit-rate stream,
the first-second transition encoding unit generates GOPs constituting the first-second transition stream (GOP length -1) for each GOP of the first bit-rate stream, encodes the AUs from the first AU to the i-1th AU (0 < i < GOP length) so as to obtain the same decoded result as the AU from the first AU to the i-1th AU of the corresponding GOP of the first bit-rate stream, and encodes the AUs from the i-th AU onwards at a bit rate closer to the second bit rate than the first bit rate. 7. The stream generating device according to claim 6,
a second first transition encoding unit configured to generate a second first transition stream used for switching from the second bit-rate stream to the first bit-rate stream;
the second-first transition encoding unit generates GOPs constituting the second-first transition stream of one type (GOP length -1) for each GOP of the second bit-rate stream, encodes the AUs from the first AU to the i-1th AU (0 < i < GOP length) so as to obtain the same decoded result as the AUs from the first AU to the i-1th AU of the corresponding GOP of the second bit-rate stream, and encodes the AUs from the i-th AU onwards at a bit rate closer to the first bit rate than the second bit rate. A program that causes a computer to function as a stream sending device according to any one of claims 1 to 5. A program causing a computer to function as the stream generating device according to claim 6 or 7.

Images