JP5213130B2 - Data distribution system, switching device, and data distribution method - Google Patents

Description

  The present invention relates to a system for distributing moving images, and more particularly to a system for distributing a moving image stream of an arbitrary bit rate.

  Conventionally, when digitalized moving image data is transmitted to a network, digital moving image data of various bit rates is generated, and digital moving image data having a bit rate that matches the usable bandwidth of the network is selected and transmitted. . That is, a moving image with a high bit rate has high image quality, but requires a bandwidth of the communication path. On the other hand, the lower the bit rate, the lower the image quality, but the communication path may be narrow. For this reason, for example, Patent Document 1 proposes an encoding system that converts a plurality of digital moving image data having different bit rates using a plurality of data conversion means (encoder systems for various bits).

Japanese Patent Laid-Open No. 11-341488

  However, according to the technique described in Patent Document 1 described above, since any one of several types of video data having a predetermined bit rate is transmitted, it is a network with a large band fluctuation (for example, a wireless communication network). In such a case, it is difficult to cope with bandwidth fluctuations for each client terminal. In addition, in order to control the bandwidth for each terminal, it is necessary to provide a bit rate conversion device for each terminal (that is, by the number of terminals).

  For this reason, there is a demand for a system that realizes bit rate conversion for each terminal with fewer resources.

A typical example of the invention disclosed in the present application is as follows. That is, a data distribution system including a conversion device that generates at least one stream data at a low rate from source stream data, and a switching device that selects and outputs the stream data , the switching device including the stream data A ring buffer for storing the stream data for each rate, selecting the ring buffer for reading the stream data, and reading the GOP data from the selected ring buffer, thereby converting the source stream data and the conversion One GOP with the selected stream data is selected for each GOP, and stream data at a desired rate is generated by the selected GOP .

  According to a typical embodiment of the present invention, a system that converts a moving image generated in real time to an appropriate bit rate and transmits the moving image can be realized with a small amount of resources.

It is a block diagram which shows the structure of the moving image stream delivery system of the 1st Embodiment of this invention. It is explanatory drawing which shows operation | movement of the multiplexer of the 1st Embodiment of this invention. It is explanatory drawing which shows the process by the stream switching apparatuses 7A-7E of the 1st Embodiment of this invention. It is a flowchart of the process by the Stream Switcher thread of the 1st Embodiment of this invention. It is an example (method 1) flowchart of the GOP selection process of the 1st Embodiment of this invention. It is an example (method 2) flowchart of the GOP selection process of the 1st Embodiment of this invention. It is explanatory drawing which shows the change of the video data by which the rate was selected by the method 1 of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the moving image stream delivery system of the 2nd Embodiment of this invention. It is explanatory drawing which shows operation | movement of the multiplexer of the 2nd Embodiment of this invention. It is explanatory drawing which shows the process by the stream switching apparatus of the 1st Embodiment of this invention. It is a flowchart of the process by the Stream Switcher thread of the 2nd Embodiment of this invention. It is an example (method 1) flowchart of the GOP selection process of the 2nd Embodiment of this invention. It is an example (method 2) flowchart of the GOP selection process of the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the moving image stream delivery system of the 3rd Embodiment of this invention. It is explanatory drawing of the video frame produced | generated by the conventional bit rate conversion apparatus. It is explanatory drawing of the video frame produced | generated by the bit rate conversion apparatus of 3rd Embodiment. It is explanatory drawing of the video frame produced | generated by the switching apparatus of 3rd Embodiment.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration of a moving picture stream distribution system according to the first embodiment of this invention.

  The video stream distribution system according to the first embodiment includes a camcorder 1, a network 2, a video reception device 3, a plurality of bit rate conversion devices 4A to 4D, a multiplexer 5, a load monitoring device 6, and a plurality of stream switching devices 7A to 7E. And a plurality of video distribution apparatuses 8A to 8E. Note that the number of bit rate conversion devices may be one instead of a plurality. In this case, the bit rate of the distributed video is controlled between the bit rate of the source video and the bit rate converted by the bit rate conversion device.

  The camcorder 1 includes a moving image capturing device and a moving image encoding device, and is, for example, a video camera or a mobile phone. The camcorder 1 compresses the captured image by encoding it with the moving image encoding device, and transmits the encoded moving image data ((source image) to the image receiving device 3 via the network 2. For example, MPEG2 is used. MPEG2 / H.264 can be encoded using the /H.264 encoder.

  The network 2 may be a wide area network such as the Internet, a mobile phone network, or an NGN (Next Generation Network), or a network in a narrow area such as a LAN (Local Area Network).

  Note that the camcorder 1 and the network 2 are not essential components of the present invention. When the moving picture stream distribution system does not include the camcorder 1 and the network 2, encoded video data (for example, recorded video data) may be directly input to the video receiver 3.

  The video reception device 3 is a computer having a processor, a memory, and a network interface, receives the encoded source video transmitted from the camcorder 1 via the network 2, and sends it to the subsequent bit rate conversion devices 4A to 4D. Output. That is, the same source video is given to each bit rate conversion device. The video receiver 3 outputs data according to, for example, the MPEG2-TS format. Note that the output data may be only the video ES (Elementary Stream) in the TS packet.

  The bit rate conversion devices 4A to 4D convert the source video into encoded data having a low bit rate different from that of the source video. The bit rate conversion devices 4A to 4D may be computers having a processor, a memory, and a network interface, and may perform processing using dedicated hardware (for example, XCode 2111 of ViXS Systems). Each of the bit rate conversion devices 4A to 4D outputs the conversion result to the multiplexer 5 at the subsequent stage.

  When the video reception device 3 and the bit rate conversion devices 4A to 4D are mounted on different computers, both devices may be connected via a network. In addition, when the bit rate conversion devices 4A to 4D are realized by dedicated hardware, the video reception device 3 and the bit rate conversion devices 4A to 4D are connected in accordance with a standard for in-device connection such as a PCI bus and PCI Express. To do.

  The multiplexer 5 is a computer including a processor, a memory, and a network interface, and may be mounted on the same computer as the computer on which the video reception device 3 is mounted or may be mounted on another computer. The multiplexer 5 receives the moving image data converted into the video stream (for example, MPEG2-TS format) of the bit rate set for each from the bit rate conversion devices 4A to 4D, and the plurality of streams in GOP units. Multiplex (see FIG. 2). At this time, the audio ES may be multiplexed.

  When the bit rate conversion devices 4A to 4D and the multiplexer 5 are mounted on different computers, both devices may be connected by a network. When the bit rate conversion devices 4A to 4D are realized by dedicated hardware, the multiplexer 5 may be operated by the same processor as the computer processor in which the video reception device 3 is mounted.

  Also, the multiplexer 5 is not provided, and each of the bit rate conversion devices 4A to 4D independently broadcasts the TS packet with the bit rate converted to the stream switching devices 7A to 7E, and the stream switching devices 7A to 7E perform a plurality of streams. May be collected, and the collected packets may be rearranged.

  The stream switching devices 7A to 7E are computers each including a processor, a memory, and a network interface, and a stream receiver thread 71, a stream writer thread 73, and a plurality of stream switch threads 74 are operating (see FIG. 3).

  The Stream Writer thread 73 demultiplexes the received stream and writes the demultiplexed stream in the shared memory having the ring buffer structure. The Stream Switcher thread 74 reads the GOP from the ring buffer while advancing the pointer forward according to the request from the video distribution apparatuses 8A to 8E, and outputs the read moving image data.

  At this time, the stream switching devices 7A to 7E select moving image data at a lower rate if the transmission rate is higher than the target rate, and select moving image data at a higher rate if the transmission rate is lower. A GOP selection method will be described later. As the target rate, the value of the communication band between the video distribution apparatuses 8A to 8E and the client terminal (not shown) may be used.

  The multiplexer 5 and the stream switching devices 7A to 7E may be connected by a network capable of broadcast communication.

  The video distribution devices 8A to 8E are computers each including a processor, a memory, and a network interface. The video distribution devices 8A to 8E are devices that distribute video to client terminals (not shown), and can be mounted on a Web server and an RTP server. Note that the video distribution devices 8A to 8E may be operated by the same CPU as the CPU of the computer on which the stream switching devices 7A to 7E are mounted.

  When the stream switching devices 7A to 7E and the video distribution devices 8A to 8E are mounted on different computers, both devices may be connected via a network.

  The load monitoring device 6 is a computer including a processor, a memory, and a network interface, and monitors the network load of the system and the load of the processor of each device. In particular, the processor load of the stream switching devices 7A to 7E, the processor load of the video distribution devices 8A to 8E, and the load of the network to which the video distribution devices 8A to 8E are connected are compared with a predetermined threshold value. . When these load factors exceed a predetermined threshold value (for example, 90% of the upper limit value), the bit rate of the moving image data output from each of the bit rate conversion devices 4A to 4D is lowered, thereby reducing the system Reduce the overall load. On the other hand, when these load factors become less than another predetermined threshold (for example, 80% of the upper limit value), initial values (25 Mbps, 20 Mbps, 15 Mbps, 10 Mbps, etc.) assigned to each bit rate conversion device ) Is the upper limit, and the bit rate of moving image data output from each bit rate conversion device is increased.

  FIG. 2 is an explanatory diagram illustrating the operation of the multiplexer 5 according to the first embodiment of this invention. FIG. 2 shows signals synthesized by multiplexing moving image data of 25 Mbps, 20 Mbps, and 15 Mbps output from each of the bit rate conversion devices 4A to 4C.

  The signal A is 25 Mbps moving image data (for example, a video stream in the MPEG2-TS format) output from the bit rate conversion device 4A, and has a PID 250 attached thereto. The signal B is 20 Mbps moving image data output from the bit rate conversion device 4B, and is given PID200. The signal C is 15 Mbps moving image data output from the bit rate conversion device 4C, and is given PID150. These moving image data are obtained by packetizing a single video ES in the TS format. Here, the PID is an identifier for identifying each stream multiplexed in the TS format.

  The multiplexer 5 concatenates 25 Mbps, 20 Mbps, and 15 Mbps moving image data in GOP units to generate a multiplexed TS stream. The multiplexed TS stream includes moving image data for each GOP at each bit rate in the order of GOP. Note that the audio ES may be multiplexed.

  The stream switching devices 7A to 7E determine the bit rate of moving image data based on the PID, and select a GOP of moving image data having a necessary bit rate.

  FIG. 3 is an explanatory diagram illustrating processing by the stream switching devices 7A to 7E according to the first embodiment of this invention. FIG. 3 shows a state in which multiplexed 25 Mbps, 20 Mbps, and 15 Mbps moving image data output from each of the bit rate conversion devices 4A to 4C is separated and output.

  As described above, in each of the stream switching devices 7A to 7E, the Stream Receiver thread 71, the Stream Writer thread 73, and the plurality of Stream Switch threads 74 are operating.

  Each of the stream switching devices 7A to 7E includes a plurality of ring buffers 75. Each ring buffer stores data of each bit rate in units of GOPs. Combined in the direction of the arrow. Each thread 71, 72, 73 processes the data stored in each ring buffer while handing over this reference pointer 76. This ring buffer is provided in a shared memory accessible from each thread.

  The Stream Receiver thread 71 receives TS data from the multiplexer 5 and stores it in a FIFO (First In First Out) buffer 72.

  The Stream Writer thread 73 demultiplexes the stream received by the Stream Receiver thread 71 and writes the data of each bit rate to the corresponding ring buffer.

  That is, the Stream Writer thread 73 sorts the TS data for each PID and writes it to the write pointer of the corresponding ring buffer. This data writing is performed by an inseparable operation (atomic operation) in which GOPs of the same frame are written at a time. In other words, during the writing of data to the ring buffer, access to the entry to which data is written by the Stream Switcher thread 74 is prohibited. The entry in which data is most recently written is the head of the ring buffer.

  The Stream Switcher thread 74 reads the GOP from the ring buffer while advancing the read pointer in accordance with a request from the video distribution apparatus, and outputs the read GOP. Note that the audio ES may be multiplexed with the read GOP.

  FIG. 4 is a flowchart of a process performed by the stream switch thread 74 according to the first embodiment of this invention.

  The processor on which the Stream Switch thread 74 operates first waits for a moving image transmission request from the client terminal on the receiving side (S101).

  After receiving the transmission request, it is determined whether (1) the write pointer has passed the read pointer or (2) the distance from the read pointer to the write pointer is 90% or more of the ring buffer length (S102). ).

  If either of the conditions (1) or (2) is satisfied, the GOP is skipped and the distance between the read pointer and the write pointer is adjusted to be about 50% of the ring buffer length. That is, when this condition (1) is satisfied, the ring buffer has already overflowed. Further, when the condition (2) is satisfied, the valid data is 90% or more of the ring buffer length, and there is a high possibility that the buffer overflows. For this reason, the read pointer is advanced without reading a predetermined number of GOPs (S103). The skipped GOP is desirably an amount corresponding to about half of the ring buffer length. If the position of the read pointer and the write pointer is such that the Stream Switcher thread 74 can stably read data, the amount of skipped GOP may be an arbitrary amount.

  On the other hand, if neither of the conditions (1) and (2) is satisfied, it is determined that the ring buffer has a writable capacity, and the process proceeds to step S104.

  In step S104, GOP selection processing for determining which rate of GOP to select is executed. Details of the process of selecting a GOP will be described in detail with reference to FIGS.

  After step S104, GOP transmission processing for transmitting the selected GOP is executed (S105), and the read pointer is advanced to the next entry in the current ring buffer (S106).

  Thereafter, it is determined whether or not the distance from the read pointer to the write pointer is 0 (S107). If the distance from the read pointer to the write pointer is not 0, valid data that can be read is stored in the ring buffer, and the process returns to step S102 to process the next GOP.

  On the other hand, if the distance from the read pointer to the write pointer is 0, no valid data that can be read is stored in the ring buffer. For this reason, it is determined whether or not the read pointer reaches the end of the stream and the Stream Switcher thread 74 is stopped (S108).

  As a result, if the read pointer has reached the end of the stream, the processing by the Stream Switcher thread 74 is terminated. On the other hand, if the read pointer has not reached the end of the stream, after the Stream Switcher thread 74 is stopped for a predetermined time (for example, 0.5 seconds) until valid data that can be read is stored in the ring buffer, Returning to S102, the next GOP is processed.

  FIG. 5 is a flowchart of an example (method 1) of the GOP selection process according to the first embodiment of this invention.

  In Method 1, the Stream Switcher thread 74 selects a lower rate if the transmission rate is higher than the target rate, and selects a higher rate if the transmission rate is lower than the target rate. This target rate may be determined by, for example, the bandwidth of the line between the video distribution apparatuses 8A to 8E and the client terminal. The transmission rate may be the amount of data transmitted during the latest predetermined time (for example, 10 seconds).

  And the video data comprised by the moving image data (GOP data) of the selected rate is output to the video delivery apparatuses 8A-8E. As a result, for example, video data whose rate changes as shown in FIG. 7 is generated.

  However, if the current position of the pointer is a predetermined position (for example, a position closer to a position advanced by 20% from the read pointer), a low rate is selected. This is because if the current position of the pointer is behind the predetermined position, the amount of valid data stored in the ring buffer is less than 20%, so the amount of data read from the ring buffer is limited. This is to avoid interruption of the video due to buffer underrun.

  The GOP selection process of Method 1 will be specifically described according to the flowchart.

  First, a transmission rate is calculated (S111). The calculation of the transmission rate defines a predetermined sampling interval (for example, the latest 10 seconds), and obtains the average value of the data actually transmitted within that interval (transmission bit amount / time in the sampling interval). be able to. Further, the transmission rate may be a slope when the amount of data actually transmitted in the sampling interval is applied to a straight line using linear regression analysis or the like.

  Thereafter, the calculated transmission rate is compared with the target rate, and it is determined whether or not a value obtained by subtracting the target rate from the transmission rate exceeds a predetermined value (C1) (S112). As a result, if transmission rate−target rate> C1, the transmission rate is larger than the target rate, so a GOP is selected from the ring buffer at a lower rate (S114), and the GOP selection process is terminated.

  On the other hand, if transmission rate−target rate ≦ C1, it is determined whether or not a value obtained by subtracting the target rate from the transmission rate is less than a predetermined value (C2) (S113). As a result, if transmission rate−target rate <C2, since the transmission rate is smaller than the target rate, a GOP is selected from the ring buffer having a higher step (S115), and the GOP selection process is terminated.

  On the other hand, if C2 ≦ transmission rate−target rate ≦ C1, it is not necessary to change the bit rate of the moving image data to be transmitted. Therefore, the GOP of the currently selected rate is selected (S116), and the GOP is selected. The process ends.

  FIG. 6 is a flowchart of an example (method 2) of the GOP selection process according to the first embodiment of this invention.

  In Method 2, the read pointer is controlled with the position facing the write pointer as a target. Specifically, when the position of the read pointer is ahead of the target position, a higher rate is selected. On the other hand, if the position of the read pointer is behind the target position, a lower rate is selected.

  That is, when data is written to the ring buffer at a rate exceeding the transmission rate, the write pointer catches up with the read pointer and the buffer overflows. On the other hand, when data is written into the ring buffer at a rate lower than the transmission rate, the read pointer catches up with the write pointer and the buffer underflows. For this reason, the effective amount of data that can be read is the capacity of the ring buffer so that the target position of the read pointer is opposite to the write pointer so that the amount of data stored in the ring buffer is appropriate. Is controlled to about 50%.

  The GOP selection process of Method 2 will be specifically described according to the flowchart.

  First, the transmission rate is calculated (S131). For the calculation of the transmission rate, the same method as that described in the GOP selection process (FIG. 5) of Method 1 can be employed.

  Thereafter, it is determined whether or not the distance from the read pointer to the write pointer is less than 20% of the ring buffer length (S132). As a result, when the distance from the read pointer to the write pointer is less than 20% of the ring buffer length, there is little effective data that can be read, so a GOP is selected from the ring buffer at a lower rate (S314), The GOP selection process is terminated.

  On the other hand, if the distance from the read pointer to the write pointer is 20% or more of the ring buffer length, it is determined whether or not the distance from the read pointer to the write pointer is greater than 80% of the ring buffer length (S133). As a result, when the distance from the read pointer to the write pointer is greater than 80% of the ring buffer length, there is a lot of valid data that can be read, so a GOP is selected from the ring buffer with a higher step (S135). The selection process ends.

  On the other hand, if the distance from the read pointer to the write pointer is not less than 20% and not more than 80% of the ring buffer length, it is determined that the amount of valid data that can be read is appropriate, and the GOP at the currently selected rate. Is selected (S136), and the GOP selection process is terminated.

  FIG. 7 is an explanatory diagram showing changes in video data whose rate is selected by the method 1 described above.

  In the graph, a broken line parallel to the time axis indicates the accumulated data amount of each GOP at a data rate of 20 Mbps, and a one-dot chain line parallel to the time axis indicates an accumulated data amount of each GOP at a data rate of 25 Mbps.

  In the state shown in FIG. 7, up to GOPn + 4, 25 Mbps data is selected, the next GOPn + 3 is low 20 Mbps data, the next GOPn + 2 is high 25 Mbps data, and the next GOPn Low 20 Mbps data was selected for +1 and GOPn.

  For this reason, the moving image data output from the stream switching devices 7A to 7E of the present embodiment can maintain a rate between 25 Mbps and 20 Mbps.

  As described above, in the first embodiment of the present invention, by selecting a GOP having an appropriate rate for each GOP, moving image data can be distributed at a rate suitable for the client terminal, and a buffer underrun is performed. You can avoid the interruption of the video.

<Embodiment 2>
Next, a second embodiment of the present invention will be described.

  Depending on the moving picture coding format, there is a dependency relationship between GOPs, and there is a case where it is not possible to easily switch between GOPs. For example, since an open GOP has a dependency relationship with the preceding and succeeding GOPs, a frame cannot be reproduced if there is no preceding or following GOP. Therefore, in the second embodiment, an open GOP for 15 frames, a rear closed GOP for 7 frames, or a front closed GOP for 8 frames is generated.

  FIG. 8 is a block diagram illustrating a configuration of a moving picture stream distribution system according to the second embodiment of this invention.

  The video stream distribution system according to the second embodiment includes a camcorder 1, a network 2, a video reception device 3, a plurality of bit rate conversion devices 4A to 4E, a multiplexer 5, a load monitoring device 6, and a plurality of stream switching devices 7A to 7E. And a plurality of video distribution apparatuses 8A to 8E. Note that, in the second embodiment, the same components as those in the first embodiment described above are denoted by the same reference numerals, and description thereof is omitted.

  Unlike the moving picture stream distribution system of the first embodiment described above, the moving picture stream distribution system of the second embodiment also inputs a source video of 30 Mbps to the multiplexer 5 via the bit rate conversion device 4E.

  The bit rate conversion devices 4A to 4D according to the second embodiment convert the data rate of the source video to convert it to low-rate moving image data, and further, open GOP, rear closed GOP, and front of the bit rate. Create a closed GOP. Also, the bit rate conversion device 4E generates an open GOP, a rear closed GOP, and a front closed GOP of the bit rate without changing the data rate of the source video (see FIG. 9). Here, an open GOP is a GOP having both front and rear dependencies between GOPs, and includes 15 frames of data. The rear closed GOP is a GOP having a dependency relationship with only the front GOP, and includes 7 frames of data. The front closed GOP is a GOP having a dependency relationship with only the rear GOP, and includes 8 frames of data. That is, the frame included in the open GOP can be reproduced by combining the rear closed GOP and the front closed GOP.

  The multiplexer 5 receives the open GOP, the rear closed GOP, and the front closed GOP of each data rate from each of the bit rate conversion devices 4A to 4D, and multiplexes the GOPs of the plurality of data rates (see FIG. 9). At this time, the audio ES may be multiplexed.

  The Stream Switcher thread 74 of the stream switching devices 7A to 7E reads the open GOP from the ring buffer when the data rate does not switch. On the other hand, when the data rate switching occurs, the Stream Switcher thread 74 reads the backward closed GOP of the switching source rate and the forward closed GOP of the switching destination rate.

  FIG. 9 is an explanatory diagram illustrating the operation of the multiplexer 5 according to the second embodiment of this invention.

  The bit rate conversion device 4A converts the bit rate of the source video (30 Mbps) to generate a low-rate open GOP, rear closed GOP, and front closed GOP. For example, an open GOP is 25 Mbps moving image data (for example, a video stream in the MPEG2-TS format) and has a PID 250 attached thereto. Similarly, the PID 251 is assigned to the rear closed GOP, and the PID 252 is assigned to the front closed GOP. That is, the PID in the second embodiment is an identifier for identifying the bit rate and type of GOP. For this reason, the stream switching devices 7A to 7E can distinguish the GOP to be selected based on the PID.

  Note that FIG. 9 illustrates only 25 Mbps moving image data output from the bit rate conversion device 4A. Similarly, the open GOP, the rear closed GOP, and the front closed GOP are also used at 30 Mbps, 20 Mbps, 15 Mbps, and 10 Mbps. Is output from each bit rate conversion device.

  The multiplexer 5 concatenates moving image data from 30 Mbps to 10 Mbps in units of GOPs, and generates a multiplexed TS stream. The multiplexed TS stream includes moving image data (open GOP, rear closed GOP, and front closed GOP) of each bit rate for each GOP in the order of GOP. Note that the audio ES may be multiplexed.

  The stream switching devices 7A to 7E determine the bit rate of moving image data and the type of GOP based on the PID, and select the GOP of moving image data having a necessary bit rate.

  FIG. 10 is an explanatory diagram illustrating processing performed by the stream switching devices 7A to 7E according to the first embodiment of this invention. FIG. 10 shows a state in which the multiplexed moving image data of 30 Mbps, 25 Mbps, and 20 Mbps output from each of the bit rate conversion devices 4E, 4A, and 4B is separated, and the moving image data by the selected GOP is output. Indicates.

  As described above with reference to FIG. 3, in each of the stream switching devices 7 </ b> A to 7 </ b> E, a Stream Receiver thread (not shown), a Stream Writer thread (not shown), and a plurality of Stream Switch threads 74 are operating.

  Each of the stream switching devices 7A to 7E includes a plurality of ring buffers, and each ring buffer stores data of each bit rate (open GOP, rear closed GOP, and front closed GOP) in units of GOP. The data of each GOP is coupled in the direction of the arrow by a reference pointer 76. In the figure, the open GOP is marked with an “O”, the rear closed GOP is marked with a “C” and a thick line behind it, and the front closed GOP is marked with a “C” and a thick line ahead. Was attached.

  Each thread processes the data stored in each ring buffer while using the reference pointer 76. This ring buffer is provided in a shared memory accessible from each thread.

  The Stream Switcher thread 74 reads the GOP from the ring buffer while advancing the read pointer in accordance with a request from the video distribution apparatus, and outputs the read GOP. That is, the Stream Switcher thread 74 reads the open GOP from the ring buffer when the data rate is not switched. On the other hand, when the data rate switching occurs, the Stream Switcher thread 74 reads the backward closed GOP of the switching source rate and the forward closed GOP of the switching destination rate. In the figure, the selected GOP is hatched.

  As a result, when the rate does not change, the open GOP with a high compression rate is selected to suppress the amount of data to be transferred, and when the rate changes, the rear closed GOP and the front closed GOP are selected. Therefore, moving image data whose data rate can be switched seamlessly can be distributed.

  FIG. 11 is a flowchart of a process performed by the stream switch thread 74 according to the second embodiment of this invention.

  The processor on which the Stream Switch thread 74 operates first waits for a moving image transmission request from the receiving client terminal (S201).

  After receiving the transmission request, it is determined whether (1) the write pointer has passed the read pointer, or (2) the distance from the read pointer to the write pointer is 90% or more of the ring buffer length (S202). ).

  If either of the conditions (1) or (2) is satisfied, the GOP is skipped and the distance between the read pointer and the write pointer is adjusted to be about 50% of the ring buffer length. That is, when this condition (1) is satisfied, the ring buffer has already overflowed. Further, when the condition (2) is satisfied, the valid data is 90% or more of the ring buffer length, and there is a high possibility that the buffer overflows. Therefore, the read pointer is advanced without reading a predetermined number of GOPs (S203). The skipped GOP is desirably an amount corresponding to about half of the ring buffer length. If the position of the read pointer and the write pointer is such that the Stream Switcher thread 74 can stably read data, the amount of skipped GOP may be an arbitrary amount.

  On the other hand, if neither of the conditions (1) and (2) is satisfied, it is determined that the ring buffer has a writable capacity, and the process proceeds to step S204.

  In step S204, a GOP selection / transmission process of determining which rate of GOP to select and transmitting the selected GOP is executed. Details of the GOP selection / transmission process will be described in detail with reference to FIGS.

  After step S204, it is determined whether or not the distance from the read pointer to the write pointer is 0 (S207). If the distance from the read pointer to the write pointer is not 0, valid data that can be read is stored in the ring buffer, and the process returns to step S202 to process the next GOP.

  On the other hand, if the distance from the read pointer to the write pointer is 0, no valid data that can be read is stored in the ring buffer. For this reason, it is determined whether or not the read pointer has reached the end of the stream and the Stream Switcher thread 74 is stopped (S208).

  As a result, if the read pointer has reached the end of the stream, the processing by the Stream Switcher thread 74 is terminated. On the other hand, if the end of the stream has not been reached, after stopping the Stream Switcher thread 74 for a predetermined time (for example, 0.5 seconds) until valid data that can be read in the ring buffer is stored in the ring buffer, Returning to step S202, the next GOP is processed.

  FIG. 12 is a flowchart of an example (method 1) of the GOP selection process according to the second embodiment of this invention.

  In Method 1, as in the first embodiment described above, the Stream Switcher thread 74 selects a lower rate if the transmission rate is higher than the target rate, and a higher rate if the transmission rate is lower than the target rate. Select.

  First, a transmission rate is calculated (S211). For the calculation of the transmission rate, the same method as described above in the first embodiment can be employed.

  Thereafter, the calculated transmission rate is compared with the target rate, and it is determined whether or not a value obtained by subtracting the target rate from the transmission rate exceeds a predetermined value (C1) (S212). As a result, if transmission rate−target rate> C1, the transmission rate is greater than the target rate, so it is determined to lower the transmission rate with the current GOP.

  For this reason, the rear closed GOP is selected from the ring buffer at the currently selected rate, and the selected rear closed GOP is transmitted (S214). Thereafter, the forward closed GOP of the ring buffer with a lower rate is selected (S215), and the selected forward closed GOP is transmitted (S216). A 15-frame image can be reproduced by the rear closed GOP and the front closed GOP selected and transmitted. Thereafter, the process proceeds to step S221.

  On the other hand, if transmission rate−target rate ≦ C1, it is determined whether or not a value obtained by subtracting the target rate from the transmission rate is less than a predetermined value (C2) (S213). As a result, when the transmission rate−the target rate <C2, the transmission rate is smaller than the target rate, so the transmission rate is increased with the current GOP.

  Therefore, the rear closed GOP is selected from the ring buffer at the currently selected rate, and the selected rear closed GOP is transmitted (S218). Thereafter, the forward closed GOP of the ring buffer having a higher level is selected (S219), and the selected forward closed GOP is transmitted (S220). A 15-frame image can be reproduced by the rear closed GOP and the front closed GOP selected and transmitted. Thereafter, the process proceeds to step S221.

  On the other hand, if C2 ≦ transmission rate−target rate ≦ C1, it is determined that there is no need to change the bit rate of the moving image data to be transmitted, and an open GOP is selected from the ring buffer of the currently selected rate. (S217), the process proceeds to step S221.

  In step S221, the pointer is advanced to the next entry in the current ring buffer, and the GOP selection / transmission process is terminated.

  FIG. 13 is a flowchart of an example (method 2) of the GOP selection process according to the second embodiment of this invention.

  In the method 2, similarly to the method 2 of the first embodiment described above, the read pointer is controlled with the position facing the write pointer as a target.

  First, the transmission rate is calculated (S231). For the calculation of the transmission rate, the same method as described above in the first embodiment can be employed.

  Thereafter, it is determined whether the distance from the read pointer to the write pointer is less than 20% of the ring buffer length (S232). As a result, when the distance from the read pointer to the write pointer is less than 20% of the ring buffer length, the effective data that can be read is small, so the transmission rate is lowered with the current GOP.

  For this reason, the backward closed GOP is selected from the ring buffer of the currently selected rate, and the selected backward closed GOP is transmitted (S234). Thereafter, the forward closed GOP of the ring buffer with a lower rate is selected (S235), and the selected forward closed GOP is transmitted (S236). A 15-frame image can be reproduced by the rear closed GOP and the front closed GOP selected and transmitted. Thereafter, the process proceeds to step S241.

  On the other hand, if the distance from the read pointer to the write pointer is 20% or more of the ring buffer length, it is determined whether or not the distance from the read pointer to the write pointer is greater than 80% of the ring buffer length (S233). As a result, when the distance from the read pointer to the write pointer is greater than 80% of the ring buffer length, there is a lot of valid data that can be read, so the transmission rate is increased with the current GOP.

  For this reason, the backward closed GOP is selected from the ring buffer of the currently selected rate, and the selected backward closed GOP is transmitted (S238). Thereafter, the forward closed GOP of the ring buffer with a higher rate is selected (S239), and the selected forward closed GOP is transmitted (S240). A 15-frame image can be reproduced by the rear closed GOP and the front closed GOP selected and transmitted. Thereafter, the process proceeds to step S241.

  On the other hand, when the distance from the read pointer to the write pointer is 20% or more and 80% or less of the ring buffer length, it is determined that the amount of valid data that can be read is appropriate and the rate does not need to be changed. Then, an open GOP is selected from the ring buffer of the currently selected rate (S237), and the process proceeds to step S241.

  In step S241, the pointer is advanced to the next entry in the current ring buffer, and the GOP selection / transmission process is terminated.

  As described above, in the second embodiment of the present invention, an open GOP, a front closed GOP, and a rear closed GOP are generated, an open GOP is selected at a timing other than the rate switching timing, and a front closed at the rate switching timing. By selecting the GOP and the rear closed GOP, the stream video rate can be changed even when there is a dependency between the GOPs.

<Embodiment 3>
Next, a third embodiment of the present invention will be described.

  FIG. 14 is a block diagram illustrating a configuration of a moving picture stream distribution system according to the third embodiment of this invention.

  In the third embodiment, when any one or more of the network bandwidth used for video distribution is lower than the bit rate of the source video, the GOP frame converted to a lower bit rate by the bit rate conversion device is used. The bit rate is adjusted by replacing it with the GOP frame of the source video at the same time. For example, the bit rate conversion device converts the bit rate according to the delivery destination of the lowest bandwidth among the plurality of delivery destinations, and the stream switching device converts the source video to the lower bit rate for the other delivery destinations. Send while switching video.

  The moving picture stream distribution system according to the third embodiment includes a video reading device 9, a bit rate conversion device 4F, a multiplexer 5, a plurality of stream switching devices 7A to 7B, and a plurality of video distribution devices 8A to 8B. Note that in the third embodiment, the same components as those in the first embodiment described above are denoted by the same reference numerals, and descriptions thereof are omitted.

  The video reading device 9 is a computer including a processor, a memory, and a network interface, and reads stream data from the recording device. In the third embodiment, similarly to the first embodiment described above, the encoded moving image data transmitted from the camcorder 1 is converted into the bit rate conversion device 4A via the video receiving device 3. It may be input to 4D.

  The bit rate conversion device 4F converts the encoded moving image data (source video) into encoded data having an arbitrary low bit rate different from that of the source video. Further, the bit rate conversion device 4F can change the bit rate of the encoded data to be output. For example, the bit rate conversion device 4F may convert the bit rate in accordance with a delivery destination having the lowest bandwidth among a plurality of delivery destinations.

  The bit rate conversion device 4F may be a computer including a processor, a memory, and a network interface, or may perform processing using dedicated hardware (for example, XCode 2111 of ViXS Systems). The bit rate conversion device 4F outputs the conversion result to the multiplexer 5 at the subsequent stage.

  Also in the third embodiment, a plurality of bit rate conversion devices 4A to 4D may be provided as in the first embodiment described above instead of the variable rate bit rate conversion device 4F. In this case, a bit rate conversion device that outputs a moving image having a bit rate equal to or lower than the lowest bandwidth may be selected.

  When the video reception device 3 and each bit rate conversion device 4F are mounted on different computers, both devices may be connected via a network. In addition, when the bit rate conversion device 4F is realized by dedicated hardware, the video reception device 3 and the bit rate conversion device 4F are connected in accordance with a connection standard such as a PCI bus or PCI Express.

  FIG. 15 shows a video frame generated by the conventional bit rate conversion apparatus, FIG. 16 shows a video frame generated by the bit rate conversion apparatus of the third embodiment, and FIG. The video frame produced | generated by the switching apparatus of embodiment is shown.

  When the source video data input to the bit rate conversion device 4F is configured by an open GOP, the first B0 frame and the B1 frame of the GOP at the bit rate conversion start time refer to the frame in the previous GOP. Therefore, the first B frame of the open GOP cannot be reproduced. As a result, a GOP in which some frames (B0, B1) are missing is generated as shown in FIG.

  Therefore, when the bit rate conversion device 4F according to the third embodiment switches from GOPn that is an open GOP to GOPn that is an open GOP of another rate, as shown in FIG. 16, GOPn at the bit rate conversion start time The bit rate is converted from the source video of GOPn-1 immediately before.

  As a result, as shown in FIG. 16, the B frame before the I frame of GOPn-1 is lost because there is no GOPn-2 data to be referenced, but all the frames of GOPn can be reproduced without being lost. A frame with a converted bit rate can be generated. Then, as shown in FIG. 17, a continuous stream can be generated by connecting GOPn with the bit rate converted to GOPn-1 of the original source video.

  Similarly, when the bit rate conversion is completed, a GOP in which B frames are not lost can be obtained by converting 1 GOP to a lower bit rate than the last GOP.

  In this case, since the bit rate of the GOP before and after switching is different, the image quality of the frame of the GOP after switching that refers to the GOP before switching may be deteriorated. For this reason, when the difference in the bit rate of GOP before and after switching is large, there is a possibility that image quality will be greatly degraded. For example, in FIG. 17, since the GOPn-1 bit rate and the GOPn bit rate are different, the image quality of the GOPn B0 frame and B1 frame referencing the GOPn-1 frame deteriorates. In order to alleviate this image quality degradation, the lower limit value of the bit rate conversion of the source video by the bit rate conversion device 4F may be set to 80% of the bit rate of the source video.

  Further, the bit rate conversion device 4F repeatedly generates a GOP with an increased bit rate (for example, generates a GOP having a bit rate of 80% or more of the bit rate of the source video at a rate of once per 10 GOP). When the stream switching device 7A switches the bit rate between the open GOPs, the bit rate may be switched at a GOP having a relatively high bit rate.

  As described above, in the third embodiment of the present invention, stream data at an arbitrary rate can be generated by using only one bit rate conversion device. In addition, since the difference in the GOP bit rate before and after the bit rate switching is reduced, it is possible to suppress image quality deterioration at the time of switching.

DESCRIPTION OF SYMBOLS 1 Camcorder 2 Network 3 Video receiving device 4A-4F Bit rate conversion device 5 Multiplexer 6 Load monitoring device 7A-7E Stream switching device 8A-8E Video distribution device

Claims (13)

A conversion device that generates at least one stream data at a low rate from the source stream data;
A data distribution system comprising a switching device that selects and outputs the stream data,
The switching device is
A ring buffer for storing the stream data for each stream data rate;
By selecting the ring buffer from which the stream data is to be read and reading GOP data from the selected ring buffer, any one GOP of the source stream data and the converted stream data is determined for each GOP. Selected,
A data distribution system for generating stream data at a desired rate by the selected GOP .   The data distribution system according to claim 1, wherein the switching device selects a GOP of stream data at a low rate when the amount of data stored in the ring buffer becomes smaller than a predetermined threshold. .   The switching device is
  Controlling the ring buffer by a write pointer indicating a position where data is written to the ring buffer and a read pointer indicating a position where data is read from the ring buffer;
  A position facing the write pointer on the ring buffer is set as a target position of the read pointer,
  2. The data distribution system according to claim 1, wherein when the read pointer is advanced from the target position, a GOP of high-rate stream data is selected.   A conversion device that generates at least one stream data at a low rate from the source stream data;
  A data distribution system comprising a switching device that selects and outputs the stream data,
  The data distribution system further includes a load monitoring device that monitors a load of the conversion device,
  When the load monitored by the load monitoring device exceeds a predetermined threshold, the conversion device lowers the rate of the stream data to be generated,
  The data switching system, wherein the switching device selects a GOP of the source stream data and the converted stream data for each GOP, and generates stream data at a desired rate.   A conversion device that generates at least one stream data at a low rate from the source stream data;
  A data distribution system comprising a switching device that selects and outputs the stream data,
  The conversion apparatus generates stream data including an open GOP, a rear closed GOP, and a front closed GOP,
  The switching device is
  For each GOP, select one of the GOPs of the source stream data and the converted stream data,
  If the stream data rate cannot be switched, select Open GOP,
  When the stream data rate is switched, the rear closed GOP of the stream data at the rate before switching and the forward closed GOP of the stream data at the rate after switching are selected,
  A data distribution system for generating stream data at a desired rate by the selected GOP.   A conversion device that generates at least one stream data at a low rate from the source stream data;
  A data distribution system comprising a switching device that selects and outputs the stream data,
  The converter is
  The rate of the stream data to be generated is variable,
  Generate stream data including open GOP,
  When the stream data rate is switched, the rate of the stream data to be generated is controlled so that the difference between the stream data rate before the switching and the stream data rate after the switching becomes a predetermined value or less,
  The data switching system, wherein the switching device selects a GOP of the source stream data and the converted stream data for each GOP, and generates stream data at a desired rate.   A switching device connected to a conversion device that generates at least one stream data at a low rate from source stream data,
  The switching device is
  A ring buffer for storing the stream data for each stream data rate;
  By selecting the ring buffer from which the stream data is to be read and reading GOP data from the selected ring buffer, any one GOP of the source stream data and the converted stream data is determined for each GOP. Selected,
  A switching device that generates stream data of a desired rate by the selected GOP.   The switching device according to claim 7, wherein the switching device selects a GOP of stream data at a low rate when the amount of data stored in the ring buffer becomes smaller than a predetermined threshold.   The switching device is
  Controlling the ring buffer by a write pointer indicating a position where data is written to the ring buffer and a read pointer indicating a position where data is read from the ring buffer;
  A position facing the write pointer on the ring buffer is set as a target position of the read pointer,
  8. The switching device according to claim 7, wherein when the read pointer is advanced from the target position, a GOP of stream data having a high rate is selected.   A data distribution method in a data distribution system comprising: a conversion device that converts a stream data rate; and a switching device that selects and outputs the stream data,
  The method
  The conversion device generates at least one stream data at a low rate from the source stream data;
  The switching device selects a GOP of the source stream data and the converted stream data for each GOP,
  When the rate of the stream data to be generated is higher than a predetermined target rate, the switching device selects a GOP of stream data with a low rate,
  When the rate of the stream data to be generated is lower than the target rate, the switching device selects a GOP of stream data with a higher rate;
  The data distribution method, wherein the switching device generates stream data of a desired rate by the selected GOP and distributes the generated stream data.   A data distribution method in a data distribution system comprising: a conversion device that converts a stream data rate; and a switching device that selects and outputs the stream data,
  The switching device includes a ring buffer for storing the stream data for each stream data rate,
  The method
  The conversion device generates at least one stream data at a low rate from the source stream data;
  The switching device selects the ring buffer from which the stream data is to be read, and reads GOP data from the selected ring buffer, so that any one of the source stream data and the converted stream data is changed to GOP. , Select for each GOP,
  The data distribution method, wherein the switching device generates stream data of a desired rate by the selected GOP and distributes the generated stream data.   The method
  12. The data distribution method according to claim 11, wherein the switching device selects a GOP of stream data at a low rate when the amount of data stored in the ring buffer becomes smaller than a predetermined threshold. .   The method
  The switching device controls the ring buffer by a write pointer indicating a position where data is written to the ring buffer and a read pointer indicating a position where data is read from the ring buffer;
  The switching device sets a position facing the write pointer on the ring buffer as a target position of the read pointer;
  12. The data distribution method according to claim 11, wherein the switching device selects a GOP of stream data having a high rate when the read pointer is advanced from the target position.

Images