JP2012029219A - Data transmitter, program and method for completing data transmission before time limit - Google Patents

Abstract

Encoding processing is performed on video data to be transmitted using encoding parameters with the highest possible quality, encoding processing and transmission processing are performed in parallel, and transmission is completed by the deadline time. Provided is a data transmission device or the like that can
A data transmission device includes an allocatable bit per pixel from a total number of pixels based on all frames of all video data, a transmission speed and a time limit (transmission completion time limit t _L -transmission start time t ₀ ). From the pixel bit number calculating means for calculating the number D (bit / pixel), the assignable bit number D per pixel for each video data, and the total number of pixels per unit time of the video data, Bit rate calculation means for calculating a bit rate (number of bits per unit time), and encode parameter selection means for selecting an encode parameter that can be transmitted at the bit rate for each video data and instructing the encode means.
[Selection] Figure 3

Description

  The present invention relates to a data transmission apparatus, program, and method for encoding data to be transmitted and transmitting the encoded data via a network.

  Conventionally, there is a technique for shortening the transmission time by compressing data (see, for example, Patent Document 1). According to this technique, the data transmitting apparatus calculates the total time of the compression processing time for the data to be transmitted and the transmission time of the compressed data. When the total time is shorter than the transmission time of the data without being compressed, the data transmitting apparatus performs compression processing on the data and transmits the compressed data.

  Further, there is a technique for distributing video data within a time limit specified by a user for a system that distributes video from a server (see, for example, Patent Document 2). According to this technique, video data stored in a server is encoded (compressed) in advance using hierarchical encoding, so that one type of video data is distributed at different bit rates depending on network conditions. be able to. By changing the distribution bit rate, the transmission time of the video data can be changed.

  Furthermore, there is a storage-type video transmission technique for transmitting video data stored in a data transmission device via a network within a time limit (until the time limit) (see, for example, Patent Document 3). According to this technique, it is considered that the encoding process and the transmission process are executed in parallel. The encoding process refers to a process of encoding video data based on a certain rule, for example, a data compression process. In the case of video data, a number of encoding functions are provided according to combinations of encoding parameters (video standard, resolution, etc.).

  Further, for example, there is a technique for encoding and transmitting video data in an emergency report of a television broadcast station by a deadline time (see, for example, Non-Patent Document 1). According to this technique, it is possible to realize as high quality video transmission as possible and to complete data transmission by the deadline time in consideration of a decrease in network quality. In this regard, unlike the technique described in Patent Document 2, the encoding process is not performed on the video data in advance. According to the technique described in Non-Patent Document 1, the encoding parameter is automatically determined so that the data transmission can be completed by the deadline time. In addition, when the transmission speed is reduced due to the deterioration of the quality of the network line during the transmission of the video data, the data transmission can be completed by the deadline time by resetting the encoding parameter.

JP-A-7-175707 JP 2006-155178 A USP 7,533,398

Yosuke Toyoda, Ikuo Yoneyama, "Storage-type video transmission within a limited time considering network bandwidth fluctuation", PCSJ / IMPS2009, I-2-17

  According to the technique described in Patent Document 1, it is only determined whether or not there is compression processing of data to be transmitted. Therefore, the encoding process and the transmission process are not executed in consideration of the quality of the video data.

  According to the technique described in Patent Document 2, since hierarchical encoding is used and processing time is long and compression efficiency is poor compared to an encoding method that does not execute hierarchical encoding, high-quality video by the deadline time It is not suitable for use in transmitting data.

  According to the techniques described in Patent Literature 3 and Non-Patent Literature 1, transmission of video data is to be completed by the time limit. However, regarding the transmission of a plurality of video data, it is not considered to efficiently execute the encoding process and the transmission process in parallel on two or more video data.

  Therefore, the present invention performs encoding processing on video data to be transmitted using encoding parameters with the highest possible quality, and performs encoding processing and transmission processing in parallel, and transmits them by the deadline time. An object of the present invention is to provide a data transmission apparatus, program, and method that can complete the process.

According to the present invention, in a data transmission apparatus having an encoding means for encoding video data and a data transmission means for transmitting encoded data to a reception apparatus via a network,
Transmission condition setting means for setting one or more video data to be transmitted and a transmission completion time limit t _L ;
A transmission rate measuring means for measuring the transmission rate in the network;
Calculate the number of allocatable bits D (bit / pixel) per pixel from the total number of pixels based on all frames of all video data and the transmission speed and time limit (transmission completion time limit t _L -transmission start time t ₀ ). Means for calculating the number of pixel bits,
Bit rate calculating means for calculating the bit rate (number of bits per unit time) of the video data from the number D of allocatable bits per pixel and the total number of pixels per unit time of the video data for each video data When,
An encoding parameter selection unit that selects an encoding parameter that can be transmitted at a bit rate for each video data and instructs the encoding unit.

According to another embodiment of the data transmission device of the present invention,
For the encoding time T _E in the encoding parameter for each video data and the transmission time T _T of the encoded data based on the transmission speed, the sum for all video data is calculated, and the values for the sum of the encoding time and the sum of the transmission time are calculated. A transmission processing time calculating means for calculating the larger one as the transmission processing time;
The encoding parameter selection means also preferably selects the encoding parameter so that the transmission processing time for all the video data to be transmitted is within the time limit.

According to another embodiment of the data transmission device of the present invention,
The encoding parameter selection means, when the transmission processing time for all video data to be transmitted is not within the time limit,
From video data with long transmission processing time, one step at a time,
From video data with a short transmission processing time, one step at a time, or
For all video data, one step at a time,
It is also preferable to select the encoding parameter so as to be within the time limit while changing to a parameter that shortens the transmission processing time.

According to another embodiment of the data transmission device of the present invention,
If the transmission processing time for all video data to be transmitted is not within the time limit by the encoding parameter selection means,
From video data with long transmission processing time, one step at a time,
From video data with a short transmission processing time, one step at a time, or
For all video data, one step at a time,
It is also preferable to further include video conversion means for converting the number of pixels in the video data.

According to another embodiment of the data transmission device of the present invention,
It is also preferable that the video conversion means performs conversion to reduce the number of pixels in the video data by reducing the frame rate or reducing the resolution of a specific color component or the resolution of all color components.

According to another embodiment of the data transmission device of the present invention,
Based on a preset upper limit bit rate and lower limit bit rate, further comprising a bit rate adjusting means for adjusting the bit rate output from the bit rate calculating means,
Bit rate adjustment means
If the bit rate is lower than the lower limit bit rate, the video conversion means is instructed to reduce the number of pixels of the video data,
When the bit rate is higher than the upper limit bit rate, it is also preferable to set the bit rate to the upper limit bit rate.

According to another embodiment of the data transmission device of the present invention,
It is also preferable to further include a transmission order changing means for changing the transmission order so that all the video data is transmitted in order from the video data having a larger ratio of the length of the transmission time T _{T to} the length of the encoding time T _E.

According to another embodiment of the data transmission device of the present invention,
The encoding parameter is
The percentage of frames to be encoded using spatial inter-pixel prediction,
Percentage of frames to be encoded using temporal inter-pixel prediction,
The size of the range to search for inter-pixel prediction;
The type of algorithm used to search for inter-pixel prediction,
The size of the quantization parameter,
The ratio of thinning out pixel components in the spatial direction,
The ratio of thinning out the color components of the image,
The number of frames per unit time,
The type of encoding algorithm used for information compression;
It is also preferable to be based on a parameter set including any one of the method types used for pixel frequency conversion or a combination thereof.

According to the present invention, there is provided a data transmission apparatus equipped with a computer, for data transmission for causing a computer to function as an encoding means for encoding video data and a data transmission means for transmitting encoded data to a reception apparatus via a network. In the program of
Transmission condition setting means for setting one or more video data to be transmitted and a transmission completion time limit t _L ;
A transmission rate measuring means for measuring the transmission rate in the network;
Calculate the number of allocatable bits D (bit / pixel) per pixel from the total number of pixels based on all frames of all video data and the transmission speed and time limit (transmission completion time limit t _L -transmission start time t ₀ ). Means for calculating the number of pixel bits,
Bit rate calculating means for calculating the bit rate (number of bits per unit time) of the video data from the number D of allocatable bits per pixel and the total number of pixels per unit time of the video data for each video data When,
An encoding parameter that can be transmitted at a bit rate is selected for each video data, and the computer is caused to function as an encoding parameter selection unit that instructs the encoding unit.

According to the present invention, in a data transmission method in a data transmission device including an encoding unit that encodes video data and a data transmission unit that transmits encoded data to a reception device via a network,
A first step of setting one or more video data to be transmitted, and a transmission completion time limit t _L,
A second step of measuring the transmission rate in the network;
Calculate the number of allocatable bits D (bit / pixel) per pixel from the total number of pixels based on all frames of all video data and the transmission speed and time limit (transmission completion time limit t _L -transmission start time t ₀ ). A third step,
Fourth step of calculating a bit rate (the number of bits per unit time) of the video data from the allocatable number of bits D per pixel and the total number of pixels per unit time of the video data for each video data When,
A fifth step of selecting an encoding parameter that can be transmitted at a bit rate for each video data and instructing the encoding unit.

  According to the data transmission device, program, and method of the present invention, encoding processing is performed on video data to be transmitted using encoding parameters with the highest possible quality, and the encoding processing and transmission processing are performed in parallel. Execute and complete the transmission by the deadline time.

It is a system block diagram used as object. It is a graph showing the change of transmission completion time. It is a functional block diagram of the data transmitter in this invention. It is explanatory drawing showing the transmission processing time in one video data. It is explanatory drawing showing the transmission processing time in all the video data which should be transmitted. It is a flowchart of the data transmitter in this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a target system configuration diagram.

  According to FIG. 1, a system for transmitting video data of news material from a news gathering site to a news bureau is assumed as an object of the present invention. Specifically, “The video data of the news material taken at 15:54 at the interview site should be in time for the broadcast time of the news program (or the deadline time considering the preparation before broadcasting such as editing the material for the program). In addition, it is assumed that the transmission to the news station is completed by 16:30. At this time, the transmitter 1 on the news gathering site connects to the access network and transmits the video data to the data receiver 3 on the news station side via the Internet.

  According to FIG. 1, the interview site is photographed using the camera 2. The camera 2 also has a built-in microphone function, and acquires video data including audio. The camera 2 is connected to the data transmission device 1 via a cable, and transfers the captured video data to the data transmission device 1. As a connection method between the camera 2 and the data transmission device 1, for example, IEEE1394 (i.LINK/FireWire), SDI (Serial Digital Interface), HDMI (High-Definition Multimedia Interface), an analog video terminal and a video capture device are combined. There is something.

  The data transmitting device 1 on the news gathering site side and the data receiving device 3 on the news report side are connected to the Internet via an access network. The access network is, for example, a wired / wireless LAN (Local Area Network), a cellular phone data communication network, a WiMAX (Worldwide Interoperability for Microwave Access), a satellite communication network such as BGAN (Broadband Global Area Network).

The data transmission device 1 encodes the video data acquired from the camera 2 and transmits the encoded data to the data reception device 3 via the Internet. At this time, the data transmission device 1 completes the transmission of the video data by the time limit t _L designated by the user. According to FIG. 1, the deadline time is “16:30” and the time limit (t _L −t ₀ ) is 36 minutes (16: 30-15: 54).

  The quality of this video data depends on the encoding parameter. That is, within 36 minutes of the time limit, video data must be encoded with the highest quality encoding parameters possible, and transmission of the encoded data must be completed.

  In addition, although the data transmission apparatus 1 has been described as including a communication interface unit connected to the access network, a network connection apparatus may be separately provided externally.

  Further, the data transmission device 1 and the camera 2 may be configured integrally.

  Furthermore, the data transmission device 1 is not necessarily limited to acquiring video data from the camera 2, but an external recording medium device such as a USB (Universal Serial Bus) memory, an external hard disk, a video tape, or an SD (Secure Digital) memory card. Video data may be acquired from

  Further, the data transmission device 1 is, for example, a personal computer or a dedicated terminal, and executes encoding processing and transmission processing of video data by application software. Similarly, the data receiving device 3 is also a personal computer or a dedicated terminal, for example, and executes the process of receiving encoded data by application software.

  According to FIG. 1, for example, a user interface screen is displayed on the display of the data transmission device 1 for a photographer at the interview site. For example, a selectable “video data list” is displayed in the left box, and “expiration time” is input in the upper right box. In response to an instruction on the user interface screen, the user can select “video data” to be transmitted and specify “expiration time”. In addition, as a format other than inputting “expiration time”, a difference (time limit) between the current time and the end time may be input. When the user presses the “setting start” button, the encoding process and the transmission process are executed.

  In a video transmission system for news reports, transmission of video data must be completed by the deadline time in consideration of the transmission speed of the network. Also, for urgent coverage, there may be cases where sufficient time cannot be secured for prompt reporting. For this reason, general-purpose equipment such as consumer cameras may be used instead of professional equipment such as commercial cameras used in normal coverage. Also, images taken by ordinary citizens near the site may be provided. In such a case, the resolution and recording bit rate of the captured video data vary depending on the type of captured equipment and the settings of the equipment. For this reason, it is necessary to complete transmission of the video data by the deadline time in a state where video data having different resolutions and recording bit rates are mixed.

In general, the resolution, playback time, and frame rate of video data have the following relationship.
For video data with the same resolution and playback time, the higher the frame rate, the longer the encoding process time.
For video data with the same frame rate and playback time, the encoding process takes longer as the resolution increases.
For video data with the same resolution and frame rate, the longer the playback time, the longer the encoding time.

  Such a relationship means that the encoding processing time is determined based on the number of pixels included in the video data. At this time, the data size of the original video data and the number of pixels do not necessarily match. For example, for a video camera that records video data with AVCHD (Advanced Video Codec High Definition), when shooting video data of one resolution, the recording bit rate is 24 Mbps, 17 Mbps, 9 Mbps, 5 Mbps, up to 5 times There are settings that vary in degree. Also, when the recording method is different such as HD or SD, the recording bit rate is equivalent even though the difference in resolution is four times different. Therefore, it is necessary to allocate a transmission processing time that is not affected by such a difference. In addition, the determination of the bit rate after encoding processing (encoding bit rate) also needs to follow a similar index because the larger the number of pixels, the larger the amount of data necessary to record it.

  FIG. 2 is a graph showing a change in transmission completion time.

The quality of video data depends on the encoding parameters in the encoding process. However, the longer the high-quality video data is generated, the longer the encoding processing time required. Further, the higher the quality of the video data, the larger the data size of the encoded data, and consequently the longer the transmission time. Here, the time from the start of encoding to the end of transmission (transmission end time t _E −transmission start time t ₀ ) is referred to as “transmission processing time”. In this case, it is necessary to select an appropriate encoding parameter by taking a trade-off between the quality of the video data and the transmission processing time.

  In the graph of FIG. 2, the horizontal axis represents the passage of time. FIG. 2A shows transmission processing times when three different parameter sets are used for the three video data. First, the video data is encoded by the parameter set p, and a certain time (encoding time) is required for the processing. Moreover, it can transmit in order from the produced | generated encoded data. In effect, the encoding process and the transmission process are executed in parallel.

FIG. 2B shows the transmission processing time of a plurality of video data. According to FIG. 2 (b), the transmission rate b ₀ to the transmission start time t ₀ is measured, then it is assumed that there is no sudden change in transmission rate. Then, the transmission processing time changes depending on the encoding parameter selection. According to FIG. 2B, when three pieces of video data are processed with the parameter set Pa, the time limit is exceeded. Therefore, when three pieces of video data are processed with the parameter set Pb, it can be understood that transmission of all the video data is completed before the time limit. Further, it can be understood that when processing is performed with the parameter set Pc, transmission of all video data is completed sufficiently earlier than the deadline time. Note that the quality of the video data deteriorates in the order of Pa>Pb> Pb. In FIG. 2, a common parameter set is used for the three video data, but a different parameter set may be used for each video data.

  FIG. 3 is a functional configuration diagram of the data transmission apparatus according to the present invention.

  According to FIG. 3, the data transmission device 1 includes a user interface unit 101, a video data interface unit 102, and a communication interface unit 103 as interfaces with the outside.

  The user interface unit 101 is a display and operation interface for the user. The display displays a user interface screen (see FIG. 1). An operation interface such as a keyboard acquires video data to be transmitted and information on a deadline time from the user.

  The video data interface unit 102 acquires video data from the camera 2 connected via a cable, for example, and outputs the video data to the video data storage unit 111.

  The communication interface unit 103 is connected to the access network and transmits the encoded video data to the data receiving device 3.

  Further, according to FIG. 3, the data transmission device 1 includes a video data storage unit 111, an encoding unit 112, a data transmission unit 113, a transmission condition setting unit 121, a transmission rate measurement unit 122, and a pixel bit number calculation. Unit 123, transmission processing time calculation unit 124, bit rate calculation unit 125, encoding parameter selection unit 126, bit rate adjustment unit 127, video conversion unit 128, and transmission order switching unit 129. These functional components are realized by executing a program that causes a computer mounted on the data transmission apparatus to function.

  The video data storage unit 111 stores the video data input from the video data interface unit 102 in a mass storage medium.

  The encoding unit 112 performs an encoding process on the video data input from the video data storage unit 111 based on the parameter set selected by the encoding parameter selection unit 126. The encoded data is output to the data transmission unit 113.

  The data transmission unit 113 transmits the encoded data to the data reception device 3 via the communication interface unit 103.

The transmission condition setting unit 121 sets one or more video data to be transmitted and a time limit t _L until the transmission of the video data is completed. The transmission conditions may be input by the user himself / herself on the user interface screen (see FIG. 1), or may be set in advance. One or more pieces of video data to be transmitted are stored in the video data storage unit 111. The time limit t _L is the time until transmission is completed for all selected video data.

  The transmission rate measuring unit 122 measures the transmission rate in the network with respect to the data receiving device. As a measuring method, for example, there are techniques such as a packet pair method and a packet train method. Further, the data transmission device may actually transmit a large amount of test data at a time and measure the transmission rate according to the amount of data that has reached the data reception device. For example, when the data transmission apparatus transmits test data at 10 Mbps for several seconds to several tens of seconds, and the data reception apparatus receives the test data at 3 Mbps, the transmission speed is measured at 3 Mbps. The initial value of 10 Mbps for transmitting the test data may be set according to the maximum speed of the network device to be used, such as 100 Mbps. Also, if the data receiving device receives at 10 Mbps when measuring at 10 Mbps (if the network speed may be 10 Mbps or higher), change the setting to 20 Mbps or 50 Mbps and re-measure, and receive data on the receiving side The amount may be measured stepwise until the amount falls below the set value on the transmission side.

  In addition, when long-time transmission is performed, the network speed may vary depending on the time zone. This is because there is a bias in the amount of data flowing through the network in each time zone of the day due to differences in the user's daily activity time zone and the time zone in which the network is used. In order to measure such a long-time characteristic, it is also preferable to transmit rough test data in advance to obtain rough characteristic information.

The pixel bit number calculation unit 123 calculates the allocatable bit number D (bit / pixel) (upper limit) per pixel for all video data. The allocatable bit number D per pixel is calculated from the total number of pixels based on all frames of all video data, and the transmission speed and time limit T _L (transmission completion time limit t _L −transmission start time t ₀ ).

The upper limit D of the allocatable bit number per pixel is calculated by the following formula.
_{D = R × T L / Σ} i = 1 n (y i × x i × f i × T i) formula (1)
R: Network transmission speed
T _L : Time limit
n: Video data (material)
y _i : Number of vertical pixels of video data
x _i : Number of horizontal pixels of video data
y _i × x _i : Number of pixels of one frame of video data
f _i : Frame rate (number of frames per unit time)
T _i : Playback time

Examples of video formats include the following.
HDV1080i (1440 × 1080)
HDV720p (1280 x 720)
DV (720 x 480)

  According to the above-mentioned video format, the HDV method for recording in HD (High Definition) with image sizes of 1440 × 1080 and 1280 × 720 and the DV method for recording in SD (Standard Definition) with image size of 720 × 480 are represented. ing. In the HDV system, MPEG-2 is used to compress video data. HDV1080i records interlaced video with 1080 scanning lines (1440 × 1080 pixels), and HDV720p records progressive video with 720 scanning lines (1280 × 720 pixels). Of course, the present invention is not limited to these formats, and may be non-compressed video data, an AVCHD (Advanced Video Codec High Definition) system compatible with 1920 × 1080 full HD, and other video formats.

  According to the present invention, based on the number of pixels included in all video data to be transmitted, each transmission processing time is calculated from the bit rate for each video data, and it is determined whether or not the transmission is completed within the time limit. To do. The present invention is effective when a plurality of pieces of video data to be transmitted have different resolutions, recording bit rates, and / or frame rates. Video data with a larger number of pixels can be assigned more transmission processing time (more encoding processing time). Regarding the determination of the encoding bit rate, since the bit rate is the data amount per unit time, the value used as the index also uses the number of pixels per unit time of the video data. That is, the transmission processing time for each video data is determined according to the number of pixels that the video data has.

The transmission processing time calculation unit 124 calculates, for each video data, the encoding time T _E in the encoding parameter and the transmission time T _T of the encoded data based on the transmission speed.
T _E : Encoding processing time of each video data T _T : Transmission processing time required for transmission processing of each encoded data T _P : Processing time from transmission start time t ₀ of all video data to transmission completion Encoding processing time =
Time required to process unit time (or pixel) data of video data
× Recording time (or number of pixels) of each video data
Transmission processing time = bit rate of encoded data / transmission speed Here, processing time required to process data per unit time for video data of a certain resolution and frame rate was recorded as a table. Use things. This table may be set by the user or may be derived by measuring the processing time of the encoding process for the test data. Then, the encoding time T _E and the transmission time T _T are output to the encoding parameter selection unit 26.

The bit rate calculation unit 125 calculates the bit rate (the number of bits per unit time) of the video data. The bit rate is calculated for each video data from the allocatable bit number D per pixel and the total number of pixels per unit time of the video data.
_{bi = D × (y i ×} x i × f i) formula (2)
By using Expression (2), even if video data with different recording formats are mixed, the bit rate can be set according to the number of pixels of each video data.

The encoding parameter selection unit 126 selects an encoding parameter that can be transmitted at a bit rate for each video data, and instructs the encoding unit 112. The encoding parameter is based on a parameter set including, for example, any one or a combination of the following.
Ratio of frames to be encoded using inter-pixel prediction in the spatial direction Ratio of frames to be encoded using inter-pixel prediction in the temporal direction Size of the range to search for inter-pixel prediction Used to search for inter-pixel prediction Type of algorithm Size of quantization parameter Ratio of thinning out pixel components in spatial direction Ratio of thinning out color components of image Number of frames per unit time Type of encoding algorithm used for information compression Method type used for pixel frequency conversion Also, In addition to the above, it is also preferable to include a setting item that can be selected for applicability in the video encoding process and that is a trade-off between image quality and processing time.

Table 1 below shows the encoding process time. As the unit time, for example, a time based on “one-second video data” may be used, or the amount of information of the video data as digital data may be used.

Further, according to Table 1, the following three types are represented as video data parameter sets.
"High speed" (Encoding processing time is short but video quality is low)
"Standard" (Encoding processing time and video quality are intermediate between "High speed" and "High quality")
"High quality" (encoding time is long, but video quality is high)
The difference between these three parameters is based on, for example, whether or not the encoding tool set is used in H.264 encoding and the search range of motion compensation. In other words, the encoding process is based on a difference in processing in which there is a trade-off between processing time and quality degradation due to encoding.

  Note that the parameter set is not limited to three, and a plurality of sets may be prepared. Various patterns of resolution and frame rate may be prepared, but patterns limited to some extent may be prepared. For example, in order to estimate the time for processing HD (1440 × 1080, 30 fps) video data, HD (1440 × 1080, 30 fps) video data is test-encoded. On the other hand, assuming that the processing time is proportional to the number of pixels, the time when HD (1440 × 1080, 15 fps) video data is encoded is the same as the above-mentioned HD (1440 × 1080 without performing actual encoding). , 30 fps) half the processing time may be used. When the difference in processing time due to the difference in bit rate is small, measurement data when encoding is performed at a certain bit rate may be used as a value when encoded at another bit rate.

Here, when the encoding time T _E and the transmission time T _T are input from the encoding parameter selection unit 26, the encoding parameter selection unit 126 selects the encoding parameters so as to satisfy the following expression (3).
Max {Σ _{i = 1} ⁿ T _Ei (p), Σ _{i = 1} ⁿ T _Ti (p)} < _TL formula (3)
Max {a, b}: a function that outputs the larger of a or b Σ _{i = 1} ⁿ T _Ei (p): sum of encoding processing times of each video data Σ _{i = 1} ⁿ T _Ti (p): wherein the value of the left side the sum of the transmission time of each image data is the processing time T _P.

Incidentally, the encoding parameter selection unit 126, the processing time T _P is, as a time limit margin time period shorter than the time _{(α · T L, 0 <} α <1) in, also select the encoding parameters preferable. By providing an allowance time, it is possible to avoid exceeding the deadline time based on a change in transmission speed as much as possible.

Encode parameter selection unit 126, the transmission processing time T _P in all of the video data to be transmitted, if not within the time limit, changing the parameters as follows.
-In order from video data with a long transmission processing time, change to a parameter that shortens the transmission processing time by one step.
-In order from video data with a short transmission processing time, change to a parameter that shortens the transmission processing time one step at a time.
-For all video data, change to a parameter that shortens the transmission processing time by one step.
Every time the encoding parameter is changed, the total transmission processing time is recalculated until the time is within the time limit. Then, the processing time T _P in all of the video data to be transmitted is repeated until the time limit.

The bit rate adjustment unit 127 adjusts the bit rate output from the bit rate calculation unit 125. The bit rate after encoding of each video data satisfies the following expression (4).
b _i ≦ D × (y _i × x _i × f _i ) Equation (4)
Here, before the video data is encoded, the value of the bit rate of the video data before the encoding process is divided into the case where the expression (4) is already satisfied and the case where it is not satisfied.

(A) When Expression (4) is satisfied The encoding process is not applied. At this time, T _Ei (p) = 0.

(B) When Expression (4) is not satisfied Encoding processing is applied. As the bit rate value, the maximum bit rate satisfying Expression (4) is selected from the bit rate values of the parameter set shown in Table 1.

Here, an upper limit bit rate and a lower limit bit rate may be set as the bit rate in order to prevent an excessive or excessive bit rate from being set with respect to the image size or frame rate of the video data. In that case, the upper limit bit rate and the lower limit bit rate are preset in the bit rate adjustment unit 127 as appropriate ranges.
“Maximum bit rate”: Bit rate at which the quality does not change even if the bit rate value is increased beyond a certain value. “Lower limit bit rate”: Required to ensure the video quality of the encoded data to some extent bit rate

Thus, an appropriate value is set within a range in which the video quality is maintained and the data amount is not wasted. Further, the upper limit / lower limit bit rate may be set in a range having a margin more than the above limit point. Table 2 is an example of upper and lower limit bit rates. Of course, resolution and frame rate values other than those listed in Table 2 can also be used.

(1) When the bit rate is smaller than the lower limit bit rate (too small), the video conversion unit 128 is instructed to reduce the number of pixels of the video data. For example, this is a case where a remarkably small bit rate is set for high-resolution video data. In this case, in order to prevent the quality of the encoded video data from being significantly lowered, the number of pixels is reduced in order to reduce the amount of data to be transmitted.
(2) When the bit rate is larger than the upper limit bit rate (excessive), the bit rate is set as the upper limit bit rate. For example, this is a case where a significantly large bit rate is set for low-resolution video data.

  Note that the bit rate adjustment unit 127 may be automatically executed, or may clearly indicate to the user interface that “a proper bit rate cannot be set”. As a result, for example, it is assumed that transmission is performed with the resolution and frame rate of the video acquired by the user as they are (assuming that adjustment is mainly performed for bit rate setting), but resolution that is not intended by the user. -It is possible to prevent the frame rate from being lowered.

  When the encoding parameter selection unit 126 does not satisfy Expression (3), the video conversion unit 128 performs conversion to reduce the number of pixels in the video data. This video conversion is performed by reducing the frame rate or by reducing the resolution of a specific color component or the resolution of all color components.

The transmission order changing unit 129 changes the transmission order of the video data so as not to generate as much network bandwidth as possible. When encoded video data is sequentially transmitted, if the data generation amount per unit time of the encoder is less than the throughput, a vacant network bandwidth is generated. In consideration of the possibility that the amount of data that can be transmitted decreases due to a decrease in throughput during transmission, it is desirable to transmit data up to the upper limit of the throughput. For this purpose, the transmission order is changed so that video data having a larger ratio of the length of the transmission time T _{T to} the length of the encoding time T _E is transmitted in order.

  Although not shown in FIG. 3, it is also preferable to have a performance measurement unit and an encoding processing time storage unit for encoding processing time.

  The performance measuring unit encodes the test data for each encoding parameter by the encoding unit 112 and derives an encoding processing time necessary for encoding the video data of unit time for the video data of a certain resolution and frame rate. This is because when the processing capability of the processor or chip set mounted on the data transmission device 1 is different, the encoding processing time is also different. First, the actual encoding process time is obtained by actually performing the encoding process on the test data.

  The performance measurement unit must measure the encoding processing time for each encoding parameter. When there are a large number of parameter sets, the encoding processing time based on the other parameter set may be estimated from the encoding processing time measured by one parameter set. This eliminates the need to measure all parameter sets. Further, the encoding processing time based on the parameter set may be estimated based on the encoding parameter value that greatly affects the encoding processing time. Furthermore, based on the encoding processing times of a plurality of parameter sets, the encoding processing times of other parameter sets may be estimated by linear interpolation, for example.

  As long as the hardware configuration is not changed, the performance measurement unit is considered to have no change in the performance of the device, and only needs to be executed once at the beginning. On the other hand, when the hardware configuration is changed, the performance measurement unit is preferably executed again. Further, when the data transmitting apparatus 1 is a personal computer and the number of softwares that are constantly operating changes, it is considered that the performance has changed, and it is also preferable that the performance measuring unit is periodically executed. .

  The encoding processing time storage unit stores in advance the encoding processing time derived by the performance measurement unit for each encoding parameter. The encoding processing time storage unit is referred to by the transmission processing time calculation unit 124.

  FIG. 4 is an explanatory diagram showing the transmission processing time for one video data.

According to FIG. 4, one video data is encoded by the parameter set p, and represents a time until transmission of the encoded data is completed. The data transmission apparatus executes the encoding process and the transmission process simultaneously in parallel. Also, according to FIG. 4, for example, the bit rate and transmission speed are expressed as follows for video data having a playback time of 1 second and an original bit rate greater than 5 Mbps.
Encoding bit rate: 5Mbps
Network transmission speed: 5Mbps

Referring to FIG. 4, a case where a high quality (low speed) parameter set is used and a case where a high speed (low quality) parameter set is used will be described.
p ₁ : High quality, long encoding process p ₂ : Medium quality, medium encoding process time p ₃ : Low quality, short encoding process time According to this example, the unit time when each parameter set is used The processing time of the video data is as follows.
p ₁ : 2 times p ₂ : 1 time p ₃ : 0.5 times

FIG. 4A shows the case where a high quality (low speed) parameter set is used.
Encoding processing time = 1 second x 2 times = 2 seconds Transmission time = 2 seconds (equal to encoding processing time)
According to FIG. 4A, the transmission time is 2 seconds because it does not end earlier than the encoding processing time. Encoded video data for 5 Mbps × 1 second is generated in 2 seconds. That is, the amount of data output by the encoding is 5 Mbit / 2 seconds = 2.5 Mbps, and only a part of the network bandwidth with a transmission rate of 5 Mbps is used.

  FIG. 4B shows a case where the transmission start time is delayed with respect to FIG. Although the transmission start time is delayed by 1 second, the encoding processing time and the transmission processing time end almost at the same time. This is because the amount of data output from the encoder in 1 to 2 seconds is 2.5 Mbps, and a space of 2.5 Mbps is generated in the network band. Transmission of data output from the encoder in 0 to 1 second is performed in this empty portion. In this case, the entire bandwidth of the network having a transmission rate of 5 Mbps can be used for the transmission time.

  FIG. 4 (c) shows a case where the original bit rate is already 5 Mbps (time is 1 second data), unlike the video data processed in FIGS. 4 (a) and 4 (b). In this case, since there is no need to encode, only the transmission process is executed (occurs when the compression rate of the original video data is already high). In this case, since 5 Mbit data is transmitted over a 5 Mbps network, the transmission time is 1 second.

FIG. 4D shows a case where a high-speed (low quality) parameter set is used.
Encoding processing time = 1 second × 0.5 times = 0.5 second Transmission time = 5 Mbit / 5 Mbps = 1 second Here, 5 Mbit data is generated with a processing time of 0.5 seconds. At this time, data of 10 Mbps is output from the encoder, but the transmission processing is executed at 5 Mbps because the bandwidth of the network is 5 Mbps which is smaller than that. Since the video data after the encoding process is 5 Mbit, the transmission time is 1 second. According to FIG. 4D, the transmission processing time continues even after the end of the encoding processing time.

  FIG. 5 is an explanatory diagram showing the transmission processing time for all video data to be transmitted.

According to FIG. 5, the following three video data are transmitted.
Video data 1 (HD): encoded with high-quality parameters (see FIGS. 4A and 4B)
Video data 2: No need to encode (see FIG. 4C)
Video data 3 (SD): encoded with low quality parameters (see FIG. 4D)

  FIG. 5A shows a case where the first, second, and third video data are transmitted in this order. In this case, it can be understood that the transmission processing time takes 4 seconds. However, there is a vacancy in the network bandwidth between 0 seconds and 2 seconds (transmission at 2.5 Mbps for a 5 Mbps network bandwidth).

  FIG. 5B shows a case where the third, first, and second video data are transmitted in this order. In this case, it can be understood that the transmission processing time takes 3.5 seconds. This is because the transmission time of the first video data took 2 seconds in FIG. 5A, whereas 1.5 seconds could be transmitted in FIG. 5B. However, during the period from 1.5 seconds to 2.5 seconds, transmission can be performed only at 2.5 Mbps with respect to the network bandwidth of 5 Mbps, similarly to the transmission of the first video data in FIG.

FIG. 5C shows a case where the second, third, and first video data are transmitted in this order. In this case, it can be understood that the transmission processing time only takes 3 seconds. This is because in FIG. 5C, the first video data can be transmitted at the same rate of 5 Mbps as the network band. For this reason, the transmission order is changed so that video data having a larger ratio of the length of the transmission time T _{T to} the length of the encoding time T _E is transmitted in order.

  FIG. 6 is a flowchart in the present invention.

(S601) As an initial step, for each encoding parameter, the test data is encoded by the encoding unit, and the encoding processing time is derived. Processing similar to that of the performance measurement unit described above is executed.
(S602) The derived encoding processing time is stored in advance for each encoding parameter. Processing similar to that of the encoding processing time storage unit described above is executed.

(S611) and one or more video data to be transmitted, thereby setting the transmission completion deadline time t _L thereof video data. Processing similar to that of the transmission condition setting unit 121 in FIG. 3 is executed.
(S612) The transmission speed in the network is measured. Processing similar to that of the transmission rate measuring unit 122 in FIG. 3 is executed.
(S613) The number of allocatable bits per pixel D (bit / pixel) from the total number of pixels based on all the frames of all video data, the transmission speed and the time limit (transmission completion deadline time t _L -transmission start time t ₀ ) ) Is calculated. The same processing as that of the pixel bit number calculation unit 123 in FIG. 3 is executed.
(S614) With respect to the encoding time T _E in the encoding parameter for each video data and the transmission time T _T of the encoded data based on the transmission speed, the sum for all the video data is calculated, respectively, and the sum of the encoding time and the total of the transmission time are calculated. One of the larger values is calculated as the transmission processing time. Processing similar to that of the transmission processing time calculation unit 124 in FIG. 3 is executed.
(S615) For each video data, the bit rate (the number of bits per unit time) of the video data is calculated from the allocatable number of bits D per pixel and the total number of pixels per unit time of the video data. The same processing as the bit rate calculation unit 125 in FIG. 3 is executed.
(S616) It is determined whether the bit rate is lower than the lower limit bit rate. If true, the process proceeds to S611 to reduce the number of pixels of the video data. Processing similar to that of the bit rate adjustment unit 127 in FIG. 3 is executed.
(S617) It is determined whether or not the bit rate is larger than the upper limit bit rate.
(S618) If true in S617, the bit rate is set to the upper limit bit rate.
(S619) The encoding parameter is selected so that the processing time for all the video data to be transmitted satisfies Expression (3). Processing similar to that of the encoding parameter selection unit 126 in FIG. 3 is executed.
(S620) Next, the transmission order is changed in order to transmit in order from video data in which the ratio of the length of the transmission time T _{T to} the length of the encoding time T _E is larger. Processing similar to that performed by the transmission order changing unit 129 in FIG. 3 is executed.
(S621) It is determined whether or not the processing time for all the video data to be transmitted satisfies Expression (3).
(S622) If false in S621, the presence / absence of a parameter that shortens the transmission processing time is determined. If there is a parameter that shortens the transmission processing time, the process proceeds to S619 to change to that parameter. Here, encoding is performed so as to be within the time limit, one step at a time in order from video data with a long transmission processing time, one step at a time from video data with a short transmission processing time, or one step at a time for all video data. Select a parameter.
(S623) When the bit rate is smaller than the lower limit bit rate or when there is no parameter that shortens the transmission processing time, conversion is performed to reduce the number of pixels in the video data. Here, the frame rate is reduced one by one in order from the video data with the longest transmission processing time, one step at a time from the video data with the shortest transmission processing time, or one step at a time for all the video data, or Conversion is performed to reduce the number of pixels in the video data by reducing the size. Then, the process of shifting to S613 is repeated again. Processing similar to that of the video conversion unit 128 in FIG. 3 is executed.
(S624) The video data is encoded according to the selected encoding parameter. Processing similar to that of the encoding unit 112 in FIG. 3 is executed.
(S625) The encoded data is transmitted to the receiving device via the network. Processing similar to that of the data transmission unit 113 in FIG. 3 is executed.

  As described above in detail, according to the data transmission apparatus, program, and method of the present invention, encoding processing is performed on video data to be transmitted using encoding parameters that have the highest possible quality, and encoding is performed. Processing and transmission processing are executed in parallel, and transmission can be completed by the deadline time. The user only inputs one or more video data to be transmitted and its time limit to the data transmission device.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Data transmission device 101 User interface part 102 Video | video data interface part 103 Communication interface part 111 Video | video data storage part 112 Encoding part 113 Data transmission part 121 Transmission condition setting part 122 Transmission speed measurement part 123 Pixel bit number calculation part 124 Transmission processing time calculation Unit 125 Bit rate calculation unit 126 Encoding parameter selection unit 127 Bit rate adjustment unit 128 Video conversion unit 129 Transmission order switching unit 2 Camera 3 Data receiving device

Claims (10)

In a data transmission apparatus having an encoding means for encoding video data and a data transmission means for transmitting encoded data to a reception apparatus via a network,
Transmission condition setting means for setting one or more video data to be transmitted and a transmission completion time limit t _L ;
A transmission rate measuring means for measuring a transmission rate in the network;
Based on the total number of pixels based on all frames of all video data, the transmission speed and the time limit (transmission completion time limit t _L -transmission start time t ₀ ), the number of allocatable bits D (bit / pixel) per pixel is calculated. A pixel bit number calculating means for calculating;
A bit rate for calculating the bit rate (bit number per unit time) of the video data from the allocatable bit number D per pixel and the total number of pixels per unit time of the video data for each video data. A calculation means;
A data transmission apparatus comprising: an encoding parameter selection unit that selects an encoding parameter that can be transmitted at the bit rate for each of the video data and instructs the encoding unit. A sum for all the video data of an encoding time T _E in the encoding parameter for each video data and a transmission time T _T of the encoded data based on the transmission speed is calculated, respectively, and the total encoding time and the transmission A transmission processing time calculating means for calculating one of the values with respect to the total time as a transmission processing time,
The data transmission apparatus according to claim 1, wherein the encoding parameter selection unit selects the encoding parameter such that the transmission processing time is within the time limit. The encoding parameter selection means, when the transmission processing time in all video data to be transmitted does not fall within the time limit,
In order from the video data having a long transmission processing time, one step at a time,
In order from the video data with a short transmission processing time, one step at a time, or
For all video data, one step at a time,
The data transmission apparatus according to claim 2, wherein the encoding parameter is selected so that the transmission processing time is within the time limit while changing to a parameter that shortens the transmission processing time. When the transmission time for all video data to be transmitted does not fall within the time limit by the encoding parameter selection means,
In order from the video data having a long transmission processing time, one step at a time,
In order from the video data with a short transmission processing time, one step at a time, or
For all video data, one step at a time,
4. The data transmission apparatus according to claim 3, further comprising video conversion means for converting the number of pixels in the video data to reduce the number of pixels.   The video conversion means performs conversion to reduce the number of pixels in the video data by reducing the frame rate or reducing the resolution of a specific color component or the resolution of all color components. 5. The data transmission device according to 4. Based on a preset upper limit bit rate and lower limit bit rate, further comprising a bit rate adjusting means for adjusting the bit rate output from the bit rate calculating means;
The bit rate adjusting means includes
If the bit rate is smaller than the lower limit bit rate, the video conversion means is instructed to reduce the number of pixels of the video data,
6. The data transmitting apparatus according to claim 4, wherein when the bit rate is larger than the upper limit bit rate, the bit rate is set to the upper limit bit rate. It further has a transmission order changing means for changing the transmission order so that all the video data is transmitted in order from video data having a larger ratio of the length of the transmission time T _{T to} the length of the encoding time T _E. The data transmission device according to any one of claims 2 to 6. The encoding parameter is
The percentage of frames to be encoded using spatial inter-pixel prediction,
Percentage of frames to be encoded using temporal inter-pixel prediction,
The size of the range to search for inter-pixel prediction;
The type of algorithm used to search for inter-pixel prediction,
The size of the quantization parameter,
The ratio of thinning out pixel components in the spatial direction,
The ratio of thinning out the color components of the image,
The number of frames per unit time,
The type of encoding algorithm used for information compression;
8. The data transmission device according to claim 1, wherein the data transmission device is based on a parameter set including any one or a combination of a plurality of scheme types used for pixel frequency conversion. 9. In a data transmission apparatus equipped with a computer, an encoding means for encoding video data, and a data transmission program for causing a computer to function as a data transmission means for transmitting encoded data to a reception apparatus via a network,
Transmission condition setting means for setting one or more video data to be transmitted and a transmission completion time limit t _L ;
A transmission rate measuring means for measuring a transmission rate in the network;
Based on the total number of pixels based on all frames of all video data, the transmission speed and the time limit (transmission completion time limit t _L -transmission start time t ₀ ), the number of allocatable bits D (bit / pixel) per pixel is calculated. A pixel bit number calculating means for calculating;
A bit rate for calculating the bit rate (bit number per unit time) of the video data from the allocatable bit number D per pixel and the total number of pixels per unit time of the video data for each video data. A calculation means;
A program for data transmission, comprising: selecting an encoding parameter that can be transmitted at the bit rate for each video data; and causing the computer to function as an encoding parameter selection unit that instructs the encoding unit. In a data transmission method in a data transmission apparatus having an encoding section that encodes video data and a data transmission section that transmits encoded data to a reception apparatus via a network,
A first step of setting one or more video data to be transmitted, and a transmission completion time limit t _L,
A second step of measuring a transmission rate in the network;
Based on the total number of pixels based on all frames of all video data, the transmission speed and the time limit (transmission completion time limit t _L -transmission start time t ₀ ), the number of allocatable bits D (bit / pixel) per pixel is calculated. A third step of calculating;
For each of the video data, a fourth bit rate (number of bits per unit time) of the video data is calculated from the allocatable bit number D per pixel and the total number of pixels per unit time of the video data. And the steps
A data transmission method comprising: a fifth step of selecting an encoding parameter that can be transmitted at the bit rate for each video data and instructing the encoding unit.

Images