JP2003199027A - Time stamp marker and encoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To avoid deteriorating the accuracy of a time stamp and ensure synchronization of video with audio to obtain a stable reproduction quality. <P>SOLUTION: The time stamp marker comprises an ATS calculator 11 for giving time stamps (ATS) to AAU data acquired with first specified time intervals by an audio capture unit 1, based on the first specified time, a VTS calculator 13 for giving time stamps (VTS) to VAU data acquired with second specified time intervals by a video capture unit 3, and a clock function estimator 12 for obtaining an approximate straight line to N sets of lasted acquisition times measured by a PC clock of the AAU data and ATS's given to the AAU data. The VTS calculator 13 sets a plurality of VTS candidate values, based on the second specified time, and gives the nearest or coincident candidate value to time stamp coordinate values of the approximate straight line at the acquisition time measured by the PC clock of the VAU data. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention comprises a time stamp giving device for giving reproduction time information (time stamp) of media data to a real-time medium consisting of video and audio, and a time stamp giving device. The present invention relates to an encoder device that transmits or stores the recorded real-time media.

[0002]

2. Description of the Related Art In recent years, personal computers (PCs)
Due to the higher performance of, the real-time media encoder device can be configured by a PC. As a general configuration, a PC is equipped with a video capture board and an audio capture board, and the video and audio data acquired from these capture boards are encoded by software and stored in a hard disk or in a network. To transmit.

IETF (The Internet Engineering Tas
Request for Comments (RFC) 3016
Describes a communication protocol for delivering an MPEG-4 audio / video stream to the Internet. According to this document, video data and audio data are divided into packets at regular time intervals, and information about the reproduction order (sequence number) and scheduled reproduction time (time stamp) is added and transmitted to the network. Upon receiving the packet, the receiving device determines the reproduction timing based on the time stamp and reproduces the video and audio.

Although the method of adding the time stamp is not described in the above document, it is generally determined based on the acquisition (capture) time of video and audio when the real-time media is encoded into the content for distribution. . In order for the PC encoder device to determine the time stamp, a reference clock is required.

Conventionally, the PC clock managed inside the PC has been used as such a reference clock. In a general PC that supports multitasking, the accuracy of the PC clock is about 1 [msec] minimum, but multiple tasks are executed, so ± 15 depending on the situation.
An error of about [msec] may occur.

[0006]

However, the video capture is operated by the internal clock which is asynchronous with the PC clock, and the audio capture is also operated by the internal clock which is asynchronous with these clocks.
For this reason, if the capture operation is continued for a long time, the accuracy of the time stamp of the video data or audio data deteriorates.

As a result, in the receiving device, the time stamps of the video data and the audio data may gradually become larger or smaller than the normal value. In the former case, the received data buffer overflows and the playback is disturbed due to the decrease in the playback speed. In the latter case, the received data underflows and the playback is disturbed due to the increased playback speed. To do.

Further, since the internal clock of the video capture and the internal clock of the audio capture have an asynchronous relationship, it is difficult to reproduce the video and the audio in synchronization (AV synchronization).

Further, the PC clock itself is ± 15 [mse
Since there is an error of about c], fluctuations occur in the time stamp and playback is not stable.

As described above, when the distribution contents are created by the conventional encoder device which determines the time stamp by the PC clock, the receiving device is likely to overflow or underflow the receiving data buffer, and at the same time, the AV synchronous reproduction is performed. There was a problem that it became difficult and stable reproduction could not be performed.

The present invention has been made in order to solve such a conventional problem, and it is possible to prevent deterioration of precision of a time stamp, guarantee synchronization of video and audio, and obtain stable reproduction quality. The purpose is to

[0012]

SUMMARY OF THE INVENTION A time stamping apparatus of the present invention provides audio data obtained at a first predetermined time measured by a first clock to audio data obtained based on the first predetermined time. An audio time stamp means for giving a time stamp, a video time stamp giving means for giving a video time stamp to the video data acquired every second predetermined time measured by the second clock, and Clock function estimating means for obtaining an approximate straight line for the latest N (N is an integer of 2 or more) sets of the acquisition time measured with the clock of 3 and the audio time stamp given to the audio data. The video time stamp assigning means is configured to generate a plurality of video time stamps based on the second predetermined time. A complementary value is set, and the candidate value that is closest to or coincides with the time stamp coordinate value of the approximate straight line at the acquisition time of the video data measured at the third clock is given as the video time stamp. It is characterized by.

Further, another time stamp assigning device of the present invention comprises a video time stamp means for giving a video time stamp to the video data acquired at every first predetermined time measured by the first clock, A video time stamp giving means for giving a video time stamp based on the second predetermined time to the video data acquired every second predetermined time measured by the second clock; The acquisition time measured by the clock,
Clock function estimating means for obtaining an approximate straight line for the latest N (N is an integer of 2 or more) pairs with the video time stamp given to the video data, wherein the audio time stamp giving means is A plurality of candidate values for the audio time stamp are set based on a predetermined time of 1, and the audio data is closest to or coincides with the time stamp coordinate value of the approximate line at the acquisition time measured at the third clock. The above candidate value is added as the audio time stamp.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment FIG. 1 is a block diagram of an encoder device equipped with a time stamp giving device according to a first embodiment of the present invention. The encoder device of FIG. 1 includes an audio capture unit 1, a PC clock unit 2, a video capture unit 3, a transmission unit 4,
The time stamp adding device 10 according to the first embodiment is provided. The time stamp assigning device 10 includes an ATS calculator 11, a clock function estimator 12, and a VTS calculator 1.
3 and 3.

In the encoder device of FIG. 1, the audio capture unit 1 acquires audio (audio access unit, AAU) data of a fixed size, and outputs this AAU data to the transmission unit 4. In addition, the audio capture unit 1 outputs an AAU end signal indicating the end to the ATS calculation unit 11 at the same time as the end of acquisition and output of one AAU data. In this audio capture unit 1, the internal clock (first clock) of the audio capture unit 1 is used as a reference,
The AAU data is acquired every first predetermined time measured by the first clock.

The PC clock unit 2 has a PC clock (third
Of the clock) of the clock estimation function unit 12 and VT
It is output to the S calculation unit 13.

The video capture unit 3 acquires video (video access unit, VAU) data of a fixed size and outputs this VAU data to the transmission unit 4. Further, each time the video capture unit 3 finishes the acquisition and output of one VAU data, at the same time as the end, it outputs a VAU end signal indicating the end to the VTS calculation unit 13.
In this video capture unit 3, the video capture unit 3
The internal clock (second clock) is used as a reference, and V is measured every second predetermined time measured by this second clock.
AU data is acquired.

The ATS calculator 11 receives the AAU end signal as an input and outputs an audio time stamp (ATS) to be added to the AAU data corresponding to the AAU end signal to the transmitter 4 and the clock function estimator 12.

The clock function estimating section 12 receives the ATS and PC clock values as inputs, and outputs the clock function parameters to the VTS calculating section 13.

The VTS calculator 13 receives the VAU end signal, P
The C clock value and the clock function parameter are input, and the video time stamp (VTS) added to the VAU data corresponding to the VAU end signal is output to the transmission unit 4.

The transmission unit 4 uses AAU data, ATS, VA.
The U data and VTS are input, and the transmission packet is distributed to the network. The transmission unit 4 audio-encodes the AAU data and adds VTS to it, video-encodes the VAU data and adds VTS to it, and divides these into packets to generate the above-mentioned transmission packet.

FIG. 2 is an operation explanatory diagram of the time stamp giving apparatus 10 according to the first embodiment of the present invention. The operation of the time stamp giving apparatus 10 according to the first embodiment will be described below with reference to FIGS. 1 and 2.

[ATS calculator 11] ATS calculator 11
When AAU data is acquired by the audio capture unit 1 and an AAU end signal is sent at the same time,
The ATS is calculated based on the U data size and output to the transmission unit 4 and the clock function estimation unit 12. For example, 10
When configuring an AAU for each [msec], the above AA
When the U data size is 10 [msec], the ATS increases by 10 [msec] each time the AAU end signal is input. Therefore, the ATS is given with reference to the internal clock (first clock) of the audio capture unit 1.

[Clock function estimating section 12] When the ATS is input, the clock function estimating section 12 reads the PC clock value from the PC clock section 2 and uses this PC clock value of the AAU data to which the ATS is added. Considered as acquisition time. Therefore, the acquisition time of the AAU data is PC
It is based on the clock (third clock) and is measured at the third clock.

Further, the clock function estimation unit 12 is
An approximate straight line for the latest N sets (N is an integer of 2 or more) of the acquisition time of the U data and the ATS to which the AAU data is added is obtained. Then, the clock function estimation unit 1
2 outputs the parameter of this approximate straight line to the VTS calculator 13 as a clock function parameter.

In FIG. 2, the horizontal axis represents the coordinates of the acquisition time of the AAU data and VAU data with respect to the PC clock (the coordinates of the PC clock value regarded as the acquisition time of the AAU data and VAU data), and the vertical axis represents the ATS. And V
It is a time stamp coordinate of TS. In addition, ATS1-A
TS7 is an ATS added to seven AAU data for every 10 [msec], A1 to A7 are a set of the acquisition time and ATS of the above seven AAU data, and L is these A1 to A7.
Is a linear approximation of.

[VTS calculator 13] VTS calculator 13
, VAU data is acquired by the video capture unit 3,
At the same time, when the VAU end signal is notified, a plurality of VTS candidate values are set based on the VAU data size. For example, in the case where VAU is composed of each frame of video having a frame rate of 29.97 [Hz] (1
/29.97 [sec] every VAU is configured), the above AAU data size is 1 / 29.97 [se
As c], the above plurality of candidate values are set at intervals of 1 / 29.97 [sec].

Furthermore, when the VTS calculation unit 13 is notified of the VAU end signal at the same time as the acquisition of the VAU data, the VTS calculation unit 13 reads the PC clock value from the PC clock unit 2 and acquires the PC clock value from the VAU data. Consider it as time. The acquisition time of the VAU data is based on the PC clock (third clock) and is measured at the third clock.

Then, the VTS calculator 13 determines the VTS
Among the plurality of candidate values, the candidate value closest to or matching the time stamp coordinate value of the approximate straight line at the acquisition time of the VAU data is searched for, and the candidate value is output to the transmission unit 4 as VTS. As a result, the deviation of VTS is
It can be corrected according to TS.

In FIG. 2, VTS1 and VTS2 are VTSs added or added to two VAU data respectively composed of video frames having a frame rate of 29.97 [Hz]. Also, T1 is VTS1
The acquisition time of the VAU data added with, T2 is VTS2
Is the acquisition time of the VAU data to which is added. Also, V
Reference numeral 1 is a set of the acquisition time T1 of VAU data and VTS1, and V2 is a set of the acquisition time T2 of VAU data and VTS2.

Further, it is assumed that V1 in FIG. 2 is a set of the acquisition time of the first video frame (first frame of video) and VTS1. Actually, when VTS1 is the VTS of the first video frame, VTS1 = VTS
It is extremely rare that it becomes 2 + 2 / 29.97 [sec]. This is an assumption for facilitating the understanding of the present invention.

First, when the VTS end signal of the first video frame (VAU data) is notified, the VTS calculation unit 13 reads the PC clock value from the PC clock unit 2 and uses it to acquire the VAU data acquisition time T1. To consider.
Note that no VTS candidate value is set for the first video frame. Then, the VTS calculation unit 13 adds the time stamp coordinate value of the approximate straight line L at the acquisition time T1 to the VAU data of the first video frame VT.
S, that is, VTS1. Therefore, V1 in FIG. 2 is the intersection of the acquisition time T1 and the approximate straight line L.

Next, when the VAU end signal of the video frame (VAU data) acquired next to the first video frame is notified, the VTS calculator 13 sets a plurality of candidate values for VTS2. The plurality of candidate values of this VTS2 are
VTS1 + n / 29.97 [sec] (n is 0, 1, 2,
3, ...).

1 / PC from the acquisition of the first video frame
When the second video frame is accurately acquired at an interval of 29.97 [msec], VTS1 + 1/2 is set as VTS2.
It is correct to add 9.97. However, if the PC consumes the CPU for processing other tasks, the video frame may not be acquired in some cases, and the video frame acquisition interval becomes p / 29.97 [sec] (p is 2, 3, ...). The case happens.

When the capture operation is continued for a long time, the time measured by the internal clock (first clock) of the audio capture unit 1 and the time measured by the internal clock (second clock) of the video capture unit 3 There is a case where the deviation from and becomes 1 / 29.97 [sec] or more.

In preparation for such a case, the VTS calculation unit 13 provides VTS1 + 1/2 even if the PC accurately acquires the video frames at 1 / 29.97 [sec] intervals.
In addition to 9.97 [sec], VTS1 + q / 29.97 [s
ec] (q is 0, 2, 3, ...) Is set as a candidate value for VTS2. This makes it possible to correct the deviation of the internal clock of the video capture unit 3 in accordance with the internal clock of the audio capture unit 1.

When the VTS calculator 13 is notified of the VAU end signal of the video frame (VAU data) acquired next to the first video frame, the VTS calculator 13 reads the PC clock value from the PC clock unit 2 and The VAU
Considering this as the acquisition time T2 of the data, an acquisition time area (hatched area in FIG. 2) composed of acquisition time coordinate ranges on both sides of the acquisition time T2 is set.

The above acquisition time region is set in consideration of the fluctuation of the PC clock. In FIG. 2, assuming that the variation of the PC clock is ± 15 [msec], the acquisition time region of ± 15 [msec] is set around the acquisition time T2.

Then, the VTS calculator 13 determines the VTS
A common area between the plurality of candidate values of 2 and the acquisition time area of the VAU data is obtained, and a common area that is closest to or intersects with the straight line approximation L is searched for in the obtained common area, and the common area is determined. The transmission unit 4 uses VTS2 as a candidate value to be configured.
Output to. As a result, the deviation of the internal clock of the video capture unit 3 can be corrected according to the internal clock of the audio capture unit 1, so that the deviation of the VTS can be corrected.
It can be corrected according to.

In FIG. 2, B1 and B are the common areas.
2, B3, ... Are obtained, and among these common areas, the common area B3 intersects the approximate straight line L, so the candidate value VTS1 + 2 / 29.97 [sec] of the common area B3 is selected and output as VTS2. It V2 in FIG. 2 is the intersection of the acquisition time T2 and the common area B3.

Setting the candidate value of the common area B3 of FIG. 2 to VTS2 means that the time stamp coordinate range of the approximate straight line L within the acquisition time area (hatched area of FIG. 2) including the acquisition time T2 of the VAU data is the best. A candidate value that is close or coincident is to be VTS2. This means that the acquisition time T2 of VAU data is set without setting the acquisition time area described above.
The candidate value that is closest to or coincides with the time stamp coordinate value of the approximate straight line L at is nothing but VTS2. Therefore, it is not always necessary to set the above acquisition time region.

However, the above-mentioned time stamp coordinate value is V
Since the probability of coincidence with the TS candidate value is low, the VTS calculation unit 13 must determine which of the two VTS candidate values is closest to the above-mentioned time stamp coordinate value when the acquisition time region is not set. The probability that it must be
The processing in the TS calculation unit 13 is burdensome. Therefore, FIG.
By setting the acquisition time region of the VAU data in consideration of the fluctuation of the PC clock as described above, the probability that any one common region intersects the linear approximation L becomes high, and either of the two common regions becomes the above-mentioned. Since the probability of having to determine whether it is the closest to the time stamp coordinate range is low, the processing in the VTS calculation unit 13 can be reduced.

As described above, in the first embodiment, the AT based on the internal clock of the audio capture unit 1 is used.
An approximate straight line between S and the acquisition time with the PC clock of the AAU data as a reference is obtained, and VT is set so as to approach this approximate straight line.
Since S is corrected, ATS and VTS are unified and output at a time with the internal clock of the audio capture unit 1 as a common and unique reference. The internal clock of the audio capture unit 1 is generally more accurate than the internal clock of the video capture unit 3 and the PC clock.

Therefore, since the precision of the time stamp does not decrease even during long-time operation, overflow or underflow does not occur in the reception data buffer of the receiving device. Further, since the reference clock for giving the time stamp is common, the synchronization of the video and audio is guaranteed. Further, since the time stamp does not fluctuate due to the error of the PC clock, stable reproduction quality can be obtained.

Second Embodiment FIG. 3 is a block diagram of an encoder device having a time stamp assigning device according to a second embodiment of the present invention. In FIG. 3, the same or corresponding parts as those in FIG. 1 are designated by the same reference numerals. The encoder device of FIG. 3 includes an audio capture unit 1 and a PC.
Clock unit 2, video capture unit 3, transmission unit 4
And the time stamp assigning apparatus 20 of the second embodiment. The time stamp assigning device 20 uses the ATS
It is composed of a calculation unit 21, a clock function estimation unit 12, and a VTS calculation unit 13.

The encoder apparatus of FIG. 3 is the encoder apparatus of FIG. 1 except that the time stamp assigning apparatus 10 is a time stamp assigning apparatus 20. The time stamp assigning apparatus 20 includes the ATS calculator 11 in the time stamp assigning apparatus 10 of the first embodiment.
This is the TS calculation unit 21. In the second embodiment, the PC clock value output from the PC clock unit 2 is also input to the ATS calculation unit 21. The clock function parameter output from the clock function estimation unit 12 is also input to the ATS calculation unit 21.

The ATS calculator 21 is provided with a function of correcting the ATS value in the ATS calculator 11 of FIG. This ATS calculator 21 receives the input AA
According to the U end signal, a temporary time stamp is set in the corresponding AAU data, and the temporary time stamp is corrected as necessary, whereby the ATS of the corresponding AAU data is corrected.
And outputs this ATS to the transmission unit 4 and the clock function estimation unit 12.

FIG. 4 is an operation explanatory diagram of the time stamp giving device 20 according to the second embodiment of the present invention. 4, the same or corresponding parts as those in FIG. 2 are designated by the same reference numerals. The operation of the time stamp assigning apparatus 20 according to the second embodiment will be described below with reference to FIGS. 3 and 4. The operations of the clock function estimating unit 12 and the VTS calculating unit 13 are the same as those of the time stamp assigning device 10 (see FIGS. 1 and 2) of the first embodiment, and therefore the operations of the ATS calculating unit 21 are the same. Mainly explained.

[ATS calculator 21] ATS calculator 21
When AAU data is acquired by the audio capture unit 1 and an AAU end signal is sent at the same time,
Calculate the temporary time stamp based on the U data size,
This temporary time stamp is set in the AAU data. For example, when configuring the AAU every 10 [msec], the provisional time stamp is incremented by 10 [msec] each time the AAU end signal is input, with the AAU data size being 10 [msec]. Therefore, the temporary time stamp is given with reference to the internal clock (first clock) of the audio capture unit 1.

When the AAU end signal is notified simultaneously with the acquisition of the AAU data, the ATS calculation unit 21 reads the PC clock value from the PC clock unit 2 and
The clock value is regarded as the acquisition time of the AAU data. The acquisition time of the AAU data is based on the PC clock (third clock) and is measured at the third clock.

Further, the ATS calculation unit 21 determines whether or not the temporary time stamp deviates from the time stamp coordinate value at the acquisition time of the approximate straight line obtained by the clock function estimation unit 12 by a predetermined value or more. To do. If the deviation of the temporary time stamp is less than the predetermined value, the above temporary time stamp is added as ATS. Also,
If the deviation of the temporary time stamp is equal to or more than a predetermined value, it is considered that the acquisition of the immediately previous AAU data has failed, and the temporary time stamp is corrected to the above time stamp coordinate value, and this time stamp coordinate value is set as ATS. Give. As a result, when the AAU data cannot be acquired due to the overload of the PC or the like, the correct ATS can be added to the AAU data that has been normally acquired thereafter.

In FIG. 4, ATS1 to ATS3, AT
S5 to ATS7 are A added to 6 AAU data
TS, A1 to A3, A5 to A7 are a set of the acquisition times of the above six AAU data and ATS, and L is these A1 data.
Is a linear approximation of A3 to A3 and A5 to A7. Also, A4 is
It is a set of the temporary time stamp before correction and the acquisition time of the AAU data in which the temporary time stamp is set,
A5 is ATS5 with this temporary time stamp corrected,
It is a set with the acquisition time of the AAU data. In addition, A1
For the AAU data of A3 to A3, A5 to A7, the set temporary time stamps are directly added as ATS.

In FIG. 4, in the ATS calculator 43, A
The provisional time stamp of A4, which is 10 [msec] after TS3, is determined to be apart from the approximate straight line L by a predetermined value or more, and A4 is corrected to A5 on the approximate straight line L.

As described above, in the second embodiment, AA
When the provisional time stamp of the U data deviates from the approximate straight line by a predetermined value or more, the ATS is corrected to the time stamp coordinate value of the approximate straight line, so even if the acquisition of the AAU data fails, it is acquired after that. A correct ATS can be added to the generated AAU data.

Third Embodiment FIG. 5 is a block diagram of an encoder device including a time stamp giving device according to a third embodiment of the present invention. In FIG. 5, the same or corresponding parts as those in FIG. 1 are designated by the same reference numerals. The encoder device of FIG. 5 includes an audio capture unit 1 and a PC.
Clock unit 2, video capture unit 3, transmission unit 4
And the time stamp assigning device 30 of the third embodiment. The time stamp assigning device 30 uses the ATS
It is composed of a calculation unit 31, a clock function estimation unit 32, and a VTS calculation unit 33.

In the third embodiment, the accuracy of the internal clock (second clock) of the video capture section 3 is determined by the accuracy of the internal clock (first clock) and the PC clock (third clock) of the audio capture section 1. ) Is higher than this is effective.

The encoder apparatus of FIG. 5 is the encoder apparatus of FIG. 1 except that the time stamp assigning apparatus 10 is a time stamp assigning apparatus 30. The time stamp assigning apparatus 30 includes the ATS calculating unit 11, the clock function estimating unit 12, and the VTS calculating unit 13 in the time stamp assigning apparatus 10 of the first embodiment, respectively. , VTS calculator 33. In the third embodiment, the PC clock value output from the PC clock unit 2 is input to the ATS calculation unit 31 and the clock function estimation unit 32, and is not input to the VTS calculation unit 33. Also,
The clock function parameter output from the clock function estimation unit 32 is input to the ATS calculation unit 31 and is not input to the VTS calculation unit 33.

The VTS calculator 33 receives the VAU end signal and outputs a video time stamp (VTS) to be added to the VAU data corresponding to the VAU end signal to the transmitter 4 and the clock function estimator 32.

The clock function estimation unit 32 receives the VTS and PC clock values as inputs, and outputs the clock function parameters to the ATS calculation unit 31.

The ATS calculator 33 receives the AAU end signal, P
The C clock value and the clock function parameter are input, and the Odin time stamp (ATS) added to the AAU data corresponding to the AAU end signal is output to the transmission unit 4.

The operation of the time stamp assigning apparatus 30 of the third embodiment will be described below. In the third embodiment, the ATS calculator 31 uses the VT of the first embodiment.
The same operation as the S calculation unit 13 is performed, and the VTS calculation unit 33 performs the same operation as the ATS calculation unit 11 of the first embodiment.

[VTS calculator 33] VTS calculator 33
, VAU data is acquired by the video capture unit 3,
At the same time, when the VAU end signal is notified, VTS is calculated based on the VAU data size and output to the transmission unit 4 and the clock function estimation unit 12. For example, 1/2
When configuring the VAU every 9.97 [sec], the VAU data size is set to 1 / 29.97 [sec], and the VTS becomes 1 / E each time the VAU end signal is input.
It increases by 29.97 [sec]. Therefore, VTS is
It is given based on the internal clock (second clock) of the video capture unit 3.

[Clock function estimating unit 32] When the VTS is input, the clock function estimating unit 32 reads the PC clock value from the PC clock unit 2 and uses this PC clock value of the VAU data to which the VTS is added. Considered as acquisition time. Therefore, the acquisition time of the VAU data is PC
It is based on the clock (third clock) and is measured at the third clock.

Further, the clock function estimation unit 32 uses the VA
An approximate straight line for the latest N sets (N is an integer of 2 or more) of the acquisition time of the U data and the VTS to which the VAU data is added is obtained. Then, the clock function estimation unit 3
2 outputs the parameter of this approximate straight line to the ATS calculation unit 31 as a clock function parameter.

[ATS calculator 31] ATS calculator 31
When AAU data is acquired by the audio capture unit 1 and an AAU end signal is sent at the same time,
Set multiple ATS candidate values based on the U data size. For example, when configuring an AAU every 10 [msec], the above-mentioned plurality of candidate values are set at intervals of 10 [msec] with the AAU data size being 10 [msec].

Further, when the ATS end signal is notified simultaneously with the acquisition of the AAU data, the ATS calculation unit 31 reads the PC clock value from the PC clock unit 2 and acquires this PC clock value for the AAU data. Consider it as time. The acquisition time of the AAU data is based on the PC clock (third clock) and is measured at the third clock.

Then, the ATS calculation section 31 uses the ATS
Among the plurality of candidate values, the candidate value closest to or matching the time stamp coordinate value of the approximate straight line at the acquisition time of the AAU data is searched for, and the candidate value is output to the transmission unit 4 as the ATS. As a result, the ATS deviation is V
It can be corrected according to TS.

As described above, in the third embodiment, an approximate straight line between the VTS based on the internal clock of the video capture unit 3 and the acquisition time based on the PC clock of the VAU data is obtained, and this approximate straight line is obtained. Since the ATS is corrected so as to approach, the VTS and the ATS are unified and output at the time with the internal clock of the video capture unit 3 as a common and only reference.

Therefore, since the precision of the time stamp does not decrease even during long-time operation, overflow or underflow does not occur in the reception data buffer of the receiving device. Further, since the reference clock for giving the time stamp is common, the synchronization of the video and audio is guaranteed. Further, since the time stamp does not fluctuate due to the error of the PC clock, stable reproduction quality can be obtained.

Fourth Embodiment FIG. 6 is a block diagram of an encoder device including a time stamp giving device according to a fourth embodiment of the present invention. In FIG. 6, the same or corresponding parts as those in FIG. 5 are designated by the same reference numerals. The encoder device of FIG. 6 includes an audio capture unit 1 and a PC.
Clock unit 2, video capture unit 3, transmission unit 4
And the time stamp assigning device 40 of the third embodiment. The time stamp assigning device 40 uses the ATS
It is composed of a calculation unit 31, a clock function estimation unit 32, and a VTS calculation unit 43.

Also in the fourth embodiment, as in the third embodiment, the accuracy of the internal clock (second clock) of the video capture unit 3 depends on the internal clock (first clock) of the audio capture unit 1. Clock) and PC clock (third clock).

The encoder device of FIG. 6 is the encoder device of FIG. 5 in which the time stamp assigning device 30 is replaced with the time stamp assigning device 40. The time stamp assigning device 40 is the same as the time stamp assigning device 30 of the third embodiment except that the VTS calculator 33 is set to V
This is the TS calculation unit 43. In the fourth embodiment, the PC clock value output from the PC clock unit 2 is also input to the VTS calculation unit 43. The clock function parameter output from the clock function estimation unit 32 is also input to the VTS calculation unit 43.

The VTS calculator 43 is provided with a function of correcting the VTS value in the VTS calculator 33 of FIG. This VTS calculator 43 receives the input VA
According to the U end signal, a temporary time stamp is set in the corresponding VAU data, and the temporary time stamp is corrected as necessary to correct the VTS of the corresponding VAU data.
And outputs this VTS to the transmission unit 4 and the clock function estimation unit 32.

The operation of the time stamp assigning apparatus 40 of the fourth embodiment will be described below. The operations of the clock function estimating unit 32 and the ATS calculating unit 31 are the same as those of the time stamp assigning device 30 (see FIG. 5) of the third embodiment.
The operation of the VTS calculator 43 will be mainly described because it is the same as the above. The VTS calculator 43 operates in the same manner as the ATS calculator 21 of the second embodiment.

[VTS calculator 43] VTS calculator 43
, VAU data is acquired by the video capture unit 3,
At the same time, when the VAU end signal is notified, a temporary time stamp is calculated based on the VAU data size, and this temporary time stamp is set in the above VAU data. For example, when a VAU is configured every 1 / 29.97 [sec], the above VAU data size is 1 / 29.97.
As [sec], each time the VAU end signal is input,
The temporary time stamp increases by 1 / 29.97 [sec]. Therefore, the temporary time stamp is given with reference to the internal clock (second clock) of the video capture unit 3.

Further, the VTS calculator 43 reads the PC clock value from the PC clock unit 2 when the VAU end signal is notified simultaneously with the acquisition of the VAU data, and the PC clock value is read from the PC clock unit 2.
The clock value is regarded as the acquisition time of the VAU data. The acquisition time of the VAU data is based on the PC clock (third clock) and is measured at the third clock.

Further, the VTS calculation unit 43 determines whether or not the temporary time stamp deviates from the time stamp coordinate value at the acquisition time of the approximate straight line obtained by the clock function estimation unit 32 by a predetermined value or more. To do. If the deviation of the temporary time stamp is less than the predetermined value, the above temporary time stamp is added as VTS. Also,
If the deviation of the temporary time stamp is more than a predetermined value, it is considered that the acquisition of the immediately previous VAU data has failed, and the temporary time stamp is corrected to the above time stamp coordinate value, and this time stamp coordinate value is set as VTS. Give. As a result, when the VAU data cannot be acquired due to the overload of the PC or the like, the correct VTS can be added to the VAU data that has been normally acquired thereafter.

As described above, in the fourth embodiment, VA
If the provisional time stamp of the U data deviates from the approximate straight line by a predetermined value or more, the VTS is corrected to the time stamp coordinate value of the approximate straight line, so even if the VAU data acquisition fails, it is acquired after that. The correct VTS can be added to the generated VAU data.

In each of the above embodiments, A
The time length of AU is 10 [msec], and the time length of VAU is 1
/29.97 [sec], PC clock time fluctuation ±
Although described as 15 [msec], these values may be appropriately changed according to the video / audio encoding method used, the specifications of the video / audio input source, the specifications of the PC, and the like. Also in this case, the same effect as that of the above-described embodiment can be obtained.

In each of the above embodiments, the case where the real-time medium is transmitted to the network has been described, but the present invention is also applicable to the case where the real-time medium is stored in a recording medium such as a hard disk. In this case, the correct time stamp information is transmitted when another distribution server having a transmission function distributes the above-mentioned stored contents, and therefore improvement in reproduction quality can be expected as in the above-described embodiment. .

[0081]

As described above, according to the time stamp assigning apparatus of the present invention, it is possible to prevent the accuracy of the time stamp from being lowered, guarantee the synchronization of video and audio, and obtain stable reproduction quality. .

[Brief description of drawings]

FIG. 1 is a configuration diagram of an encoder device including a time stamp assigning device according to a first embodiment of this invention.

FIG. 2 is an operation explanatory diagram of the time stamp giving device according to the first embodiment of the present invention.

FIG. 3 is a configuration diagram of an encoder device including a time stamp giving device according to a second embodiment of the present invention.

FIG. 4 is an operation explanatory diagram of the time stamp giving device according to the second embodiment of the present invention.

FIG. 5 is a configuration diagram of an encoder device including a time stamp giving device according to a third embodiment of the present invention.

FIG. 6 is a configuration diagram of an encoder device including a time stamp giving device according to a fourth embodiment of the present invention.

[Explanation of symbols]

1 audio capture part, 2 PC clock part,
3 video capture part, 4 transmission part, 10, 2
0,30,40 time stamp giving device, 11,2
1,31 ATS calculation unit, 12,32 Clock function estimation unit, 13,33,43 VTS calculation unit.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G11B 27/00 H04N 5/91 Z H04N 5/92 5/92 HF term (reference) 5C053 FA23 GB06 GB37 JA03 JA22 5D044 AB05 AB07 DE02 DE03 DE14 DE24 DE25 DE39 FG21 GK12 HL14 5D110 AA27 AA29 DA17 DB05 DC02 DC17

Claims (7)

[Claims] 1. A time stamp giving apparatus for giving a time stamp to a real-time medium composed of video and audio, wherein the first data is added to the audio data acquired every first predetermined time measured at a first clock. An audio time stamp means for giving an audio time stamp based on a predetermined time, and a video time stamp giving means for giving a video time stamp to the video data acquired every second predetermined time measured by the second clock. And a clock for obtaining an approximate straight line for the latest N (N is an integer of 2 or more) pairs of the acquisition time measured at the third clock of the audio data and the audio time stamp given to the audio data. A function estimating means, and the video time stamp adding means is A plurality of candidate values for the video time stamp are set based on a second predetermined time, and are closest to the time stamp coordinate value of the approximate straight line at the acquisition time of the video data measured at the third clock, or A time stamp assigning device, which assigns the matching candidate value as the video time stamp. 2. The video time stamp assigning means sets a plurality of candidate values of the video time stamp based on the second predetermined time, and obtains the video data at an acquisition time measured by a third clock. The acquisition time area including the acquisition time is set based on the acquisition time area, and the candidate value closest to or matching the time stamp coordinate range of the approximate straight line in the acquisition time area is given as the video time stamp. The time stamp assigning device according to claim 1. 3. The audio time stamp assigning means sets a temporary time stamp on the audio data based on the first predetermined time, and the temporary time stamp is measured at the third clock of the audio data. If the time stamp coordinate value of the approximated straight line at the obtained acquisition time deviates from the time stamp coordinate value by a predetermined value or more, the time stamp coordinate value is given as the audio time stamp, and the temporary time stamp deviates from the time stamp coordinate value. If the value is less than a predetermined value, the temporary time stamp is added as the audio time stamp. 4. A time stamp assigning device for assigning a time stamp to data of real-time media consisting of video and audio, wherein the video time is added to video data acquired every first predetermined time measured by a first clock. A video time stamp means for giving a stamp, and a video time stamp for giving a video time stamp based on the second predetermined time to the video data acquired every second predetermined time measured by the second clock. Approximate straight lines for the latest N (N is an integer of 2 or more) sets of the adding means, the acquisition time measured at the third clock of the video data, and the video time stamp added to the video data. And a clock function estimating means for obtaining the audio time stamp. A plurality of candidate values of the audio time stamp are set based on time, and the candidate closest to or matching the time stamp coordinate value of the approximate straight line at the acquisition time of the audio data measured at the third clock. A time stamp giving apparatus, wherein a value is given as the audio time stamp. 5. The audio time stamp assigning means sets a plurality of candidate values for the audio time stamp based on the first predetermined time, and sets the plurality of candidate values for the audio time stamp at an acquisition time measured at a third clock of the audio data. The acquisition time region including this acquisition time is set based on the above, and the candidate value closest to or matching the time stamp coordinate range of the approximate line in the acquisition time region is given as the audit time stamp. The time stamp giving device according to claim 4. 6. The video time stamp adding means sets a temporary time stamp on the video data based on the second predetermined time, and the temporary time stamp is measured at the third clock of the video data. If the time stamp coordinate value of the approximated line at the obtained acquisition time deviates by a predetermined value or more,
The time stamp coordinate value is given as the video time stamp, and if the deviation of the temporary time stamp from the time stamp coordinate value is less than a predetermined value, the temporary time stamp is given as the video time stamp. The time stamp giving apparatus according to claim 4. 7. An encoder device for adding a time stamp to a real-time medium composed of video and audio, transmitting or storing the time stamp, comprising the time stamp adding device according to any one of claims 1 to 6. Encoder device.