CN101079992B - Synchronous playing device and method for video and audio signals - Google Patents

Abstract

A playing device is used for playing an audio-video signal. The video signal has a video signal portion and an audio signal portion. The playback apparatus includes: a reference time point acquisition unit for acquiring the playing time sequence of the audio signal part as a reference time point; and a video and audio synchronization control unit for controlling the playing of the video signal part according to the reference time point.

Description

Device and method for synchronously playing video and audio signals

technical field

The present invention provides an audio-visual playback device and its method, especially a device and related method for synchronizing video and audio playback according to the time stamp generated by the audio signal part.

Background technique

Technology comes from human nature, and the driving force for all technological progress comes from human beings' longing for a more enjoyable life; if there is anything that can explain the greatest development of human technology, the storage and playback of audio and video must be the most important indicator. If an audio-visual (AV) segment can be recorded with a larger file capacity, the displayed picture will be more refined and meet people's requirements. With the rapid progress of storage media, the file size of audio-visual signals is no longer the core of the main problem, but to improve the way of playing audio-visual information in response to the needs of the public is the direction that all R&D personnel are committed to. Usually, when playing audio-visual files, the most likely problem to encounter is that besides the errors that may occur when the file itself is stored, the out-of-synchronization of the image and sound during playback is the most likely to cause displeasure to the viewers. In this way, good audio-visual Decoding devices will become more and more important in the future leisure market.

Please refer to FIG. 1 . FIG. 1 is a functional block diagram of a conventional audio-visual playing device 100 . The audio-video player 100 is used to drive an external display 112 to display images and drive an external speaker 114 to output sound. As shown in Figure 1, the audio-visual playback device 100 is provided with an audio-visual signal splitting unit 102, an image decoding unit 104, an audio decoding device 106, an image driving unit 108, an audio driving unit 110, and an audio data counting unit 116 , a timing generator 118 , an image data counting unit 120 , a delay detection unit 122 , and a frame control unit 124 . The operation of the audio-visual playing device 100 is described as follows.

First, an audio-visual signal AV_data is input to the audio-visual signal splitting unit 102, and then the audio-visual signal splitting unit 102 further separates the audio-visual signal AV_data into an image signal part DATA_V and an audio signal part DATA_A, wherein the video-audio signal splitting unit 102 divides the video The signal portion DATA_V is transmitted to the video decoding unit 104 , and the audio signal portion DATA_A is transmitted to the audio decoding unit 106 . The image signal part DATA_V includes a plurality of image frames (image frames), and the audio signal part DATA_A includes a plurality of audio frames (audio frames), wherein each audio frame includes a specific number of audio samples, for example, each audio frame can be Contains 8000 audio samples. Then, the plurality of image frames will be sequentially sent to the image decoding device 104 for decoding one by one, and the image decoding device 104 will sequentially input the decoded image frames to the image drive unit one by one 108. Next, the image driving unit 108 drives the display 112 to display the image frames one by one sequentially. In addition, the plurality of audio frames will be sequentially transmitted to the audio decoding unit 106 for decoding, and the audio decoding device 106 will sequentially transmit the decoded audio frames to the audio driving unit 110 one by one. Then, the audio driving unit 110 outputs the received audio frames to the speaker 114 in sequence. It can be known from the above that the image frame and the audio frame are decoded and played separately. Generally speaking, the decoding time required for each audio frame is approximately the same. However, the decoding time required for each image frame is different. When the computing power of the hardware is insufficient or interfered by other variables, it will be impossible to decode a predetermined image frame within the predetermined time, so the playback of the image frame and audio frame may be out of sync. , which in turn affects the playback quality of video and audio signals.

In order to solve such problems, the existing audio-visual playback device 100 provides remedial measures when audio-video playback cannot be synchronized. First, the audio data counting unit 116 reads audio samples from the audio driver unit 106. As mentioned above, each audio frame contains a specific number of audio samples. In addition, the generation time interval of all audio samples is the same. Therefore, the audio data counting unit 116 can calculate the audio frame number to be output by counting the audio samples in the audio driving unit 106, and then obtain the corresponding number of the image frame to be displayed, which is called the first serial number. At this time, the image data counting unit 120 also calculates the number of the image frame that has been decoded and is about to be played, which is called the second number. Next, the delay detection unit 122 compares the first serial number and the second serial number. When the first serial number is the same as the second serial number, it means that the image frame to be played is consistent with the image frame to be played, and thus no adjustment is required for playing the image. When the first number is less than the second number, it means that the speed of video playback exceeds the speed of audio playback, so the frame control unit 124 sends a control signal Sc to the video decoding device 104 to delay the video decoding device 104 from processing the video. Decoding speed of data DATA_V. On the other hand, when the first number is greater than the second number, it means that the video playback speed is slower than the audio playback speed, then the frame control unit 124 sends a control signal Sc to the video decoding device 104 to control video decoding. The decoding device 104 skips decoding certain image frame(s).

However, such a synchronous control method is to adjust the playing time of the image frame to achieve audio-video synchronization when it is observed that there is an out-of-sync situation between the currently played video and audio. That is to say, the playback of the image frame is adjusted only after the out-of-synchronization of the video and audio has occurred. The playback device 100 can only remedy the out-of-sync situation, but cannot prevent the out-of-sync occurrence in advance. That is, the existing video playback device 100 is inefficient in improving the playback quality of videos.

Contents of the invention

Therefore, one of the main objectives of the present invention is to provide an audio-visual signal playback device and related method capable of avoiding asynchronous audio-visual playback, so as to solve the above-mentioned problems.

According to an embodiment of the present invention, a playback device is disclosed for playing an audio-visual (AV) signal, the audio-visual signal includes an image signal part and an audio signal part, and the playback device includes: a A reference time point acquisition unit, used to collect the playback timing of the audio signal part, as a reference time point; and an audio-visual synchronization control unit, used to compare a time stamp with the reference time point, to control the video signal part Playing; wherein, each time stamp is a playing time sequence of a corresponding image frame in the image signal part.

According to this embodiment, a method for playing an audio-visual signal is disclosed. The audio-visual signal includes an image signal part and an audio signal part. The method includes: (a) collecting the playback timing of the audio signal part, and as a reference time point; and (b) comparing a time stamp with the reference time point to control the playback of the image signal part; wherein each time stamp is the playback timing of a corresponding image frame in the image signal part .

The audio-visual signal playback device and method provided by the present invention adjust before the image and audio are out of sync, so the quality of audio-visual playback can be effectively improved.

Description of drawings

FIG. 1 is a functional block diagram of an existing audio-visual playback device.

FIG. 2 is a functional block diagram of an embodiment of an audio-video signal synchronous playback device of the present invention.

Symbol Description:

102, 202 AV signal distribution unit

106, 206 Audio decoding unit

110, 210 Audio drive unit

114, 214 Speakers

118 Timing Generator

122 Delay detection unit

216 Comparison unit

220 Time stamp acquisition unit

224 Reference time point acquisition unit

102, 202 AV signal distribution unit

106, 206 Audio decoding unit

110, 210 Audio drive unit

114, 214 Speakers

118 Timing Generator

Detailed ways

Please refer to FIG. 2 . FIG. 2 is a functional block diagram of an embodiment of an audio-video signal synchronous playback device 200 of the present invention. The video and audio signal synchronous playback device 200 is used to drive an external display 212 to display images and drive an external speaker 214 to output sound. In this embodiment, the audio-visual signal playback device 200 includes an audio-visual signal splitting unit (audio-visual separation unit) 202, a video decoding unit (video decoding unit) 204, an audio decoding unit (audio decoding unit) 206, a An image driving unit 208 , an audio driving unit 210 , a comparison unit 216 , an audio-video synchronization control unit 218 , and a time stamp extraction unit (time stamp extraction unit) 220 . In addition, in this embodiment, the comparison unit 216 includes a register 222 , a reference time point collection unit 224 and a comparator 226 . Wherein, the reference time point acquisition unit 224 is a counter. The function of each component in the audio-video signal synchronous playing device 200 is described as follows.

The audio-visual signal splitting unit 202 is used to further separate the input audio-visual signal AV_data into an image signal part DATA_V and an audio signal part DATA_A, wherein the image signal part DATA_V includes a plurality of image frames (image frames), and the audio signal Part DATA_A contains multiple audio frames (audio frames). Each audio frame contains a certain number of audio samples, for example, each audio frame may contain 8000 audio samples. Then, the plurality of image frames will be sequentially sent to the image decoding unit 204 for decoding one by one, and the image decoding unit 204 will sequentially input the decoded image frames to the image drive unit one by one 208. Next, the image driving unit 208 drives the display 212 to display image frames one by one. In addition, the multiple audio frames will be sequentially transmitted to the audio decoding unit 206 for decoding, and the audio decoding unit 206 will sequentially transmit the decoded audio frames to the audio driving unit 210 one by one. Then, the audio driving unit 210 outputs the received audio frames to the speaker 214 in sequence. It can be known from the above that the image frame and the audio frame are decoded and played separately.

In this embodiment, the time stamp acquisition unit 220 is used to obtain the time stamp of each image frame, wherein each time stamp is a playing time sequence of a corresponding image frame, called the first time sequence. The time stamp acquisition unit 220 transmits the first timing to the audio-video synchronization control unit 218 , and the video-audio synchronization control unit 218 transmits the first timing to the comparison unit 216 to be stored in the register 222 in the comparison unit 216 .

In addition, the reference time acquisition unit 224 in the comparison unit 216 reads the playing timing of the audio frame to be played in the audio driving unit 210, which is called the second timing. The comparator 226 compares the first timing with the second timing, and transmits the comparison result to the audio-video synchronization control unit 218 . When the first timing is smaller than the second timing, the audio-video synchronization control unit 218 will control the image decoding unit 204 to skip the decoding of some (some) image frames, and when the first timing is greater than the second timing, the video-audio synchronization control The unit 218 controls the image decoding unit 204 to delay the decoding of certain (some) image frames. Only when the first timing is the same as the second timing, the audio-video synchronization control unit 218 controls the image driving unit 208 to display the image frame corresponding to the first timing.

Therefore, the audio-visual signal playback device 200 of the present invention will use the second timing collected by the reference timing acquisition unit 224, that is, the playback timing of the audio frame that is about to be played, as the reference timing. When the playback timing corresponding to the image frame, i.e. the first timing, is the same as the second timing, the audio-visual synchronization control unit 218 will control the image drive unit 208 to display the image frame. The audio message frame is matched with the image frame to be played, so as to achieve synchronous playback of audio and video data.

In other words, the audio-visual signal playback device 200 of the present invention uses the time stamp acquisition unit 220 to find out the playback timing (first timing) of the next image frame to be played, and stores it in the register 222 in the comparison unit 216 . When the reference time point collection unit 224 in the comparison unit 216 counts the playing timing (second timing) of a corresponding audio frame, that is, when the first timing is the same as the second timing, it means that the next one that should be played has been reached. The playback timing of the image frame. At this time, the comparing unit 216 sends a control signal to the video-audio synchronization control unit 218 to control the video driving unit 208 to display the next video frame to be played.

Since the audio-visual signal playing device 200 of the present invention uses the time stamp acquisition unit 220 to find out the playing timing of the next image frame to be played, the audio-visual signal playing device 200 of the present invention must pre-set the next image frame to be played. The frame decoding is completed, so that when the reference time acquisition unit 224 counts the playback timing of a corresponding audio frame, the next video frame to be played can be displayed in real time. In other words, the audio-visual signal playing device 200 of the present invention will have a period of pre-processing time before starting to play the audio-visual data. During this time, the audio-visual signal playing device 200 will pre-decode the first image frame. When the reference time point collection unit 224 counts the playing timing of a corresponding first audio frame, the audio-visual signal playing device 200 will play the first image frame and the first audio frame synchronously.

If the reference time acquisition unit 224 has counted the playback timing of a specific audio frame, but the corresponding image frame has not been decoded yet, the audio/video signal playback device 200 of the present invention may have different processing methods. For example, the audio-visual signal playback device 200 of the present invention can be designed to still play the undecoded image frame, but in this case, part of the frame will be noise. In this case, the audio-video synchronization control unit 218 can drive the image driving unit 208 to output the undecoded image frame.

Furthermore, the audio-visual signal playback device 200 of the present invention can also be designed to wait until the corresponding image frame is decoded before playing, but in this case, although the audio-visual signal is played synchronously, there will be a delay occur. In this situation, the audio-video synchronization control unit 218 waits until the image decoding unit 204 completes the decoding operation of the corresponding image frame before driving the image driving unit 208 to output the decoded image frame.

In addition, the audio-visual signal playing device 200 of the present invention can also be designed to pre-decode more than two image frames. In this case, the pre-processing time required by the audio-visual signal playback device 200 will be longer, and more system resources, that is, memory space, will be used to store the pre-decoded image frames.

Assuming that the audio-visual signal playing device 200 of the present invention is designed to decode M image frames in advance, then when the image frame to be displayed is the Nth image frame, the image decoding unit 204 will Decode an image frame. Among them, M, N≥1. As mentioned above, the audio-video signal playing device 200 needs an image frame buffer (not shown) to store the M pre-decoded image frames. Furthermore, the playing timing of the M pre-decoded image frames is also stored in the buffer 222 of the comparison unit 216 .

Compared with the prior art, the audio-visual signal playback device and the related method of the present invention use the time stamp to know the playback timing of the picture frame in advance, so the correction can be completed before the image and audio are out of sync, so that it can be effectively Improve the quality of audio and video playback.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

Claims (19)

1. playing device is used for synchronous playing one video-audio signal, and this video-audio signal includes a signal of video signal part and an audio signals part, and this playing device includes:

One with reference to the time point collecting unit, is used for gathering the broadcast sequential of this audio signals part, with as one with reference to time point; And

One audio-visual synchronous control unit is used for one time of comparison stamp and this with reference to time point, to control the broadcast of this signal of video signal part;

Wherein, each time stamp is the broadcast sequential of a corresponding image frame in this signal of video signal part.

2. playing device according to claim 1, it includes in addition:

One video-audio signal dividing cell is used for this video-audio signal is separated into this signal of video signal part and this audio signals part.

3. playing device according to claim 2, wherein this signal of video signal partly comprises a plurality of image frames, and this playing device includes in addition:

One image decoding unit is coupled to this video-audio signal dividing cell, is used for deciphering described image frame.

4. playing device according to claim 3, it includes in addition:

One image driver element is coupled to this image decoding unit, is used for driving a display to show the image frame after deciphering one by one in proper order.

5. playing device according to claim 2, wherein this audio signals partly comprises a plurality of message frames, and this playing device includes in addition:

One message decoding unit is coupled to this video-audio signal dividing cell, is used for deciphering described message frame.

6. playing device according to claim 5, it includes in addition:

One message driver element is coupled to this message decoding unit, and the message frame after being used for will deciphering in regular turn exports a loud speaker to play described message frame;

Should be to be coupled to this message driver element wherein, and can read the broadcast sequential that is about to the message frame of broadcast in this message driver element with reference to the time point collecting unit.

7. playing device according to claim 2, wherein this signal of video signal partly comprises a plurality of image frames, and this playing device includes in addition:

One time stamp collecting unit is used for obtaining the time stamp of each image frame.

8. playing device according to claim 7, it includes in addition:

One buffer is used for storing the broadcast sequential of this corresponding image frame.

9. playing device according to claim 8, wherein this audio signals partly comprises a plurality of message frames, and this playing device includes in addition:

One comparator is coupled to this buffer and this with reference to the time point collecting unit, is used for relatively being about to the broadcast sequential of the message frame play and the broadcast sequential of this corresponding image frame, to produce at least one comparative result;

Wherein this audio-visual synchronous control unit is decoding and/or the broadcast of controlling this signal of video signal part according to this comparative result.

10. playing device according to claim 1 should be a counter with reference to the time point collecting unit wherein.

11. a method that is used for synchronous playing one video-audio signal, this video-audio signal include a signal of video signal part and an audio signals part, this method includes:

(a) gather the broadcast sequential of this audio signals part, with as one with reference to time point; And

(b) a time stamp and this is with reference to time point, to control the broadcast of this signal of video signal part relatively;

Wherein, each time stamp is the broadcast sequential of a corresponding image frame in this signal of video signal part.

12. method according to claim 11, it includes in addition:

This video-audio signal is separated into this signal of video signal part and this audio signals part.

13. method according to claim 12, wherein this signal of video signal partly comprises a plurality of image frames, and this method includes in addition:

Decipher described image frame.

14. method according to claim 13, it includes in addition:

Drive a display to show the image frame after deciphering one by one in proper order.

15. method according to claim 12, wherein this audio signals partly comprises a plurality of message frames, and this method includes in addition:

Decipher described message frame.

16. method according to claim 15, it includes in addition:

Message frame after will deciphering in regular turn exports a loud speaker to play described message frame; And

Read the broadcast sequential of the message frame that is about to broadcast.

17. method according to claim 12, wherein this signal of video signal partly comprises a plurality of image frames, and this method includes in addition:

Obtain the time stamp of each image frame.

18. method according to claim 17, it includes in addition:

Store the broadcast sequential of this corresponding image frame.

19. method according to claim 18, wherein this audio signals partly comprises a plurality of message frames, and this method includes in addition:

The broadcast sequential of the message frame of relatively being ready to play and the broadcast sequential of this corresponding image frame are to produce at least one comparative result; And

Control the decoding and/or the broadcast of this signal of video signal part according to this comparative result.

Images