CN101466045B - Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system - Google Patents
Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system Download PDFInfo
- Publication number
- CN101466045B CN101466045B CN2007101726059A CN200710172605A CN101466045B CN 101466045 B CN101466045 B CN 101466045B CN 2007101726059 A CN2007101726059 A CN 2007101726059A CN 200710172605 A CN200710172605 A CN 200710172605A CN 101466045 B CN101466045 B CN 101466045B
- Authority
- CN
- China
- Prior art keywords
- decode rate
- pts
- video
- rate mode
- video decode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a self-adapting decoding synchronous device, a self-adapting decoding synchronous method and a video decoding and display system; wherein, the self-adapting decoding synchronous device comprises a PTS synchronous unit, a display buffer unit and a decoding speed adjusting unit. The PTS synchronous unit is used for receiving PTS information decoded by a video decoder and comparing PTS with a system clock to output a comparing result; the display buffer unit is used for storing the control information of the image data to be displayed and controlling the output of the control information stored in the display buffer unit according to the comparing result of the PTS synchronous unit; and the decoding speed adjusting unit is used for adjusting the decoding speed of the video decoder according to the overflow state or underflow state of the display buffer unit. In addition to the self-adapting decoding synchronous device, the video decoding and display system further comprises the video decoder, a frame buffer and a post-processing and display unit. According to the device, the method and the system, the synchronous effect of decoding and displaying can be achieved by adjusting the decoding speed in a self-adapting manner; besides, the conversion of decoding frame rate and the displaying of multi skillful mode can also be realized.
Description
Technical field
The relevant image processing techniques of the present invention, especially relevant for the synchronizer and the method for synchronous of video decode and demonstration, and video decode and display system.
Background technology
The audio/video encoding/decoding system is because therefore the time-delay that has storage and transmit need utilize synchro system to reach the synchronous playing of audio, video data.For example, for traditional simulated television, the transmission delay end to end of each frame data is fixed, and sending and receiving is strict synchronism.But for digital compression systems, the shared data volume of each two field picture is different, can change to some extent according to the coded system and the complexity of image, and the transmission channel code check of general radio broadcasting is fixed.Therefore, for each frame, transmission delay is variable, can't reach naturally synchronously between transmission and the demonstration.In the audio/video encoding/decoding technical standard MPEG series standard, data are compressed the back packing, transmit with the code stream form.Moving Picture Experts Group-2 is provided with relevant clock information in the header information of code stream, for example in the header information of transport stream TS, add program clock reference PCR, decoded time stamp DTS and show information such as timestamp PTS, by these temporal informations reach the coding and decoding end synchronously and audio frequency, video and broadcast synchronous.
For video data decoding synchronously, industry scheme relatively more commonly used is to adopt decoded time stamp DTS synchronization scenario at present.Yet, in the video code flow of mpeg standard definition, generally include three kinds of different picture frames, promptly can be used as reference frame I frame that other image prediction uses, only carry out the P frame of forward prediction and can carry out bi-directional predicted B frame.Show that timestamp PTS can the corresponding zero-time that shows, and the zero-time that decoded time stamp DTS can corresponding decoding.I frame, P frame and B frame all can carry in head (header) information of code stream usually and show timestamp PTS, and just have decoded time stamp DTS sometimes in head (header) information of code stream.Therefore, when adopting decoded time stamp DTS synchronization scenario, the partial frame that does not carry decoded time stamp DTS just can't carry out synchronous clock adjustment.When adopting DTS synchronous,, need frame-skipping if find that system clock lags behind DTS.I frame and P frame are also referred to as key frame (Anchor Frame), can not at will skip, and have only the B frame to skip, otherwise will influence more other frames that will use I frame and P frame, make float more severe.
Also there are some to utilize at present in the prior art and show that timestamp PTS carries out synchronous scheme.For example, the name be called " Method And Apparatus For Controlling Display Time Point OfMPEG Bit Stream Of Recording Medium " the 7th, 068, No. 915 United States Patent (USP)s disclose a kind of to show that timestamp PTS carries out synchronous apparatus and method.This device comprises timer and PTS controller.Under the normal decoder pattern, timer is set initial value according to system clock SCR, and under the special solution pattern, the PTS controller can receive and store the PTS of default image, and with the PTS of the storage initial value as timer.Comparator compares the system clock STC of timer output and the PTS of default image.Only when the PTS of the system clock STC of timer output and default image was identical, comparator just can be exported a displays the command signal.This patent has been avoided the shake of display end better by adopting PTS to carry out making that the demonstration of video image is more accurate synchronously at display end, meets the basic conception of audio video synchronization more.Simultaneously also reduced the unnecessary frame-skipping that DTS brings synchronously, what make frame-skipping and image repeats more smoothly, and has avoided the waste of decoder waits.
But this patent is only reached the synchronous of demonstration by PTS, the speed of decoding is not adjusted, and therefore, need provide a large amount of cushion spaces to deposit decoded data.And speed of video decoding be not to be fully uniformly, and Video Decoder often has certain delay to display end, postpones also may exist inhomogeneous for each frame data.
On the other hand, along with the continuous development of digital sound, video coding and decoding technology, relatively, more requirement has been proposed also on performance and function the electronic product that relates to sound, video decode.For example, increase, increase the support of technique mode (Trick Mode) etc. for the support of output equipment line by line.The decode rate of the display frequency of output equipment and decoder is usually also inequality line by line, needs by frame rate conversion, just can finish the harmonious of output equipment and video code flow line by line.Under technique mode (TrickMode), video be required to finish various different rates F.F., rewind down, put slowly, frame by frame, quick play mode such as location, this has also brought a difficult problem synchronously for the decoding of video code flow and broadcast.And in order to make video code flow can adapt to the television broadcasting system of different systems, for example conversion between TSC-system formula and the pal mode, the frame per second that video decoding system need carry out between the different systems is changed mutually.Adopt decoded time stamp DTS synchronization scenario, only be that the decoding and the coding side of video code flow are reached synchronously, and can not carry out the conversion of frame per second.The 7th, 068, though No. 915 United States Patent (USP)s adopt the PTS synchronization scenario, owing to correspondingly decoding speed is not adjusted, thereby can not realize the conversion of frame per second simultaneously, improve the performance and the function of the electronic product that relates to sound, video decode with lower cost.
Summary of the invention
The problem that will solve of the present invention provides a kind of adaptive decoding synchronous device, can reach the synchronous of decoding and demonstration by adjusting decode rate adaptively, and can realize the conversion of decoded frame rate and the demonstration of many technique modes simultaneously.
Another problem that the present invention will solve provides a kind of adaptive decoding method for synchronous.
The present invention to solve another be problematic in that a kind of adaptive video decode and display system be provided.
According to an aspect of the present invention, provide a kind of adaptive decoding synchronous device, comprising:
The PTS lock unit, the PTS information that goes out in order to the receiver, video decoder decode, and PTS compared with system clock, export a comparative result;
The display buffer unit in order to the control information of view data to be shown such as storage, and leaves the output of described control information wherein in according to the comparative result control of described PTS lock unit;
The decode rate adjustment unit is in order to the decoding speed of adjusting Video Decoder according to the overflow state or the underflow condition of display buffer unit.
According to above-mentioned synchronizer, described decode rate adjustment unit comprises overflow counter and the underflow counter in order to the overflow state and the underflow condition of display buffer unit are counted respectively.
According to above-mentioned synchronizer, described decode rate adjustment unit also comprises the self adaptation state machine that multiple decode rate state model is provided in order to the decode rate according to Video Decoder.
According to a further aspect in the invention, provide a kind of video decode and display system, comprise Video Decoder;
Frame buffer zone is in order to deposit vedio data;
The PTS lock unit, the PTS information that goes out in order to the receiver, video decoder decode, and PTS compared with system clock, export a comparative result;
The display buffer unit in order to the control information of view data to be shown such as storage, and leaves the output of described control information wherein in according to the comparative result control of described PTS lock unit;
The decode rate adjustment unit is in order to the decoding speed of adjusting Video Decoder according to the overflow state or the underflow condition of display buffer unit; And
Reprocessing and display unit are in order to further to handle the synchronized images data and to show.
According to above-mentioned system, described reprocessing directly links to each other with described PTS lock unit with display unit.
According to a further aspect in the invention, provide a kind of adaptive decoding method for synchronous, may further comprise the steps:
A. accelerate according to the demand of image display speed or slow down the access speed of PTS;
B. by accelerating or slow down the access speed of PTS to be influenced the amount of storage of view data control information in the display buffer unit;
C. when the amount of storage of view data control information reaches the overflow state of display buffer unit, send overflow signals; When the amount of storage of view data control information reaches the underflow condition of display buffer unit, send underflow signal;
D. when the underflow signal of accumulative total reaches preset threshold value, current video decode rate mode is brought up to higher video decode rate mode;
E. when the overflow signals of accumulative total reaches preset threshold value, current video decode rate mode is reduced to lower video decode rate mode.
According to above-mentioned method, when current video decode rate mode is the highest video decode rate mode or minimum video decode rate mode, provide corresponding event to the upper strata that drives.
According to above-mentioned method, current video decode rate mode is the first video decode rate mode, when the underflow signal of accumulative total reaches presetting first threshold, current video decode rate mode is brought up to the second video decode rate mode; When overflow signals occurring, provide corresponding event to the upper strata that drives.
According to above-mentioned method, current video decode rate mode is the second video decode rate mode, when the underflow signal of accumulative total reaches default the 3rd threshold value, current video decode rate mode is brought up to the 3rd video decode rate mode; When the overflow signals of accumulative total reaches default second threshold value, current video decode rate mode is reduced to the first video decode pattern.
According to above-mentioned method, current video decode rate mode is the 3rd video decode rate mode, when the underflow signal of accumulative total reaches default the 5th threshold value, current video decode rate mode is brought up to the 4th video decode rate mode; When the overflow signals of accumulative total reaches default the 4th threshold value, current video decode rate mode is reduced to the second video decode pattern.
According to above-mentioned method, current video decode rate mode is the 4th video decode rate mode, when the underflow signal of accumulative total reaches default the 7th threshold value, provides corresponding event to the upper strata that drives; When the overflow signals of accumulative total reaches default the 6th threshold value, current video decode rate mode is reduced to the 3rd video decode pattern.
According to above-mentioned method, among the step a to the judgement of the demand of image display speed by the PTS acquisition of comparing with system clock, when PTS slows down to the access speed of PTS during greater than system clock; When PTS accelerates access speed to PTS during less than system clock.
According to above-mentioned method, the first video decode rate mode is the normal decoder pattern, and the second video decode rate mode is for skipping the single frames decoding schema, and the 3rd video decode rate mode is the key frame decoding schema.
According to above-mentioned method, the 4th video decode rate mode is a skipped frame sequential decoding pattern.
The present invention adopts PTS to carry out audio video synchronization, and the PTS to code stream compares at the video display end, utilizes the sky of display buffer unit completely to drive the variation of decode rate, makes audio video synchronization needn't be subjected to the control of DTS.At the video decode end, provide the unified state machine of a cover to realize the speed adjustment.This state machine defines the various state models that Video Decoder can be adjusted self decode rate speed, and can adaptively between various state models, move, thereby the speed of adaptive adjustment decoding is used for following the display end PTS desired speed of success synchronously.Simultaneously, for frame rate conversion, the present invention's state by utilizing the display buffer unit and automatic state machine fully unites the adjustment of decode rate and finishes.For Trick Mode, the present invention is according to its corresponding speed, adjusts the speed of axle growths/minimizing lock in time, and finally still state by the display buffer unit and automatic state machine are united the adjustment of decode rate and finished.Video synchronizing system also can be finished frame rate conversion and TrickMode simultaneously among the present invention, makes systemic-function strengthen, and needn't increase extra cost.
Description of drawings
The following drawings is the aid illustration to exemplary embodiment of the present, to the elaboration of the embodiment of the invention, be to disclose feature of the present invention place, but do not limit the present invention in conjunction with the following drawings for further, same-sign is represented respective element or step among the embodiment among the figure, wherein:
Fig. 1 is the structured flowchart according to an adaptive decoding synchronous device of the present invention.
Fig. 2 is the structured flowchart of decode rate adjustment unit according to an embodiment of the invention.
Fig. 3 is the structured flowchart of video decode and display system according to an embodiment of the invention.
Fig. 4 is the fundamental diagram of self adaptation state machine according to an embodiment of the invention.
Fig. 5 is the flow chart according to an adaptive decoding method for synchronous of the present invention.
Fig. 6 is the flow chart of adaptive decoding method for synchronous in accordance with another embodiment of the present invention.
Fig. 7 is the flow chart of adaptive decoding method for synchronous according to still another embodiment of the invention.
Fig. 8 is the flow chart of adaptive decoding method for synchronous according to still another embodiment of the invention.
Embodiment
Video is the module of a key synchronously in the video decode, in one embodiment of the present of invention, adopts mode soft, that hardware combines that audio video synchronization is proposed improved plan.Among other embodiment of the present invention, also can directly utilize hardware circuit to realize audio video synchronization.In integrated circuit was realized, synchronizer of the present invention can the existing hardware resource of multiplexing video decoding system self, as Video Decoder and frame buffer etc.The present invention utilizes the demonstration timestamp (PTS) in the code stream to carry out the synchronous of video.
With reference to figure 1, the adaptive decoding synchronous device that the present invention proposes comprises display buffer unit 11, PTS lock unit 12 and decode rate adjustment unit 13 at least.The control information of the view data that display buffer unit storage etc. are to be shown.The control information here for example can comprise the information such as scaling of aspect ration, frame/field information and the reprocessing of view data.The control information of each frame image data is corresponding with the two field picture in leaving frame buffer in, and being decoded from code stream by Video Decoder draws.The PTS lock unit receives the PTS information that decodes, and PTS is compared with system clock (STC).The display buffer unit is according to PTS lock unit result relatively, and the output of control information is wherein left in control in.The decode rate adjustment unit can be adjusted the decoding speed of Video Decoder according to the state of display buffer unit.
When the PTS lock unit compared PTS and system clock, as PTS during greater than system clock, PTS was out of date in expression, that is to say that decoding speed is faster than display speed.At this moment, the corresponding control information in the display buffer unit is suspended output.And the decoding of decoder is also underway, will continue the control information to display buffer unit input image data.The control information of the view data of depositing in the display buffer unit surpasses the maximum size of display buffer unit, and then overflow can take place in the display buffer unit.As PTS during less than system clock, the expression decoding speed has been slower than display speed.At this moment, the control information of output image data can be accelerated in the display buffer unit, and the speed that decoder is sent into the view data control information to the display buffer unit still remains unchanged, therefore, the amount of storage of the view data control information in the display buffer unit can reduce gradually, the control information of the view data of depositing in the display buffer unit is lower than the capacity lower limit of display buffer unit, makes display buffer unit generation underflow.In another embodiment of the present invention, the control information of view data and corresponding view data is deposited in the display buffer unit.Overflow or underflow condition are indicated according to the view data and the corresponding control information thereof of wherein depositing in the display buffer unit.
The overflow of display buffer unit and underflow condition can define according to the demand of practical application.According to one embodiment of present invention, if for having taken place in empty once reasonably the fetch data operation of (Display FIFO Get) of display buffer unit is then produced display buffer unit underflow event one time in the display buffer unit.What is called is reasonably to the peek operation of display buffer unit, once the fetch data PTS of the resulting frame of operation of display buffer unit lost efficacy before being meant, the frame that must take out new PTS once more shows.The display buffer unit can be provided with the threshold value that frame number produces overflow.When Video Decoder by frame decoder is managed, during the permission that obtains to decode, the phase related control information of the frame that trial will show is put into the display buffer unit, frame number in the display buffer unit has reached or has surpassed certain overflow threshold value at this moment, then produces display buffer unit overflow incident.
In one embodiment of the invention, when display buffer unit generation overflow, send an overflow signals and give the decode rate adjustment unit; When display buffer unit generation underflow, send a underflow signal and give the decode rate adjustment unit.
Referring to Fig. 2, according to one embodiment of present invention, can comprise overflow counter 131 and underflow counter 132 in the decode rate adjustment unit 13, respectively the overflow state and the underflow condition of display buffer unit are counted.For example, when the display buffer unit overflow state occurred at every turn, the overflow counter was from increasing 1; When underflow condition occurring, the underflow counter is from increasing 1 at every turn.After the decode rate of decoder changes, overflow counter and underflow counter O reset.
Be provided with self adaptation state machine 133 in the decode rate adjustment unit 13, the self adaptation state machine can be divided into multiple decoded state pattern according to the difference of the decode rate of Video Decoder.For example, in one embodiment of the invention, Video Decoder can be adjusted under four kinds of different decode rate and decode, and correspondingly, the self adaptation state machine also is arranged to four kinds of decode rate states.The self adaptation state machine can carry out transition adaptively according to the state of detected display buffer unit between these four kinds different decode rate states.Be understandable that in other embodiments of the invention, Video Decoder can be adjusted to the different decode rate of other quantity, and the self adaptation state machine also can relative set becomes the decode rate state of other quantity.
As shown in Figure 4, in one embodiment of the invention, the self adaptation state machine 133 of decode rate adjustment unit 13 has following four kinds of decode rate states, and the decode rate state that speed of video decoding then can be set according to the state model of state machine is adaptively adjusted.
First kind of decode rate state is normal decoder pattern (Normal Decode Mode) 41.Under the normal decoder pattern, Video Decoder all solves respectively according to the maximum capacity of self decoding each frame with video code flow.At this moment, when the display buffer unit provides underflow signal, only utilize the underflow counter that underflow signal is added up,,, for example be adjusted into and skip the single frames decoding schema to higher rate transition between states when the underflow number of times of accumulative total during greater than the first threshold T1 that sets; When the display buffer unit provided overflow signals, the decode rate adjustment unit provided corresponding event to the upper strata that drives.When the upper strata that drives provides corresponding event, show that the adjustment of the decode rate of Video Decoder can not have been satisfied the limit that PTS requires synchronously, at this moment, for example utilize upper layer software (applications) that Video Decoder is handled, temporarily hinder Video Decoder and decode.
Second kind of decode rate state is for skipping single frames decoding schema (Single Skip Mode) 42.Skipping under the single frames decoding schema, Video Decoder is still attempted each frame of video code flow is all solved respectively.Different with the normal decoder pattern is when the display buffer unit provides underflow signal, when the underflow counter adds up underflow signal, to make Video Decoder attempt skipping a frame.In the execution mode, seek the B frame and skip preferably).When the underflow number of times of accumulative total during greater than the 3rd threshold value T3 that sets, to higher rate transition between states, key frame decoding schema for example; When the display buffer unit provided overflow signals, the overflow counter added up overflow signals.When the overflow number of times of accumulative total during greater than the second threshold value T2 that sets, to more low rate transition between states, normal decoder pattern for example.
The third decode rate state is a key frame decoding schema (Anchor Decode Mode) 43.Under the key frame decoding schema, Video Decoder is the key frame (AnchorFrame) in the decoded video code stream only, i.e. I frame and P frame, and the B frame in the code stream (B Frame) is all filtered out.When the display buffer unit provides underflow signal, only underflow signal is added up by the underflow counter.When the underflow number of times of accumulative total during greater than the 5th threshold value T5 that sets, to higher rate transition between states, skipped frame sequential decoding pattern for example.When the display buffer unit provided overflow signals, the overflow counter added up overflow signals.When the overflow number of times of accumulative total during,, for example skip the single frames decoding schema to more low rate transition between states greater than the 4th threshold value T4 that sets.
The 4th kind of decode rate state is skipped frame sequential decoding pattern (Sequence Skip Mode) 44.Under skipped frame sequential decoding pattern, Video Decoder attempts seeking the head of each frame sequence (Sequence), and first frame (being generally the I frame) of this frame sequence is decoded, and then seeks the head of next frame sequence.When the display buffer unit provides underflow signal, when the underflow counter adds up underflow signal, attempt skipping a frame sequence, and directly search the head of next frame sequence.When the underflow number of times of accumulative total during greater than the 7th threshold value T7 that sets, provide anomalous event to the upper strata that drives, the ability that shows the acceleration decoding of Video Decoder itself has surpassed the limit that PTS requires synchronously, at this moment needs the cooperation of other modules of system just can finish the synchronous requirement of PTS.For example, demultiplexing unit is sought head etc. according to random access information (Random Access Info).When the display buffer unit provided overflow signals, the overflow counter added up overflow signals, when the overflow number of times of accumulative total during greater than the 6th threshold value T6 that sets, and to more low rate transition between states, key frame decoding schema for example.
The first threshold T1 that mentions in aforementioned four kinds of decode rate states, the second threshold value T2, the 3rd threshold value T3, the 4th threshold value T4, the 5th threshold value T5, the 6th threshold value T6 and the 7th threshold value T7 etc. all can adjust according to practical application.
When adopting DTS synchronous, key frame (Anchor Frame) I frame and P frame can not be skipped, and can only skip the B frame, otherwise can have influence on other frame that will use I frame and P frame.And the present invention has adopted PTS synchronous, no-load speed requirement according to display end, when the display buffer unit reflects that decode rate is less than showing speed, and when speed differs big, state machine can be adjusted to skipped frame sequential decoding pattern, allow to skip the part key frame, decode rate is significantly improved, help the repetition of frame-skipping more smoothly and frame.
Referring to Fig. 3, according to one embodiment of present invention, video decode and display synchronization system comprise Video Decoder 31, decode rate adjustment unit, the frame buffer (not shown) of depositing vedio data, the display buffer unit 34 of depositing the view data control information that can adjust the decode rate of Video Decoder, carry out synchronous PTS lock unit 35 and reprocessing and display unit 36 according to PTS.Video Decoder is used for the video code flow that receives is decoded.It is temporary that the view data that video decoder decodes obtains is sent into frame buffer, and it is temporary that the display buffer unit is sent in the view data control information that decoding obtains.View data control information in the display buffer unit is corresponding with the view data in leaving frame buffer in.Reprocessing and display unit are used for view data is further processed, and display image.Among the present invention, reprocessing directly links to each other with the PTS lock unit with display unit, directly utilizes PTS to carry out synchronously at display end, because display end makes that directly in the face of the user demonstration of video is more accurate, also meets the notion of audio video synchronization more.
The PTS information that video decoder decodes goes out is sent into the PTS lock unit.The PTS lock unit compares PTS and system clock (STC).The display buffer unit is according to PTS lock unit result relatively, and the output of control information is wherein left in control in.The decode rate adjustment unit can be adjusted the decoding speed of Video Decoder according to the state of display buffer unit.The operation principle of PTS lock unit, display buffer unit and decode rate adjustment unit can be referring to aforementioned description to adaptive decoding synchronous device.
Video decode of the present invention and display synchronization system provide the unified state machine of a cover to realize the synchronous of video decode at the Video Decoder end.This state machine defines the various state models that are used for adjusting Video Decoder self decode rate speed, and can adaptively between various state models, move, thereby the speed of adaptive adjustment decoding, thereby can follow the display end PTS desired speed of success synchronously.At this, decoder needn't wait for that the synchronized result of DTS need to judge whether frame-skipping, and only need adjust the decode rate of decoder according to the signal adaptive ground of display buffer unit feedback by the self adaptation state machine, thus can make full use of the high speed performance of decoder, improve the efficient of decoder.Simultaneously, because state machine can be decoder different decode rate is set, the frame-skipping frequency of decoder also can correspondingly be adjusted, thereby avoids unnecessary frame-skipping.
Because output equipment and video code flow and inequality line by line, when video decode and display system increase during to the support of output equipment line by line, correspondingly, the frame per second of output equipment and video code flow itself need be changed line by line.The television broadcasting system that be to adapt to different systems, for example NTSC and pal mode, video decode and display system also need to finish the frame rate conversion between the different systems.According to one embodiment of present invention, when video decode and the conversion of display system conducting frame rate, for example be transformed into for 60 frame/seconds from 30 frame/seconds, every frame time of staying reduced to 1/60th seconds from 1/30th seconds, the decode rate of Video Decoder need be added fast again.At this moment, needn't special frame rate conversion module be set, finish but utilize PTS lock unit, display buffer unit and the decode rate adjustment unit of adaptive decoding synchronous device to unite at Video Decoder.The PTS lock unit judges according to the needs that show whether the PTS of present frame is expired, when expired, directly abandons the PTS of present frame, and takes the PTS of next frame.Like this, the consumption of display buffer unit has been speeded up, and Video Decoder is still decoded with original speed this moment and offer the display buffer unit.Because the display buffer unit produces the speed imbalance with consumption, the view data control information that the display buffer unit is deposited is fewer and feweri, until producing underflow.The display buffer unit sends the underflow condition signal to the decode rate adjustment unit when underflow condition occurs.Suppose that current decode rate is in the normal decoder pattern, then when the underflow number of times of the display buffer unit of accumulative total during greater than the first threshold T1 that sets, the decode rate adjustment unit will automatically be adjusted the decode rate of Video Decoder, make decode rate to higher rate transition between states, for example be adjusted into and skip the single frames decoding schema.Skipping when Video Decoder and can make the production and consumption of display buffer unit reach balance under the single frames decoding schema, then the decode rate of Video Decoder will maintain and skip the single frames decoding schema.If the control information that the display buffer unit is deposited is still consumption greater than production, at the underflow number of times of the display buffer unit of accumulative total during greater than the second threshold value T2 that sets, the decode rate adjustment unit will automatically be adjusted the decode rate of Video Decoder, make decode rate further to higher rate transition between states, for example be adjusted into the key frame decoding schema.If skipping under the single frames decoding schema, the control information that the display buffer unit is deposited consumption occurs less than the situation of producing, at the overflow number of times of the display buffer unit of accumulative total during greater than the 4th threshold value T4 that sets, the decode rate adjustment unit will automatically be adjusted the decode rate of Video Decoder, make decode rate to more low rate transition between states, for example come back to the normal decoder pattern.And may between these two kinds of decoding schemas, adjust repeatedly constantly, reach balance with the production and consumption that makes the display buffer unit adaptively.Certainly, first threshold T1 and/or the 4th threshold value T4 also can suitably be set, make the self adaptation state machine of decode rate adjustment unit maintain a decode rate state comparatively approaching, thereby Video Decoder is maintained on the comparatively approaching decode rate with the requirement of balance of display buffer unit.
According to one embodiment of present invention, when using many technique modes (Trick Mode), the for example F.F. of the different rates of video, rewind down, slowly put, frame by frame, demands such as quick location, can adjust the speed of axle growth/minimizing lock in time equally, the final collaborative work of PTS lock unit, display buffer unit and the decode rate adjustment unit by adaptive decoding synchronous device is still united and is finished.For example from normally being shown to 2 times of F.F.s, the PTS lock unit judges according to the needs that show whether the PTS of present frame is expired, when expired, directly abandons the PTS of present frame, and takes the PTS of next frame.Like this, the consumption of display buffer unit has been speeded up, and Video Decoder is still decoded with original speed this moment and offer the display buffer unit.Because the display buffer unit produces the speed imbalance with consumption, the view data control information that the display buffer unit is deposited is fewer and feweri, until producing underflow.The display buffer unit sends the underflow condition signal to the decode rate adjustment unit when underflow condition occurs.Suppose that current decode rate is in the normal decoder pattern, then when the underflow number of times of the display buffer unit of accumulative total during greater than the first threshold T1 that sets, the decode rate adjustment unit will automatically be adjusted the decode rate of Video Decoder, make decode rate to higher rate transition between states, for example be adjusted into and skip the single frames decoding schema.When normal demonstration is got back in 2 times of F.F.s, the speed that the PTS lock unit will slow down and fetch data from the display buffer unit, and the decode rate of Video Decoder still remains on previous level at this moment, promptly skips the single frames decoding schema.Therefore, the view data control information that the display buffer unit is deposited is more and more, until producing overflow.The display buffer unit sends the overflow state signal to the decode rate adjustment unit when overflow state occurs.This moment, decode rate was skipped the single frames decoding schema, when the overflow number of times of the display buffer unit of accumulative total during greater than the 4th threshold value T4 that sets, the decode rate adjustment unit will automatically be adjusted the decode rate of Video Decoder, make decode rate to more low rate transition between states, get back to the normal decoder pattern.
According to one embodiment of present invention, when system need finish normal decoder demonstration, frame rate conversion and many technique modes different application such as (Trick Mode), can with this moment Video Decoder initial decoding state model be arranged on the different decode rate patterns so that make the speed of Video Decoder work can mate the needs of practical application apace.For example, can be set at 2 times of F.F.s (* 2 Forward) and skip the single frames decoding schema, 4 times of F.F.s (* 4 Forward) are set at the key frame decoding schema, and 16 times of F.F.s (* 16 Forward) are set at the single frames sequential decoding pattern etc. of skipping.
The present invention also correspondingly provides a kind of video decode method for synchronous, referring to Fig. 5, is the flow chart according to an adaptive decoding method for synchronous of the present invention, and this method comprises the steps:
Step S51 accelerates or slows down to the access speed of PTS according to the demand of image display speed.Step S52 influences the amount of storage of view data control information in the display buffer unit by accelerating or slowing down to the access speed of PTS.Step S53 judges whether the amount of storage of view data control information reaches the overflow state of display buffer unit.In this way, flow process enters step S54, sends overflow signals.Then, flow process judges further at step S55 whether the overflow signals of accumulative total reaches preset threshold value.In this way, flow process is reduced to lower video decode rate mode at step S56 with current video decode rate mode.Judge that at step S53 the amount of storage of view data control information does not reach the overflow state of display buffer unit as flow process, then flow process judges further in step 57 whether the amount of storage of view data control information reaches the underflow condition of display buffer unit.As not, flow process is returned step S53.In this way, flow process enters step S58, sends underflow signal.Then, flow process judges further at step S59 whether the underflow signal of accumulative total reaches preset threshold value.In this way, flow process enters step S50, and current video decode rate mode is brought up to higher video decode rate mode.
Fig. 6 is the flow chart according to another adaptive decoding method for synchronous of the present invention, also is a special case of method shown in Figure 5.Referring to Fig. 6, step S61 accelerates or slows down to the access speed of PTS according to the demand of display speed; Step S62, the quickening of the access speed of PTS or slow down and influence the amount of storage of view data control information in the display buffer unit; Step S63 when the amount of storage of view data control information reaches the overflow state of display buffer unit, sends overflow signals; When the amount of storage of view data control information reaches the underflow condition of display buffer unit, send underflow signal; Step S64, under the first decode rate pattern, when the underflow signal of accumulative total reaches presetting first threshold, video decode speed adds near the second decode rate pattern (that is, bring up to higher video decode rate mode, down with); When overflow signals occurring, provide corresponding event to the upper strata that drives.Under the first decode rate pattern, when when the upper strata that drives provides corresponding event, the adjustment that shows the decode rate of Video Decoder can not have been satisfied the limit that PTS requires synchronously, at this moment, for example utilize upper layer software (applications) that Video Decoder is handled, temporarily hinder Video Decoder and decode.Step S65, under the second decode rate pattern, when accumulative total underflow signal reach the 3rd default threshold value, video decode speed adds near the 3rd decode rate; When the overflow signals of accumulative total reaches the second default threshold value, video decode speed slows to the first decode rate pattern (that is, be reduced to lower video decode rate mode, down with).Step S66, under the 3rd decode rate pattern, when accumulative total underflow signal reach the 5th default threshold value, video decode speed adds near the 4th decode rate; When accumulative total overflow signals reach the 4th default threshold value, video decode speed slows to the second decode rate pattern.Step S67 under the 4th decode rate pattern, when the underflow signal reaches the 7th default threshold value, provides corresponding event to the upper strata that drives; When accumulative total overflow signals reach the 6th default threshold value, video decode speed slows to the 3rd decode rate pattern.When the upper strata that drives provides corresponding event, the adjustment that shows the decode rate of Video Decoder can not have been satisfied the limit that PTS requires synchronously, at this moment need the cooperation of other modules of system just can finish the synchronous requirement of PTS, for example Video Decoder is sought header information etc. according to the Random Access Info in the code stream.
Wherein the judgement of the demand of display speed is by the PTS acquisition of comparing with system clock (STC), as PTS during greater than system clock, need slow down to the access speed of PTS; As PTS during less than system clock, PTS is expired, accelerates the access speed to PTS.
Fig. 7 is the flow chart according to another adaptive decoding method for synchronous of the present invention, also is a special case of method shown in Figure 5.Referring to Fig. 7, step S71 accelerates or slows down to the access speed of PTS according to the demand of display speed; Step S72, the quickening of the access speed of PTS or slow down and influence the amount of storage of view data control information in the display buffer unit; Step S73 when the amount of storage of view data control information reaches the overflow state of display buffer unit, sends overflow signals; When the amount of storage of view data control information reaches the underflow condition of display buffer unit, send underflow signal; Step S74, under the second decode rate pattern, when accumulative total underflow signal reach the 3rd default threshold value, video decode speed adds near the 3rd decode rate; When accumulative total overflow signals reach the second default threshold value, video decode speed slows to the first decode rate pattern.
Fig. 8 is the flow chart according to another adaptive decoding method for synchronous of the present invention, also is a special case of method shown in Figure 5.Referring to Fig. 8, step S81 accelerates or slows down to the access speed of PTS according to the demand of display speed; Step S82, the quickening of the access speed of PTS or slow down and influence the amount of storage of view data control information in the display buffer unit; Step S83 when the amount of storage of view data control information reaches the overflow state of display buffer unit, sends overflow signals; When the amount of storage of view data control information reaches the underflow condition of display buffer unit, send underflow signal; Step S84 under the 4th decode rate pattern, when the underflow signal reaches the 7th default threshold value, provides corresponding event to the upper strata that drives; When accumulative total overflow signals reach the 6th default threshold value, video decode speed slows to the 3rd decode rate pattern.When the upper strata that drives provides corresponding event, show Video Decoder decode rate adjustment can not satisfy the limit that PTS requires synchronously, at this moment need the cooperation of other modules of system just can finish the synchronous requirement of PTS, for example Video Decoder is sought header information etc. according to the Random Access Info in the code stream.
The present invention is not limited to the elaboration that embodiment does, and anyly all should be encompassed within the spirit and scope of claim of the present invention based on modification of the present invention and equivalent of the present invention.
Claims (12)
1. adaptive decoding synchronous device comprises:
The PTS lock unit, the PTS information that goes out in order to the receiver, video decoder decode, and PTS compared with system clock, export a comparative result;
The display buffer unit in order to the control information of view data to be shown such as storage, and leaves the output of described control information wherein in according to the comparative result control of described PTS lock unit;
The decode rate adjustment unit, in order to the decode rate of adjusting Video Decoder according to the overflow state or the underflow condition of display buffer unit, it comprises overflow counter and the underflow counter in order to the overflow state and the underflow condition of display buffer unit are counted respectively, and the self adaptation state machine that multiple decode rate state model is provided in order to the decode rate according to Video Decoder.
2. video decode and display system comprise
Video Decoder;
Frame buffer zone is in order to deposit vedio data;
The PTS lock unit, the PTS information that goes out in order to the receiver, video decoder decode, and PTS compared with system clock, export a comparative result;
The display buffer unit in order to the control information of view data to be shown such as storage, and leaves the output of described control information wherein in according to the comparative result control of described PTS lock unit;
The decode rate adjustment unit, in order to the decode rate of adjusting Video Decoder according to the overflow state or the underflow condition of display buffer unit, it comprises overflow counter and the underflow counter in order to the overflow state and the underflow condition of display buffer unit are counted respectively, and the self adaptation state machine that multiple decode rate state model is provided in order to the decode rate according to Video Decoder; And
Reprocessing and display unit are in order to further to handle the synchronized images data and to show.
3. system as claimed in claim 2 is characterized in that described reprocessing directly links to each other with described PTS lock unit with display unit.
4. adaptive decoding method for synchronous may further comprise the steps:
A. accelerate according to the demand of image display speed or slow down the access speed of PTS;
B. by accelerating or slow down the access speed of PTS to be influenced the amount of storage of view data control information in the display buffer unit;
C. when the amount of storage of view data control information reaches the overflow state of display buffer unit, send overflow signals; When the amount of storage of view data control information reaches the underflow condition of display buffer unit, send underflow signal;
D. when the underflow signal of accumulative total reaches preset threshold value, current video decode rate mode is brought up to higher video decode rate mode;
E. when the overflow signals of accumulative total reaches preset threshold value, current video decode rate mode is reduced to lower video decode rate mode;
F. provide multiple decode rate state model according to video decode speed by the self adaptation state machine.
5. method as claimed in claim 4 is characterized in that, when current video decode rate mode is the highest video decode rate mode or minimum video decode rate mode, provides corresponding event to the upper strata that drives.
6. method as claimed in claim 4, it is characterized in that, current video decode rate mode is the first video decode rate mode, when the underflow signal of accumulative total reaches presetting first threshold, current video decode rate mode is brought up to the second video decode rate mode; When overflow signals occurring, provide corresponding event to the upper strata that drives.
7. as claim 4 or 6 described methods, it is characterized in that, current video decode rate mode is the second video decode rate mode, when the underflow signal of accumulative total reaches default the 3rd threshold value, current video decode rate mode is brought up to the 3rd video decode rate mode; When the overflow signals of accumulative total reaches default second threshold value, current video decode rate mode is reduced to the first video decode pattern.
8. method as claimed in claim 7, it is characterized in that, current video decode rate mode is the 3rd video decode rate mode, when the underflow signal of accumulative total reaches default the 5th threshold value, current video decode rate mode is brought up to the 4th video decode rate mode; When the overflow signals of accumulative total reaches default the 4th threshold value, current video decode rate mode is reduced to the second video decode pattern.
9. as claim 4 or 8 described methods, it is characterized in that current video decode rate mode is the 4th video decode rate mode, when the underflow signal of accumulative total reaches default the 7th threshold value, provide corresponding event to the upper strata that drives; When the overflow signals of accumulative total reaches default the 6th threshold value, current video decode rate mode is reduced to the 3rd video decode pattern.
10. method as claimed in claim 4 is characterized in that, among the step a to the judgement of the demand of image display speed by the PTS acquisition of comparing with system clock, when PTS slows down to the access speed of PTS during greater than system clock; When PTS accelerates access speed to PTS during less than system clock.
11. method as claimed in claim 8 is characterized in that the first video decode rate mode is the normal decoder pattern, the second video decode rate mode is for skipping the single frames decoding schema, and the 3rd video decode rate mode is the key frame decoding schema.
12. method as claimed in claim 9 is characterized in that the 4th video decode rate mode is a skipped frame sequential decoding pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101726059A CN101466045B (en) | 2007-12-20 | 2007-12-20 | Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101726059A CN101466045B (en) | 2007-12-20 | 2007-12-20 | Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101466045A CN101466045A (en) | 2009-06-24 |
| CN101466045B true CN101466045B (en) | 2011-03-16 |
Family
ID=40806356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007101726059A Expired - Fee Related CN101466045B (en) | 2007-12-20 | 2007-12-20 | Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101466045B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106028066A (en) * | 2015-03-24 | 2016-10-12 | 英特尔公司 | Distributed media stream synchronization control |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101951517B (en) * | 2010-09-27 | 2014-02-26 | 深圳市融创天下科技股份有限公司 | Method, system and terminal equipment for decoding and playing video |
| CN102547449A (en) * | 2011-12-22 | 2012-07-04 | 华为技术有限公司 | Method, set-top box and media server of control terminal buffer media stream data |
| CN102737681B (en) * | 2012-05-02 | 2015-02-04 | 中国联合网络通信集团有限公司 | Performance testing method and device of stream media player |
| CN102752571A (en) * | 2012-05-30 | 2012-10-24 | 曙光信息产业股份有限公司 | High-definition video device and implementation method thereof |
| CN102932640B (en) * | 2012-11-08 | 2014-12-10 | 天津市亚安科技股份有限公司 | Method for controlling video stream frame rate |
| CN103686311B (en) * | 2013-11-22 | 2017-02-15 | 乐视致新电子科技（天津）有限公司 | Audio-video playing method and audio-video playing device |
| CN104822008B (en) * | 2014-04-25 | 2019-01-08 | 腾讯科技（北京）有限公司 | video synchronization method and device |
| CN105704506A (en) * | 2016-01-19 | 2016-06-22 | 北京流金岁月文化传播股份有限公司 | Device and method for synchronizing audio and video coding labial sound |
| CN106792154B (en) * | 2016-12-02 | 2020-02-11 | 广东赛特斯信息科技有限公司 | Frame skipping synchronization system of video player and control method thereof |
| CN109194985B (en) * | 2018-11-19 | 2021-03-26 | 上海高骏精视信息技术有限公司 | Audio and video Ethernet transmission method and system |
| CN111601427A (en) * | 2020-06-29 | 2020-08-28 | 深圳匠明科技有限公司 | LED driver chip and LED driver system |
| CN111586937A (en) * | 2020-06-29 | 2020-08-25 | 深圳匠明科技有限公司 | LED driving chip and cascade driving circuit |
| CN112073751B (en) * | 2020-09-21 | 2023-03-28 | 苏州科达科技股份有限公司 | Video playing method, device, equipment and readable storage medium |
| CN113301332B (en) * | 2021-04-12 | 2024-06-21 | 阿里巴巴创新公司 | Video decoding method, system and medium |
| JP2024538799A (en) * | 2021-10-15 | 2024-10-23 | シェンチェン ティーシーエル ニュー テクノロジー カンパニー リミテッド | Data processing method, device, computer device and storage medium |
| CN114339406B (en) * | 2022-01-27 | 2023-01-24 | 重庆紫光华山智安科技有限公司 | Method, system, device and medium for adjusting decoding speed |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5812201A (en) * | 1995-04-11 | 1998-09-22 | Lg Semicon Co., Ltd. | Data synchronizing apparatus and method thereof |
| CN1436001A (en) * | 2002-01-28 | 2003-08-13 | 北京华诺信息技术有限公司 | Method for synchronizing video with audio in decoding system |
| CN1491516A (en) * | 2001-10-18 | 2004-04-21 | ���µ�����ҵ��ʽ���� | Audiovisual reproduction device, audiovisual reproduction method, program and medium |
-
2007
- 2007-12-20 CN CN2007101726059A patent/CN101466045B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5812201A (en) * | 1995-04-11 | 1998-09-22 | Lg Semicon Co., Ltd. | Data synchronizing apparatus and method thereof |
| CN1491516A (en) * | 2001-10-18 | 2004-04-21 | ���µ�����ҵ��ʽ���� | Audiovisual reproduction device, audiovisual reproduction method, program and medium |
| CN1436001A (en) * | 2002-01-28 | 2003-08-13 | 北京华诺信息技术有限公司 | Method for synchronizing video with audio in decoding system |
Non-Patent Citations (1)
| Title |
|---|
| JP特开平8-307862A 1996.11.22 |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106028066A (en) * | 2015-03-24 | 2016-10-12 | 英特尔公司 | Distributed media stream synchronization control |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101466045A (en) | 2009-06-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101466045B (en) | Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system | |
| JP3836143B2 (en) | Method and apparatus for decoding a plurality of digital video bitstreams and receiving apparatus comprising such an apparatus | |
| US6469744B1 (en) | Methods and apparatus for encoding, decoding and displaying images in a manner that produces smooth motion | |
| CN1941913B (en) | Method and apparatus for skipping pictures | |
| AU756355B2 (en) | Video encoder and encoding method with buffer control | |
| EP1968327A2 (en) | MPEG video decoder and MPEG video decoding method | |
| EP0431319A2 (en) | Video signal coding apparatus, coding method and video signal coding transmission system | |
| CN105933800A (en) | Video play method and control terminal | |
| US7460599B2 (en) | Video decoding device and method, and program product therefor | |
| CN103237255A (en) | Multi-thread audio and video synchronization control method and system | |
| CN101010959A (en) | Trickmodes and speed transitions | |
| US20020049969A1 (en) | Method for playing multimedia applications | |
| US20040264577A1 (en) | Apparatus and method for controlling the synchronization of a video transport stream | |
| CN103458271A (en) | Audio-video file splicing method and audio-video file splicing device | |
| JPH04504040A (en) | High Definition TV Encoder/Decoder | |
| WO2006075844A1 (en) | Reference frame ordering for multi-view coding | |
| CN101316369B (en) | Image processing device and method | |
| JP3468115B2 (en) | Data switching device | |
| KR19990068138A (en) | Digital signal multiplexing method and apparatus, digital signal transmission method and apparatus, digital signal recording method and apparatus and recording medium | |
| US8837599B2 (en) | System, method and apparatus for clean channel change | |
| US10694240B2 (en) | Method for decoding an audio/video stream and corresponding device | |
| KR20040065170A (en) | Video information decoding apparatus and method | |
| US7050497B2 (en) | Process and device for the video coding of high definition images | |
| EP0735782A2 (en) | Motion picture play apparatus capable of returning from trick play to normal play | |
| US6628838B1 (en) | Picture decoding apparatus, picture decoding method and recording medium for storing the picture decoding method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110316 Termination date: 20131220 |