CN106686445A - Method of carrying out on-demand jump on multimedia file - Google Patents
Method of carrying out on-demand jump on multimedia file Download PDFInfo
- Publication number
- CN106686445A CN106686445A CN201510745515.9A CN201510745515A CN106686445A CN 106686445 A CN106686445 A CN 106686445A CN 201510745515 A CN201510745515 A CN 201510745515A CN 106686445 A CN106686445 A CN 106686445A
- Authority
- CN
- China
- Prior art keywords
- frame
- video
- audio
- target
- timestamp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000005538 encapsulation Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4343—Extraction or processing of packetized elementary streams [PES]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention provides a method of carrying out on-demand jump on a multimedia file. The method comprises steps: an inputted multimedia file is de-multiplexed through ffmpeg, an ES (Elementary Stream) of the multimedia file is obtained, and video and audio information of the multimedia file is acquired; the video and audio information is used for initializing an Intel decoder; according to jump accuracy demand information and target frame information, a time stamp of the target frame in the ES is determined; stream operation is carried out on the ES through the ffmpeg, and video and audio frame data are read starting from the position of a left-side key frame of the target frame; the initialized Intel decoder is used for decoding the read video and audio frame data, and the time stamp of the video and audio frame is obtained; and whether the time stamp of the video and audio frame is larger than or equal to the time stamp of the target frame is judged, and the video and audio frame data are put to a video and audio output queue for being outputted. The ffmpeg can be used for carrying out on-demand jump based on the Intel decoder.
Description
Technical field
The present invention relates to a kind of method that multimedia file is redirected on demand, more particularly to a kind of comprehensive utilization English spy
Your Multimedia Software Development bag (intel media sdk) realizes that video and audio is synchronously redirected with ffmpeg software development kits
(seek) method.
Background technology
Have many multimedia video audio processing software kits on the market at present, these kits respectively have the advantage of oneself with
Inferior position, for example, ffmpeg software development kits, it can provide including video and audio collection, video and audio form conversion,
The functions such as video capture, video with watermark, have the advantages that powerful, broad covered area.But ffmpeg is in hardware
Encoding and decoding acceleration aspect but shorter mention, still very imperfection, far away from the superior performance of intel media sdk.intel
Media sdk provide remarkable performance in hardware-accelerated decoding, encoding context, and it can make full use of graphic process unit
(GPU) it is hardware-accelerated to carry out.Through test, for same section of material, compared with ffmpeg sdk, using intel
Media sdk carry out transcoding, and speed can improve 30 or so percent and its CPU takes efficiency and substantially reduces.It is identical
Under running environment, surveyed with the performance decoded to same story using intel media sdk using ffmpeg sdk
Test result is as follows：
Obviously intel media sdk have a clear superiority at hardware-accelerated aspect.However, intel media sdk but can not
Seek operations are carried out based on multimedia file stream, can only be provided and (Elementary Stream, basic code are flowed based on ES
Stream) hardware-accelerated order decoding operate.
In broadcasting and TV field, multimedia file is being carried out to decode in playing process or nonlinear editing system, Ren Menjing
To often play or preview location drags to another time point from a time point of file.To check that it is interior that oneself is concerned about
Hold.This process is known as seek.This operation is relatively conventional, if can not be to file stream seek on demand, even if tool
There is superior function of hardware acceleration, also can be very limited in the application of product.
The content of the invention
The present invention provides a kind of method redirected on demand to multimedia file, with solve one of the prior art or
Multinomial disappearance.
The embodiment of the present invention provides a kind of method redirected on demand to multimedia file, including：Receiving user's input
Multimedia file, the multimedia file is demultiplexed by ffmpeg, obtain the multimedia file
ES flows, and obtains the audiovisual information of the multimedia file；Using the audiovisual information to Intel's decoder
Initialized；According to the information for redirecting accuracy requirement information and a target frame, target in the ES streams is determined
The timestamp of frame；Stream operation is carried out to the ES streams by the ffmpeg, and is closed from a left side of the target frame
The position of key frame starts to read video and audio frame data；Intel's decoder after using initialization is to described in reading
Video and audio frame data are decoded, and obtain the timestamp of the video and audio frame；Judge that the timestamp of the video and audio frame is big
In or equal to the target frame timestamp when, the video and audio frame data are put into into video and audio output queue, to carry out
Output.
In one embodiment, according to the information for redirecting accuracy requirement information and a target frame, the ES streams are determined
The timestamp of middle target frame, including：According to it is described redirect accuracy requirement information setting described in ES stream in frame occur
The threshold value for redirecting；Judge current decoding output frame in the target frame and ES stream frame number difference whether be more than or
Equal to the threshold value；If it is, obtaining the time of the target frame according to the position calculation of the current decoding output frame
Stamp.
In one embodiment, also include：The timestamp of the first frame frame of video in ES streams is to the target
The timestamp of the frame of video in frame is corrected, and the timestamp of the first frame audio frame in ES streams is to described
The timestamp of the audio frame in target frame is corrected.
In one embodiment, also include：When judging that the timestamp of the video and audio frame is less than the timestamp of the target frame,
Video and audio frame data of the timestamp less than the timestamp of the target frame are abandoned, and continues to read the video and audio frame of next frame
Data, until the timestamp of the video and audio frame for reading is more than or equal to the timestamp of the target frame.
In one embodiment, when the video-voice frequency flow data are video data, Intel's decoder is video solution
Code device；When the video-voice frequency flow data are voice data, Intel's decoder is audio decoder.
In one embodiment, when the video-voice frequency flow data are video data, the timestamp of the target frame is：
Vt=(the times of the inverse/video flowing of (target video frame frame number * (1.0/ video frame rate) * fiducial times) * fiducial times
Base).
In one embodiment, when the video-voice frequency flow data are voice data, the timestamp of the target frame is：
At=((target audio frame frame number * (the hits * fiducial times/target audio sample rate of the frame audio frequency of 1.0* mono-) * fiducial times
Inverse/audio stream time base), wherein, the hits=target audio sample rate/video frame rate of a frame audio frequency,
The hits of target audio frame frame number=target audio sample index value/mono- frame audio frequency, target audio sample index value can lead to
Cross and obtained in the incoming interface parameters of interface interchange person when seek operations occur.
In one embodiment, the video and audio frame data for reading are put into into video and audio output queue, to be exported,
Including：Video data in the video and audio frame data of the timestamp more than or equal to the reading of the target frame is put into
Video frequency output queue, to carry out video frequency output；Sound is regarded described in the reading that timestamp is more than or equal to the target frame
Voice data in frequency frame data is put into audio output queue, to carry out audio output；Judge the video frequency output queue
During with the audio output queue containing the video and audio frame data for being exported, the ffmpeg stops reading regarding sound
Frequency frame data.
In one embodiment, the audiovisual information of the multimedia file includes：Multimedia file total length, multimedia
Document encapsulation form, Video coding mode, video frame rate, video flowing length, vedio color form, video code rate,
One or more in audio coding formats, audio sample rate, audio sample form and audio stream length.
In one embodiment, the left side key frame is positioned at target frame left side and closest to the pass of the target frame
Key frame.
The embodiment of the present invention, both can make full use of Intel's decoder (intel media sdk) to carry out hardware and add
Speed decoding, can allow user to be redirected (seek) on demand to the multimedia file for being decoded broadcasting again, can be very
Realize well based on Intel decoder intel media sdk multimedia file hardware-accelerated decoding play with demand
Redirect seek.The present invention can comprehensively utilize intel media sdk with ffmpeg sdk to realize the synchronous seek of video and audio
Method.Simultaneously as the matching to target video frame, target audio frame is adopted closest to principle (leftmost side key
Frame), thus for the Multi-media Material for there are some existence times to stab drift phenomenon, equally can carry out accurately
Seek is operated.
Description of the drawings
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only
It is some embodiments of the present invention, for those of ordinary skill in the art, in the premise for not paying creative work
Under, can be with according to these other accompanying drawings of accompanying drawings acquisition.In the accompanying drawings：
Fig. 1 is the schematic flow sheet of the method redirected on demand to multimedia file of the embodiment of the present invention；
Fig. 2 is the schematic flow sheet of the acquisition methods of the target frame timestamp in one embodiment of the invention；
Fig. 3 is the schematic flow sheet of the acquisition methods of the target frame timestamp of another embodiment of the present invention；
Fig. 4 is the flow process of the method that video and audio frame data are put into video and audio output queue in one embodiment of the invention
Schematic diagram；
Fig. 5 is the schematic flow sheet of the method that one embodiment of the invention is redirected on demand to multimedia file.
Specific embodiment
Purpose, technical scheme and advantage to make the embodiment of the present invention becomes more apparent, below in conjunction with the accompanying drawings to this
Bright embodiment is described in further details.Here, the schematic description and description of the present invention is used to explain the present invention,
But it is not as a limitation of the invention.
The embodiment of the present invention provides a kind of method redirected on demand to multimedia file, and the method can both utilize English
Te Er decoders (intel media sdk) carry out hardware-accelerated decoding, and the multimedia file that be decoded broadcasting can be entered again
Row is redirected on demand (seek).
Fig. 1 is the schematic flow sheet of the method redirected on demand to multimedia file of the embodiment of the present invention.Such as Fig. 1
Method shown, that multimedia file is redirected on demand, it may include step：
S110：The multimedia file of receiving user's input, is demultiplexed by ffmpeg to the multimedia file,
The ES streams of the multimedia file are obtained, and obtains the audiovisual information of the multimedia file；
S120：Intel's decoder is initialized using the audiovisual information；
S130：According to the information for redirecting accuracy requirement information and a target frame, target frame in the ES streams is determined
Timestamp；
S140：Stream operation is carried out to the ES streams by the ffmpeg, and it is crucial from a left side of the target frame
The position of frame starts to read video and audio frame data；
S150：Intel's decoder after using initialization is decoded to the video and audio frame data for reading,
Obtain the timestamp of the video and audio frame；
S160：When judging that the timestamp of the video and audio frame is more than or equal to the timestamp of the target frame, regard described
Audio frame number evidence is put into video and audio output queue, to be exported.
In above-mentioned steps S120, the audiovisual information of the multimedia file may include much information.For example, it is above-mentioned
The audiovisual information of multimedia file may include multimedia file total length, multimedia file encapsulation format, Video coding
Mode, video frame rate, video flowing length, vedio color form, video code rate, audio coding formats, audio sample
Rate, audio sample form and audio stream length can be in information one or more.
The embodiment of the present invention is solved by above-mentioned steps S120, the audiovisual information obtained using step S110 to Intel
Code device is initialized, and can get out hardware-accelerated decoding context.
In the embodiment of the present invention, such as in above-mentioned S120 and S150, for different multi-medium datas, can be using not
Same decoder.For example, when the video-voice frequency flow data for reading are video data, above-mentioned Intel's decoder is video
Decoder；When the video-voice frequency flow data for reading are voice data, above-mentioned Intel's decoder is audio decoder.Together
Sample ground, is initialized using video information to Video Decoder, and audio decoder is carried out initially using audio-frequency information
Change.
In above-mentioned steps S140, the groups of frame that many is made up of multiple frames is may include on the left of the target frame, often
First frame of the leftmost side of one framing can be key frame, then can start to read from the different key frame of each on the left of target frame
Take video/audio.It is preferred that the left side key frame in step S140 is positioned at the target frame left side and closest to institute
The key frame of target frame is stated, with this, the stream operation task amount of ffmpeg can be reduced, so as to improve seek efficiency.And
And, for the Multi-media Material for there are some existence times to stab drift phenomenon, can equally carry out accurate seek behaviour
Make.
The embodiment of the present invention, for the hardware encoding of ffmpeg in prior art defect and Intel's decoder are accelerated
(intel media sdk) can not be redirected the defect of (seek) based on stream, by effectively utilizing ffmpeg in the lump
Carry out flowing operation and read video and audio frame data using Intel's decoder (intel media sdk), can both can be with profit
Hardware-accelerated decoding is carried out with Intel's decoder (intel media sdk), again can be literary to being decoded the multimedia of broadcasting
Part is redirected on demand (seek).
Fig. 2 is the schematic flow sheet of the acquisition methods of the target frame timestamp in one embodiment of the invention.As shown in Fig. 2
In above-mentioned steps S130, according to the information for redirecting accuracy requirement information and a target frame, the ES is determined
The method of the timestamp of target frame in stream, it may include step：
S131：According to it is described redirect accuracy requirement information setting described in the threshold value that redirects of frame in ES streams；
S132：Judge whether the target frame is more than or waits with the frame number difference of current decoding output frame in ES streams
In the threshold value；
S133：If it is, obtaining the timestamp of the target frame according to the position calculation of the current decoding output frame.
In above-mentioned steps S131, the threshold value that frame is redirected, e.g. minimum interval or minimum frame number difference
Interval.In above-mentioned steps S132, it is above-mentioned redirect accuracy requirement information can be redirect before (seek) video and audio
The frame number of frame and difference DELTA t of the frame number of the required target frame for redirecting (seek), for example, can use time difference or frame number poor
Represent.If the threshold value that frame is redirected is Δ m, when can work as Δ t >=Δ m, the timestamp of target frame is just calculated, to carry out
Seek is operated.
In the embodiment of the present invention, the threshold value that accuracy requirement information setting is redirected is redirected by basis, and according to threshold
Value judges whether to redirect, and can meet and various redirect accuracy requirement.By the position according to current decoding output frame
Putting can effectively calculate the timestamp of target frame.
Fig. 3 is the schematic flow sheet of the acquisition methods of the target frame timestamp of another embodiment of the present invention.As shown in figure 3,
The acquisition methods of target frame timestamp as shown in Figure 2, may also include step：
S134：According to the ES stream in the first frame frame of video timestamp to the frame of video in the target frame when
Between stab and be corrected, the timestamp of the first frame audio frame in ES streams is to the audio frame in the target frame
Timestamp be corrected.
In the preferred embodiment, it is corrected by the timestamp to target frame, more accurately target frame can be obtained
Timestamp.In above-mentioned steps S140～S160, video and audio can be read according to the target frame timestamp after correction
Frame data.
Again as shown in figure 1, the method redirected on demand to multimedia file of the embodiment of the present invention, may also include step
Suddenly：
S170：When judging that the timestamp of the video and audio frame is less than the timestamp of the target frame, abandon timestamp and be less than
The video and audio frame data of the timestamp of the target frame, and continue to read the video and audio frame data of next frame, until reading
Video and audio frame timestamp more than or equal to the target frame timestamp.
In the embodiment of the present invention, the timestamp of target frame is less than according to the timestamp of video and audio frame, it can be determined that read
Video and audio frame also in the left side of target frame, not yet reach redirect broadcasting position.Abandon timestamp less than target frame when
Between stab video and audio frame data, contribute to save decoder spatial cache.
Fig. 4 is the flow process of the method that video and audio frame data are put into video and audio output queue in one embodiment of the invention
Schematic diagram.As shown in figure 4, in above-mentioned steps S160, the video and audio frame data for reading are put into into video and audio
Output queue, to be exported, it may include step：
S161：By the video data in the video and audio frame data of the timestamp more than or equal to the reading of the target frame
Video frequency output queue is put into, to carry out video frequency output；
S162：By the voice data in the video and audio frame data of the timestamp more than or equal to the reading of the target frame
Audio output queue is put into, to carry out audio output；
S163：Judge the video frequency output queue and the audio output queue containing the video and audio frame number for being exported
According to when, the ffmpeg stops reading video and audio frame data.
In the embodiment of the present invention, in above-mentioned steps S161 and S162, being put into the video/audio of queue is used to wait
Output.In above-mentioned steps S163, during according to having output depending on, audio frequency, stop reading video and audio frame data, i.e.,
Terminate seek, can not only jump at target frame, and can make simultaneously user pass through sound or picture any one
Can determine to redirect and complete, and then the preferable audition of user and visual experience can be brought.
What deserves to be explained is, with regard to the description of " video and audio " in various embodiments of the present invention, such as video and audio frame, regarding sound
Frequency frame data, audiovisual information, can be at least one of Voice ＆ Video.Each step, operation or place
In reason, voice data can be optionally corresponding with audio operation, and video data can be optionally corresponding with vision operation,
For example voice data is decoded by audio decoder, and video data is decoded by video, target frame sound intermediate frequency
The timestamp of frame can be corrected by the timestamp of the audio frame in the first frame, the timestamp of the frame of video in target frame
Can be corrected by the timestamp of the frame of video in the first frame.
Fig. 5 is the schematic flow sheet of the method that one embodiment of the invention is redirected on demand to multimedia file.Such as Fig. 5
It is shown, first, de-multiplex operation is carried out to the multimedia file being input into using ffmpeg, ES streams are obtained, by solution
Multiplexing operation can get file total length, Document encapsulation form, Video coding mode, video frame rate, video flowing
Length, vedio color form, video code rate, audio coding formats, audio sample rate, audio sample form, audio frequency
The code-related informations (step S201) such as stream length.
Then, intel decoders are initialized using these code-related informations for getting, gets out hardware
Accelerate decoding context relevant information (step S202).Next, according to the demand that accuracy is operated to seek,
The threshold value for judging seek operation generations is set as Δ m, for example, setting Δ m=5 frames (step S203).Calculate current
Target frame and difference DELTA t of previous frame frame number, enter seek operation logics (step S204) if Δ t >=Δ m.According to
Target frame frame number calculates the timestamp of target video and audio frame, and the timestamp using first frame video and audio carries out school to the value
Just (step S205～S206).
Specifically, the timestamp vt of initial target frame of video can be calculated according to target frame frame number, first frame video is recycled
Timestamp vs is corrected to the timestamp vt of the initial target frame of video, obtains the timestamp VT of target video frame.
For example, the computing formula of the timestamp VT of target video frame can be：
VT=vt+vs, (1)
The computing formula that the timestamp vt of initial target frame of video is calculated according to target frame frame number can be various expression-forms
Or calculation, such as computing formula can be：
Vt=(inverse/video flowing of (target video frame frame number * (1.0/ video frame rate) * fiducial times) * fiducial times when
Between base), (2)
Wherein, target video frame frame number can be according to obtaining in the information of a target frame, video frame rate, fiducial time
The time base of AV_TIME_BASE and video flowing, can pass through the video and audio letter of the multimedia file that demultiplexing is obtained
Breath is obtained.The occurrence of fiducial time AV_TIME_BASE can be selected optionally, for example,
AV_TIME_BASE=1000000.
Similarly, the timestamp at of initial target audio frame can be calculated according to target frame frame number, first frame audio frequency is recycled
Timestamp as is corrected to the timestamp at of the initial target audio frame, obtains the timestamp AT of target audio frame.
For example, the computing formula of the timestamp AT of target audio frame can be：
AT=at+as. (3)
The computing formula that the timestamp at of initial target audio frame is calculated according to target frame frame number can be various expression-forms
Or calculation, such as computing formula can be：
At=((target audio frame frame number * (the hits * fiducial times/target audio sample rate of the frame audio frequency of 1.0* mono-) * benchmark
The time base of the inverse/audio stream of time) (4)
Wherein, formula (4) can meet：The hits of one frame audio frequency=target audio sample rate/video frame rate, target
The hits of audio frame frame number=target audio sample index value/mono- frame audio frequency.Target audio sample index value can pass through
Obtain in the incoming interface parameters of interface interchange person when seek operations occur.Target audio sample rate is referred to needed for caller
The audio sample rate wanted, target audio sample rate can be obtained according in the target decoder argument structure body of interface interchange person setting
Arrive.The occurrence of fiducial time AV_TIME_BASE can be selected optionally, for example, AV_TIME_BASE=
1000000。
In above-described embodiment, by step S205～S206, for example, by formula (1)～(2), standard is obtained
The true target video timestamp VT based on video flowing time base；Similarly, by formula (3)～(4), can
Obtain the timestamp AT of accurate target audio frame.
It is available after the timestamp AT of the timestamp VT and target audio frame that obtain accurate target video frame
Ffmpeg carries out stream operation.It is preferred that it (can be target video frame or target first to jump to distance objective video and audio frame
Audio frame) nearest left side key frame position, and can notify that intel decoders empty its inner buffer, so as to follow-up
Read data (step S207).
Using ffmpeg by the nearest left side key frame position, audio frequency and video are read from the file stream (ES streams)
Frame data (step S208).If read for frame of video, send into the decoding of intel Video Decoders, judge that this is regarded
Whether the timestamp of frequency frame is less than VT, if so, then directly abandons, and otherwise, is put into video frequency output queue etc. to be output
(step S209～S211).If read for audio frame, send into the decoding of intel audio decoders, and judge output
Audio frame whether be less than AT, if so, then directly abandon, otherwise, be put into audio output queue, wait (step to be output
Rapid S212～S214).When frame of video and audio frame have output, seek terminates.
The embodiment of the present invention, both can make full use of Intel's decoder (intel media sdk) to carry out hardware and add
Speed decoding, can allow user to be redirected (seek) on demand to the multimedia file for being decoded broadcasting again, can be very
Realize well based on Intel decoder intel media sdk multimedia file hardware-accelerated decoding play with demand
Redirect seek.The present invention can comprehensively utilize intel media sdk with ffmpeg sdk to realize the synchronous seek of video and audio
Method.Simultaneously as the matching to target video frame, target audio frame is adopted closest to principle (leftmost side key
Frame), thus for the Multi-media Material for there are some existence times to stab drift phenomenon, equally can carry out accurately
Seek is operated.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, system or computer journey
Sequence product.Therefore, the present invention can adopt complete hardware embodiment, complete software embodiment or combine software and hardware
The form of the embodiment of aspect.And, the present invention can be adopted and wherein include computer available programs at one or more
The computer-usable storage medium (including but not limited to disk memory, CD-ROM, optical memory etc.) of code
The form of the computer program of upper enforcement.
The present invention is the stream with reference to method according to embodiments of the present invention, equipment (system) and computer program
Journey figure and/or block diagram are describing.It should be understood that can be by computer program instructions flowchart and/or block diagram
The combination of each flow process and/or square frame and flow chart and/or the flow process in block diagram and/or square frame.These can be provided
Computer program instructions are processed to general purpose computer, special-purpose computer, Embedded Processor or other programmable datas and set
Standby processor is producing a machine so that held by the processor of computer or other programmable data processing devices
Capable instruction is produced for realizing in one flow process of flow chart or one square frame of multiple flow processs and/or block diagram or multiple sides
The device of the function of specifying in frame.
These computer program instructions may be alternatively stored in can guide computer or other programmable data processing devices with spy
In determining the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory produces bag
The manufacture of command device is included, the command device is realized in one flow process of flow chart or multiple flow processs and/or block diagram one
The function of specifying in individual square frame or multiple square frames.
These computer program instructions also can be loaded in computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented process, so as to calculate
The instruction performed on machine or other programmable devices provide for realizing in one flow process of flow chart or multiple flow processs and/or
The step of function of specifying in one square frame of block diagram or multiple square frames.
Particular embodiments described above, has been carried out further in detail to the purpose of the present invention, technical scheme and beneficial effect
Describe in detail it is bright, should be understood that the foregoing is only the present invention specific embodiment, be not used to limit this
Bright protection domain, all any modification, equivalent substitution and improvements within the spirit and principles in the present invention, done etc.,
Should be included within the scope of the present invention.
Claims (10)
1. a kind of method that multimedia file is redirected on demand, it is characterised in that include：
The multimedia file of receiving user's input, is demultiplexed by ffmpeg to the multimedia file, is obtained
The ES streams of the multimedia file, and obtain the audiovisual information of the multimedia file；
Intel's decoder is initialized using the audiovisual information；
According to the information for redirecting accuracy requirement information and a target frame, the time of target frame in the ES streams is determined
Stamp；
Stream operation, and the position of the left side key frame from the target frame are carried out to the ES streams by the ffmpeg
Put and start to read video and audio frame data；
Intel's decoder after using initialization is decoded to the video and audio frame data for reading, and obtains institute
State the timestamp of video and audio frame；
When judging that the timestamp of the video and audio frame is more than or equal to the timestamp of the target frame, by the video and audio frame
Data are put into video and audio output queue, to be exported.
2. the method for multimedia file being redirected on demand as claimed in claim 1, it is characterised in that according to
One information for redirecting accuracy requirement information and a target frame, determines the timestamp of target frame in the ES streams, including：
According to it is described redirect accuracy requirement information setting described in the threshold value that redirects of frame in ES streams；
Judge the target frame with the frame number difference of current decoding output frame in ES streams whether more than or equal to described
Threshold value；
If it is, obtaining the timestamp of the target frame according to the position calculation of the current decoding output frame.
3. the method for multimedia file being redirected on demand as claimed in claim 2, it is characterised in that also wrap
Include：The timestamp of the first frame frame of video in ES streams enters to the timestamp of the frame of video in the target frame
Row correction, the time of the timestamp to the audio frame in the target frame of the first frame audio frame in ES streams
Stamp is corrected.
4. the method for multimedia file being redirected on demand as claimed in claim 1, it is characterised in that also wrap
Include：When judging that the timestamp of the video and audio frame is less than the timestamp of the target frame, abandon timestamp and be less than the mesh
The video and audio frame data of the timestamp of mark frame, and continue to read the video and audio frame data of next frame, until reading regarding sound
Timestamp of the timestamp of frequency frame more than or equal to the target frame.
5. the method for multimedia file being redirected on demand as claimed in claim 1, it is characterised in that
When the video-voice frequency flow data are video data, Intel's decoder is Video Decoder；
When the video-voice frequency flow data are voice data, Intel's decoder is audio decoder.
6. the method for multimedia file being redirected on demand as claimed in claim 3, it is characterised in that work as institute
State video-voice frequency flow data for video data when, the timestamp of the target frame is：
Vt=(inverse/video flowing of (target video frame frame number * (1.0/ video frame rate) * fiducial times) * fiducial times when
Between base).
7. the method for multimedia file being redirected on demand as claimed in claim 6, it is characterised in that work as institute
State video-voice frequency flow data for voice data when, the timestamp of the target frame is：
At=((target audio frame frame number * (the hits * fiducial times/target audio sample rate of the frame audio frequency of 1.0* mono-) * benchmark
The time base of the inverse/audio stream of time), wherein, the hits=target audio sample rate/frame of video of a frame audio frequency
Rate, the hits of target audio frame frame number=target audio sample index value/mono- frame audio frequency.
8. the method for multimedia file being redirected on demand as claimed in claim 5, it is characterised in that will read
The video and audio frame data for taking are put into video and audio output queue, to be exported, including：
Video data in the video and audio frame data of the timestamp more than or equal to the reading of the target frame is put into
Video frequency output queue, to carry out video frequency output；
Voice data in the video and audio frame data of the timestamp more than or equal to the reading of the target frame is put into
Audio output queue, to carry out audio output；
When judging the video frequency output queue and the audio output queue containing the video and audio frame data for being exported,
The ffmpeg stops reading video and audio frame data.
9. the method for multimedia file being redirected on demand as claimed in claim 1, it is characterised in that described
The audiovisual information of multimedia file includes：Multimedia file total length, multimedia file encapsulation format, Video coding
Mode, video frame rate, video flowing length, vedio color form, video code rate, audio coding formats, audio sample
One or more in rate, audio sample form and audio stream length.
10. the method for multimedia file being redirected on demand as claimed in claim 7, it is characterised in that described
Left side key frame is positioned at target frame left side and closest to the key frame of the target frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510745515.9A CN106686445B (en) | 2015-11-05 | 2015-11-05 | The method that multimedia file is jumped on demand |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510745515.9A CN106686445B (en) | 2015-11-05 | 2015-11-05 | The method that multimedia file is jumped on demand |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106686445A true CN106686445A (en) | 2017-05-17 |
| CN106686445B CN106686445B (en) | 2019-06-11 |
Family
ID=58857766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510745515.9A Active CN106686445B (en) | 2015-11-05 | 2015-11-05 | The method that multimedia file is jumped on demand |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106686445B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107566890A (en) * | 2017-09-15 | 2018-01-09 | 深圳国微技术有限公司 | Handle audio stream broadcasting abnormal method, apparatus, computer installation and computer-readable recording medium |
| CN107566889A (en) * | 2017-09-15 | 2018-01-09 | 深圳国微技术有限公司 | Audio stream flow rate error processing method, device, computer installation and computer-readable recording medium |
| CN108401188A (en) * | 2018-03-05 | 2018-08-14 | 青岛海信传媒网络技术有限公司 | A kind of method and device of media play |
| CN108737874A (en) * | 2018-06-05 | 2018-11-02 | 武汉斗鱼网络科技有限公司 | A kind of video broadcasting method and electronic equipment |
| CN110418186A (en) * | 2019-02-01 | 2019-11-05 | 腾讯科技（深圳）有限公司 | Audio and video playing method, apparatus, computer equipment and storage medium |
| CN111918121A (en) * | 2020-06-23 | 2020-11-10 | 南斗六星系统集成有限公司 | Method and device for accurately editing streaming media file |
| CN113141522A (en) * | 2020-01-17 | 2021-07-20 | 北京达佳互联信息技术有限公司 | Resource transmission method, device, computer equipment and storage medium |
| CN113726778A (en) * | 2021-08-30 | 2021-11-30 | 咪咕视讯科技有限公司 | Streaming media seek method, device, computing equipment and computer storage medium |
| US20210409473A1 (en) * | 2017-07-20 | 2021-12-30 | Disney Enterprises, Inc. | Frame-accurate video seeking via web browsers |
| CN113965786A (en) * | 2021-09-29 | 2022-01-21 | 杭州当虹科技股份有限公司 | Method for accurately controlling video output and playing |
| CN114245231A (en) * | 2021-12-21 | 2022-03-25 | 威创集团股份有限公司 | Multi-video synchronous skipping method, device, equipment and readable storage medium |
| CN114466238A (en) * | 2020-11-09 | 2022-05-10 | 华为技术有限公司 | Frame demultiplexing method, electronic device and storage medium |
| CN116230029A (en) * | 2023-02-28 | 2023-06-06 | 深圳市中科蓝讯科技股份有限公司 | Audio file playback method and device, audio equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150729A (en) * | 2006-09-22 | 2008-03-26 | 康佳集团股份有限公司 | Audio and video synchronization playing method |
| CN102368835A (en) * | 2011-06-28 | 2012-03-07 | 上海盈方微电子有限公司 | Audio and video Seek synchronization strategy |
| CN102404624A (en) * | 2011-11-07 | 2012-04-04 | 深圳市佳创视讯技术股份有限公司 | Full-format media player supporting hardware decoding for digital set-top box |
| CN104754349A (en) * | 2013-12-25 | 2015-07-01 | 炫一下（北京）科技有限公司 | Method and device for hardware decoding of audio/video |
| CN104766608A (en) * | 2014-01-07 | 2015-07-08 | 深圳市中兴微电子技术有限公司 | Voice control method and voice control device |
-
2015
- 2015-11-05 CN CN201510745515.9A patent/CN106686445B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150729A (en) * | 2006-09-22 | 2008-03-26 | 康佳集团股份有限公司 | Audio and video synchronization playing method |
| CN102368835A (en) * | 2011-06-28 | 2012-03-07 | 上海盈方微电子有限公司 | Audio and video Seek synchronization strategy |
| CN102404624A (en) * | 2011-11-07 | 2012-04-04 | 深圳市佳创视讯技术股份有限公司 | Full-format media player supporting hardware decoding for digital set-top box |
| CN104754349A (en) * | 2013-12-25 | 2015-07-01 | 炫一下（北京）科技有限公司 | Method and device for hardware decoding of audio/video |
| CN104766608A (en) * | 2014-01-07 | 2015-07-08 | 深圳市中兴微电子技术有限公司 | Voice control method and voice control device |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210409473A1 (en) * | 2017-07-20 | 2021-12-30 | Disney Enterprises, Inc. | Frame-accurate video seeking via web browsers |
| US11722542B2 (en) * | 2017-07-20 | 2023-08-08 | Disney Enterprises, Inc. | Frame-accurate video seeking via web browsers |
| CN107566890B (en) * | 2017-09-15 | 2020-05-22 | 深圳国微技术有限公司 | Method, device, computer device and computer readable storage medium for processing audio stream playing abnormity |
| CN107566889A (en) * | 2017-09-15 | 2018-01-09 | 深圳国微技术有限公司 | Audio stream flow rate error processing method, device, computer installation and computer-readable recording medium |
| CN107566890A (en) * | 2017-09-15 | 2018-01-09 | 深圳国微技术有限公司 | Handle audio stream broadcasting abnormal method, apparatus, computer installation and computer-readable recording medium |
| CN107566889B (en) * | 2017-09-15 | 2020-06-16 | 深圳国微技术有限公司 | Audio stream flow velocity error processing method and device, computer device and computer readable storage medium |
| CN108401188A (en) * | 2018-03-05 | 2018-08-14 | 青岛海信传媒网络技术有限公司 | A kind of method and device of media play |
| CN108737874A (en) * | 2018-06-05 | 2018-11-02 | 武汉斗鱼网络科技有限公司 | A kind of video broadcasting method and electronic equipment |
| CN110418186A (en) * | 2019-02-01 | 2019-11-05 | 腾讯科技（深圳）有限公司 | Audio and video playing method, apparatus, computer equipment and storage medium |
| CN113141522A (en) * | 2020-01-17 | 2021-07-20 | 北京达佳互联信息技术有限公司 | Resource transmission method, device, computer equipment and storage medium |
| CN113141522B (en) * | 2020-01-17 | 2022-09-20 | 北京达佳互联信息技术有限公司 | Resource transmission method, device, computer equipment and storage medium |
| CN111918121A (en) * | 2020-06-23 | 2020-11-10 | 南斗六星系统集成有限公司 | Method and device for accurately editing streaming media file |
| CN114466238B (en) * | 2020-11-09 | 2023-09-29 | 华为技术有限公司 | Frame demultiplexing method, electronic device and storage medium |
| CN114466238A (en) * | 2020-11-09 | 2022-05-10 | 华为技术有限公司 | Frame demultiplexing method, electronic device and storage medium |
| CN113726778A (en) * | 2021-08-30 | 2021-11-30 | 咪咕视讯科技有限公司 | Streaming media seek method, device, computing equipment and computer storage medium |
| CN113965786A (en) * | 2021-09-29 | 2022-01-21 | 杭州当虹科技股份有限公司 | Method for accurately controlling video output and playing |
| CN113965786B (en) * | 2021-09-29 | 2024-03-26 | 杭州当虹科技股份有限公司 | Method for precisely controlling video output playing |
| CN114245231B (en) * | 2021-12-21 | 2023-03-10 | 威创集团股份有限公司 | Multi-video synchronous skipping method, device and equipment and readable storage medium |
| CN114245231A (en) * | 2021-12-21 | 2022-03-25 | 威创集团股份有限公司 | Multi-video synchronous skipping method, device, equipment and readable storage medium |
| CN116230029A (en) * | 2023-02-28 | 2023-06-06 | 深圳市中科蓝讯科技股份有限公司 | Audio file playback method and device, audio equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106686445B (en) | 2019-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106686445A (en) | Method of carrying out on-demand jump on multimedia file | |
| KR102650850B1 (en) | Video sound processing device, video sound processing method , and computer readable recording medium storing program | |
| CN112866586B (en) | Video synthesis method, device, equipment and storage medium | |
| CN109905719B (en) | Generating video with multiple viewpoints | |
| CN101409831B (en) | A method for processing multimedia video objects | |
| CN111836111A (en) | Technique for generating barrage | |
| CN105359544B (en) | Special play-back in digital video frequency flow transmission | |
| CN109218629B (en) | Video generation method, storage medium and device | |
| CN106686452B (en) | Method and device for generating dynamic picture | |
| CN104091607B (en) | Video editing method and device based on IOS equipment | |
| WO2015031548A1 (en) | Audio video playback synchronization for encoded media | |
| RU2011106942A (en) | PROCESSING 3D SUBTITLE DISPLAY | |
| US20170180746A1 (en) | Video transcoding method and electronic apparatus | |
| CN104091608B (en) | A kind of video editing method and device based on ios device | |
| JP2020528680A (en) | Modify digital video content | |
| CN104822078B (en) | The occlusion method and device of a kind of video caption | |
| TW200836510A (en) | Method and system for switching elementary streams on a decoder with zero delay | |
| CN105847252B (en) | A kind of method and device of more account switchings | |
| KR101854469B1 (en) | Device and method for determining bit-rate for audio contents | |
| KR20170067546A (en) | System and method for audio signal and a video signal synchronization | |
| JP2024079768A (en) | Information processing device and method, program, and information processing system | |
| CN106688015B (en) | Processing parameters for operations on blocks when decoding images | |
| CN105210366B (en) | Image processing apparatus and image processing method | |
| CN115359409A (en) | Video splitting method and device, computer equipment and storage medium | |
| US20140369422A1 (en) | Remultiplexing Bitstreams of Encoded Video for Video Playback |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |