CN109587581A - Video breviary generation method and video breviary generating means - Google Patents
Video breviary generation method and video breviary generating means Download PDFInfo
- Publication number
- CN109587581A CN109587581A CN201710909929.XA CN201710909929A CN109587581A CN 109587581 A CN109587581 A CN 109587581A CN 201710909929 A CN201710909929 A CN 201710909929A CN 109587581 A CN109587581 A CN 109587581A
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- video frame
- breviary
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 96
- 230000033001 locomotion Effects 0.000 claims description 82
- 239000013598 vector Substances 0.000 claims description 70
- 210000003813 thumb Anatomy 0.000 claims description 23
- 230000002123 temporal effect Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 8
- 230000001174 ascending effect Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000012552 review Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 abstract description 13
- 238000010586 diagram Methods 0.000 description 15
- 230000006835 compression Effects 0.000 description 11
- 238000007906 compression Methods 0.000 description 11
- 238000013139 quantization Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 239000000203 mixture Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005538 encapsulation Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000007423 decrease Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000032696 parturition Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 210000003811 finger Anatomy 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000005641 tunneling Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8549—Creating video summaries, e.g. movie trailer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the present application provides a kind of generation method of video breviary and the generating means of video breviary, this method include provide include multiple video frames video file；Obtain the one or more associations information of the video frame；Using one or more of related informations, the video frame for meeting and imposing a condition is determined；Using the video frame for meeting and imposing a condition, video breviary is generated.The application can be quickly generated video breviary, reduce the consumption to the computing resource of system to avoid being decoded to all videos frame and content analysis compared with the prior art.
Description
[technical field]
The invention relates to computer application technology, in particular to a kind of video breviary generation method and video
Breviary generating means.
[background technique]
Video is the important means that social people obtain information now as multimedia messages carrier.Relative to text and
Picture, video can carry more information, the content expressed required for faster accurately expression.
As the process that video generates, the photographer of video by photographic device, by raw data (such as multiframe picture and
Corresponding audio) it acquires and is stored in photographic device, and these original videos are handled using video generation tool, it is allowed to generate
The video file of specified format is watched for viewer.
But the viewer of video is difficult directly to find out the general contents of the video at a glance from one section of video.Especially
When not only there is important content in one section of video there are when insignificant content, user is difficult to directly find the emphasis paragraph of video.
For example, the content outline of the TV series can not be obtained when viewer is when viewing one collects TV series；Work as viewer
When in face of the video of bout, the excellent paragraph of the video can not be directly found；When viewer is broadcast live in face of primary video
When, it can not also obtain the excellent paragraph of the net cast.
To solve this problem, the prior art obtains video frequency abstract by the method for generating video breviary.The prior art is logical
It crosses and each frame in video file is decoded, and depth is carried out by one or more existing modes to decoded picture
The analysis and study of degree obtain key video sequence frame.Key video sequence frame refers to information content video frame abundant or a video
It is capable of the video frame of preferably reflecting video content in file compared to other video frames.It is generated using these key video sequence frames
Video breviary is capable of the general contents of more accurately reflecting video, guidance viewer's viewing.The prior art is by these key video sequences
Frame is integrated according to time shaft, video breviary is generated, with the summary info of reflecting video.
However, the computing resource consumption analyzed decoded each video frame and learnt for system is very big.It is right
In computing resource natively nervous system (such as server of heavy load), this method being decoded to each frame is not
Desirable.
In some video scenes, for example, net cast and video playback, the not exigent video definition of this kind of video,
But its duration is usually longer.When generating video breviary for this kind of video, if according to the prior art one by one to view
Frequency frame decoding simultaneously carries out explication de texte, then can waste computing resource significantly；In addition it is main that being broadcast live in class video, which has quite a few,
It broadcasts before being sitting in camera lens always, has no very big motion amplitude, it, also can not be accurately even if carrying out fine analysis to each frame picture
Obtain desired important video frame.
[summary of the invention]
In view of this, the embodiment of the present application application embodiment provides a kind of video breviary generation method and video breviary is raw
At device, to solve of the existing technology to need to carry out full dose decoding and/or content analysis to video, calculation amount is caused to increase
Add, generate the slow-footed technical problem of video breviary.
To solve the above problems, one embodiment of the invention proposes a kind of generation method of video breviary, which comprises
Video file including multiple video frames is provided；
Obtain the one or more associations information of the video frame；
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Using the video frame for meeting and imposing a condition, video breviary is generated.
Another embodiment of the present invention proposes a kind of net cast method, comprising:
Send live streaming access request；
Live video file is received, the live video file is the view generated according to above-mentioned video breviary generation method
Frequency file.
Further embodiment of this invention proposes a kind of generation method of video breviary, is applied to video and provides terminal, the view
The generation method of frequency breviary includes:
Video file including multiple video frames is provided；
Obtain the one or more associations information of the video frame；
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Using the video frame for meeting and imposing a condition, video breviary is generated；
The video breviary is uploaded to server-side.
Further embodiment of this invention proposes a kind of generating means of video breviary, comprising:
Video file provides module, for providing the video file including multiple video frames；
Related information obtains module, for obtaining the one or more associations information of the video frame；
Video frame determining module determines the video for meeting and imposing a condition for utilizing one or more of related informations
Frame；
Video breviary generation module generates video breviary for meeting the video frame to impose a condition using described.
Further embodiment of this invention proposes a kind of net cast device, comprising:
Access request sending module, for sending live streaming access request；
Live file receiving module, for receiving live video file, the live video file is according to above-mentioned view
The video file that frequency breviary generating means generate.
Further embodiment of this invention proposes a kind of generating means of video breviary, is applied to video and provides terminal, the view
The generating means of frequency breviary include:
Second video file provides module, for providing the video file including multiple video frames；
Second related information obtains module, for obtaining the one or more associations information of the video frame；
Second video frame determining module determines for utilizing one or more of related informations and meets setting condition
Video frame；
Second video breviary generation module generates video breviary for meeting the video frame to impose a condition using described；
Uploading module, for the video breviary to be uploaded to server-side.
The application also proposes a kind of computer equipment, including memory, processor and storage on a memory and can located
The computer program run on reason device, which is characterized in that the processor is realized when executing the computer program as above-mentioned
Method.
The application also proposes a kind of computer readable storage medium, is stored thereon with computer program, and the program is processed
Device realizes above-mentioned method when executing.
The embodiment of the present application is filtered out by sequence important by obtaining the significance level information dimension of each video frame
Degree value sorts forward video frame to be decoded, generates video breviary after the decoding, compared with the prior art can be to avoid right
All videos frame is decoded and content analysis, can be quickly generated the output of video breviary, disappear in the computing resource to system
In the lesser situation of consumption demand, the video breviary greatly reduced relative to original video size is generated, and greatly reducing
The viewing efficiency of user is promoted in the case where code stream.
[Detailed description of the invention]
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, below will be to institute in embodiment
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the application
Example, for those of ordinary skill in the art, without creative efforts, can also obtain according to these attached drawings
Obtain other attached drawings.
Fig. 1 show the RTP/RTMP transformat of the video file by H264/H265 encapsulation.
Fig. 2 show the form schematic diagram of RTP header.
Fig. 3 show the code flow structure figure of H264, and it is single that NAL (Network Abstraction Layer) is shown
The composition of member.
Fig. 4 show the code stream schematic diagram of Active SPS.
Fig. 5 show the application scenario diagram of one embodiment of the invention.
Fig. 6 and Fig. 7 show the schematic diagram of the video of two frame type of sports of front and back.
Fig. 8 show schematic diagram of the Fig. 7 relative to the motion vector of Fig. 6.
Fig. 9 show viewing terminal thumb up or comment information is synchronized to the schematic diagrames of other viewing terminals.
Figure 10 to Figure 14 show the flow chart of the video breviary generation method of each embodiment of the application.
Figure 15 to Figure 17 show the block diagram of the video breviary generating means of each embodiment of the application.
Figure 18 schematically show can be used for realize the disclosure described in each embodiment exemplary system (or
Device).
[specific embodiment]
It is with reference to the accompanying drawing and specific real in order to keep the purposes, technical schemes and advantages of the embodiment of the present application clearer
Example is applied the embodiment of the present application is described in detail.
The core idea of the embodiment of the present application is, proposes that a kind of video breviary generation method and video breviary generate dress
It sets, this method comprises: providing the video file including multiple video frames；Obtain the one or more associations letter of the video frame
Breath；Using one or more of related informations, the video frame for meeting and imposing a condition is determined；It is imposed a condition using the satisfaction
Video frame generates video breviary；Wherein, the related information may include the quantizing factor of video frame, video frame data volume,
Motion vector or the corresponding additional information of the video frame.User can quickly and accurately obtain information by the above method
Video frame abundant is measured, to generate the video breviary comprising key video sequence frame, avoids the prior art to each in video
Frame is decoded and learns the huge calculation amount of bring, saves the time for generating video breviary.
Scheme is generated in order to illustrate more clearly of video breviary of the invention, below by way of one of video format ---
For the video file of H264/H265 format, the video breviary generation method proposed to the embodiment of the present invention is illustrated.However
As those skilled in the art can clearly learn, the video file of following H264/H265 formats, which is only for example, to be used, not
As limitation.
One, the format of H264/H265 video file:
Video file by H264/H265 encapsulation has the transformat of RTP/RTMP as shown in Figure 1.In Fig. 1
Code stream after store the coding of H264 standard in NAL.Each NAL unit size is about 1.4KB, for encoded video frame
Larger situation can be stored in multiple RTP packets, is distinguish between different video frames using sequence number.
Fig. 2 show the format chart of RTP header.As shown in Fig. 2, wherein P indicates padding, for for alignment
VB vacancy bit；X indicates extension, indicates whether there is extension information and auxiliary information；CC is the identification of each data packet
Code, for the sequence of characterize data packet；M indicates marker, for indicating frame boundaries；PT indicates payload type, for fixed
The load format of adopted RTP format；Sequence Number is the sequence number of RTP data packet；Timestamp indicates the RTP number
According to the timestamp of packet.
In H264/AVC video encoding standard, whole system frame has been divided into two levels: Video coding level
(VCL) and network abstraction level (NAL).Wherein, the former is responsible for effectively indicating the content of video data, and the latter is then responsible for format
Change data and simultaneously provide information, to guarantee transmission that data are suitble on various channels and storage medium.Fig. 3 show the code of H264
The composition schematic diagram of NAL (Network Abstraction Layer) unit is shown in flow structure figure.As shown in figure 3,
In the Slice data in the Slice layer under the NAL unit, the MB of third position is Macroblock layer, the
One data is the type of video frame, i.e. I frame or P/B frame；In addition, the 4th data of the Macroblock layer
QP is quantizing factor, for characterizing the compression degree of video frame.5th Residual is residual error, for characterizing effective information；
Comprising INTRA and interframe INTER two types in frame in the data of second Prediction, wherein the of interframe INTER
Two are motion vector Motion Vector data.Motion change i.e. from previous frame to next frame.
About quantizing factor characterization video compress degree principle, it can be understood as using quantizing factor by initial data into
The decline of row data resolution.Therefore, quantizing factor is bigger, then the compression degree of video frame is higher, and effective information is fewer；Quantization because
Son is smaller, then the compression degree of video frame is lower, and effective information is more.
Fig. 4 is the code stream schematic diagram of the active SPS of video file.In first SPS in Fig. 4,29 can be passed through
This two rows data of=pic_width-in-mbs-minus1 and 29=pic_height-in_map_units-minus1 calculate
The resolution ratio of video frame, so that the data volume of the frame, i.e. size of data are calculated, to characterize the abundant information journey of the frame video
Degree.
Table 1 show PT coding signals.As shown in table 1, in PT, 0 to 19 is audio bits of coded, and 35-
127 be space bit, these positions can be reserved as from embedding subtitle, background music etc..
Table 1
Two, the coding and decoding in net cast and playback scenario
It is illustrated in figure 5 the schematic diagram of a kind of net cast and playback scenario.In existing net cast and playback, depending on
Frequency provides terminal 10, server 20 and at least one viewing terminal 30 and constitutes live streaming and playback scenario.Net cast and playback
Scene includes live scene and playback scenario, and live scene provides terminal 10 and viewing terminal 30 while online, video for video
The scene of 10 uploaded videos content of terminal, the acquisition video content of viewing terminal 30 is provided, playback scenario provides terminal 10 for video
It has completed to upload, and viewing terminal 30 is in the scene of viewing playback of another moment.
In live scene, video provides terminal 10 for acquiring original data stream, uploads to service as video content
Device 20 is encoded by server 20 according to agreements such as H264, H265.Original data stream is that each pixel of each frame is
The picture element matrix of RGB or yuv data composition.By coding, this original data stream is by specific protocol conversion at specific
Format, such as RTP or RTMP.
Viewing terminal 30 can be from the video file after obtaining coding in server 20, such as RTP RTMP format
Each frame after being decoded according to same agreement, is returned to the picture element matrix of RGB or yuv data composition by video file
Viewing.
Decoded video file is different and different with the content of the data of original upload, generally comprises image and sound
Frequently, additional informations such as including subtitle in some cases.
Three, inhomogeneous video file
3.1 are directed to convention video file
A usual video file includes image and audio.In this latter case, the video for uploading the video provides eventually
The original video data stream including image and audio of 10 acquisition of end is simultaneously uploaded to server 20.In server 20, to above-mentioned original
Beginning video data stream is encoded, and according to tunneling, generates the coded video frequency file based on time shaft, such as can be for such as
Lower format:
" video starting character++ 1 frequency starting character of video code flow 1+ video end mark+audio code stream 1+ audio end mark
+ video starting character+video code flow 2+ video end mark+audio starting character+audio code stream 2+ audio end mark ...
+ video starting character+video code flow n+ video end mark+audio starting character+audio code stream n+ audio end mark "
Or can be following format:
" video starting character+video length 1+ video code flow 1+ audio starting character+audio length 1+ audio code stream 1
+ video starting character+video length 2+ video code flow 2+ audio starting character+audio length 2+ audio code stream 2 ...
+ video starting character+video length 3+ video code flow n+ audio starting character+audio length n+ audio code stream n "
Above-mentioned video format is only illustrated for convenience of description and is used, and is not intended as limiting herein.Video code flow 1 to n can be with
A respectively frame image, or corresponding to the video code flow of a specified duration, audio code stream 1 to n can be corresponded to a frame figure
As or corresponding to a specified duration audio code stream.
Server 20 is directed to the video file generation video breviary after the coding.It in embodiments of the present invention, can be to volume
Video file after code is analyzed, obtain video code flow 1, video code flow 2 ..., the data volume of video code flow n.Each section
Video code flow can correspond to a video frame, can not also exactly correspond to each video frame.When each section of video code flow
When corresponding to a video frame, the data volume of each section of video code flow is the data volume of the video frame；When each section of video codes
Stream is not when exactly corresponding to a video frame, can be according to time shaft (time of each frame is, for example, 1/24 second) by these videos
Code stream is corresponding with each frame.In the following description, for convenience's sake, a video is corresponded to each section of video code flow
It is illustrated for frame.Similarly, from the video file after coding, can obtain audio code stream 1, audio code stream 2 ... sound
The data volume of frequency code stream n.By the way that the data volume of video code flow 1 to be added with the data volume of audio code stream 1, first frame can be obtained
The data volume of video frame.
Can analogically, in some only video files including video code flow, the data volume of above-mentioned first frame video is
For the data volume of video code flow 1.
The data volume of video frame can characterize the abundant information degree of the video frame.In the data for obtaining each video frame
After amount, the video frame for meeting and imposing a condition can be obtained according to the data volume of video frame.For example, the setting condition can be with are as follows:
The video frame that data volume is greater than the set value.In order to obtain the video frame that data volume is greater than the first setting value, video frame can be pressed
It is ranked up according to data volume, and selected and sorted result is greater than the part of the first setting value from multiple video frames according to ranking results
Video frame；Alternatively, in order to obtain the video frame that data volume is greater than the first setting value, it can be in the data for obtaining each video frame
After amount, the data volume of the video frame is compared with the first setting value, if data volume is greater than the first setting value, is selected
The video frame.After selecting the video frame for meeting and imposing a condition, these video frames are decoded, and are utilized decoded
Video frame generates video breviary.
In addition to data volume, the quantizing factor of each video frame also can reflect the video frame in the video code flow after coding
Abundant information degree.Quantizing factor is for declining the resolution ratio of data, and quantizing factor is bigger, then the compression of the frame video
Degree is higher, and data volume is smaller.Therefore as another optional mode, the video file after coding can be analyzed,
The quantizing factor of each video frame is obtained, the video frame for meeting and imposing a condition is obtained.For example, the setting condition can be with are as follows: amount
Changing the factor can be to view in order to obtain video frame of the quantizing factor less than the second setting value less than the video frame of the second setting value
Frequency frame is ranked up according to quantizing factor, and according to ranking results from multiple video frames selected and sorted result less than second setting
The partial video frame of value；Alternatively, in order to obtain video frame of the quantizing factor less than the second setting value each view can be being obtained
After the quantizing factor of frequency frame, the quantizing factor of the video frame is compared with the second setting value, if quantizing factor is less than
Second setting value then selects the video frame.After selecting the video frame for meeting and imposing a condition, these video frames are solved
Code, and video breviary is generated using decoded video frame.
It is worth noting that, although above-mentioned is the video text for RTP the RTMP format of H264/H264 protocol encapsulation
The explanation that part carries out, but the present invention is not limited thereto.For example, being directed to extended formatting, such as the video text of Vp8/Vp9 format
Part, method provided in an embodiment of the present invention is equally feasible, as long as can obtain the number of each video frame in video code flow
The case where according to amount and quantizing factor, is included in the range of of the invention protect.
By above-mentioned it is found that be directed to some species of video file, can by obtain each video frame data volume or
Person's quantizing factor, and select the video frame for meeting and imposing a condition using data volume or quantizing factor and generate video breviary.Pass through
This mode can quickly and accurately generate video breviary for video file, avoid being decoded all videos frame with it is interior
Hold analysis, under the premise of the computing resource consumption requirements to system are lesser, generates the video contracting for being less than original video data amount
Slightly, facilitate user's quick obtaining video frequency abstract.
3.2 are directed to the video file comprising subtitle
In video file comprising subtitle, subtitle can be embedded or plug-in.Illustrate individually below:
When subtitle is embedded subtitle, there are two kinds of situations:
The subtitle can be for example integrated into the position 35-127 in Fig. 5 in the case of the first.It in this latter case, can be with
The case where according to 3.1, analogizes, can be by the data volume of video code flow 1, audio code in the video file for containing caption information
Flow the data volume of the sum of 1 data volume and the data volume of subtitle as the first video frame.
In this latter case, the data volume comprising video code flow, the data volume of audio code stream and subtitle number can be obtained
According to the data volume of the video frame of amount, and the data volume of video frame is selected to meet the video frame to impose a condition.It can for example, imposing a condition
Think that the data volume of video frame is greater than third designated value, therefore the data volume of video frame can be ranked up, obtains data volume
Greater than multiple video frames of third designated value.
Alternatively, can use the data volume of subtitle, individually sorted to corresponding video frame.Such as it can be from 35-127
The data volume that subtitle is extracted in position, is ranked up video frame.In this latter case, the data of multiple video frames can be obtained
Amount, and the data volume of subtitle is selected to meet the video frame to impose a condition.For example, imposing a condition can be greater than for the data volume of subtitle
4th designated value, therefore the data volume of subtitle can be ranked up, the data volume for obtaining subtitle is greater than the more of the 4th designated value
A video frame.
Subtitle is inserted into video frame under second situation, becomes a part of the image of each video frame.This
In the case of, the data volume of subtitle can not be extracted individually in the case, can only be as a part of the data volume of image, this
In the case of, it still can use the data volume or quantizing factor of each video frame, these video frames be ranked up.
When subtitle is plug-in subtitle, which usually can be used as independent file, provide terminal 10 by video
It is uploaded to server 20, is located at from the video file after coding in different files.In such a case, it is possible to by every
The data volume of each frame of the data volume and video file after coding of the corresponding subtitle of one video frame adds up, and obtains each
The corresponding total amount of data of video frame, and total amount of data is selected to meet the video frame to impose a condition.Setting condition for example can be total
Data volume is greater than the 5th designated value.Therefore it can use total amount of data to be ranked up these video frames, select total amount of data big
In the video frame of the 5th designated value；Alternatively, can obtain merely with the data volume of the corresponding subtitle of each video frame and meet setting
The video frame of condition.
By above-mentioned it is found that be directed to the video file comprising subtitle, each video frame after same available coding
Data volume perhaps quantizing factor and selects data volume or quantizing factor to meet the video frame to impose a condition, generates video breviary.
According to the difference of subtitle type, the data volume of subtitle may include that can also individually extract as row in above-mentioned data volume
The foundation of sequence.It can be quickly generated video breviary by this mode, avoid being decoded all videos frame and content is divided
Analysis generates the video breviary for being less than original video data amount, side in the lesser situation of computing resource consumption requirements to system
Just user's quick obtaining video frequency abstract.
3.3 are directed to the biggish video type of motion amplitude
Video frame can be divided into I frame, P frame, B frame according to the difference of frame type: I frame indicates key frame, and when decoding only needs
This frame data can be completed, and this kind of video frame key message is abundant, therefore compression ratio is lower, due to its be not related to movement to
Amount, therefore can be ranked up according to its data volume.P frame indicates the difference of this current frame with an I frame or P frame before,
It needs the picture cached before to be superimposed with the difference of this frame definition when decoding, generates final picture；B frame indicates two-way difference
Frame, that is, what is recorded is the difference of this frame and before and after frames.
The biggish video type of motion amplitude is, for example, that the movement between sport type games or every two frame of moving person becomes
Change other biggish videos, it is characterized in that the motion vector amplitude of variation of the video of adjacent two frame is larger.Fig. 6 show a kind of fortune
The schematic diagram of the dynamic relatively large two frames video frame of amplitude.
Motion vector difference between the frame and frame of this type video is larger.In some cases, motion vector difference
It is larger, show that the frame is to be able to reflect more important content.Due to the P frame of motion vector reflection and the video of B frame type
In frame, therefore, different rule selection video frames can be respectively adopted, use in conjunction with the frame type (I frame, P frame, B frame) of video frame
To generate video breviary.
The compression ratio of P frame and B frame is higher.It is related to motion vector, satisfaction can be selected according to the size of motion vector and set
P frame/B frame of fixed condition.For example, can be ranked up to P frame/B frame, Fig. 7 show the video frame of Fig. 6 according to motion vector
Motion vector schematic diagram.As shown in fig. 7, in the motion vector figure, it can be seen that next frame is compared to a upper Zhang Yundong
Track, the more long then motion amplitude of motion vector is bigger.It is worth noting that motion vector is with coordinate in P frame and B frame
What form indicated, it in order to visualize is illustrated, is illustrated in form of arrows in fig. 8.
It since there are multiple motion vectors in same frame, can be normalized, obtain motion vector scoring.Example
Such as, according to multiple motion vector amplitudes of video frame carry out read group total, or be homogenized calculate obtain motion vector it is equal
Value, or the Energy distribution of the motion vector of video frame can also be calculated, that is, pass through the coordinate form of vector --- (x, y) meter
Calculate its energy value (x2+y2)2, then the energy value summation of the cumulative all motion vectors for obtaining the video frame, for example, by sequence
Or the modes such as compare, obtain the video frame for meeting and imposing a condition.Above-mentioned setting condition is, for example, that numerical value is more than or equal to the 6th finger
Definite value, or be ranked up according to numerical values recited, select I-frame video frame, P frame video frame, the B frame video of certain ratio
Frame.
In the present embodiment, the frame type of video frame can be obtained according to above-mentioned video code flow.For example, setting I frame, P frame,
The selection percentage of B frame is 6:3:1, then includes 60% I frame, 30% P frame and 10% B in selected video frame out
Frame.When the totalframes of expected video skimming is 100, need to select to meet 60 frames to impose a condition from I-frame video frame, from P
Selection meets 30 frames to impose a condition in frame video frame, and selection meets 10 frames to impose a condition from B frame video frame.
In the present embodiment, it if imposing a condition is the N number of video frame to sort before examination, can be ranked up for I frame,
And it is ranked up for the combination of P frame and B frame.Later according still further to the forward I frame of certain proportion selected and sorted and P frame and B frame
Combination.In the present embodiment, P frame and B frame are not distinguished.For example, the combined selection ratio of setting I frame and P frame and B frame
Example is 6:4, then includes the combination of 60% I frame and 40% P frame and B frame in selected video frame out.When expected breviary
When the totalframes of video is 100, need to select 60 frames from the sequence of I-frame video frame, from the combined sequence of P frame and B frame
Select 40 frames.
In the combination for P frame and B frame is ranked up, or sort respectively for P frame and B frame video frame, normalizing
The motion vector for changing processing can characterize motion amplitude of the current video frame relative to previous or latter video frame, and then characterize and work as
The abundant information degree of preceding video frame.P frame and/or B frame video frame can be ranked up according to the motion vector.
It later, can the forward partial video frame of selected and sorted result be solved from multiple video frames according to ranking results
Code, or selection meets the video frame to impose a condition from multiple video frames, also according to H264, H265 agreement, by video codes
Stream reduction becomes RTP or RTMP format, and generates video breviary using decoded video frame.
It, such as can be with needle by above-mentioned it is found that the method that a kind of video breviary that third embodiment of the invention proposes generates
It, can be with for the P frame and B frame of such video file to movement, type of play or the biggish video file of other motion amplitudes
Motion vector scoring is obtained, and selection meets the P frame and B frame to impose a condition accordingly；For the I frame of such video, can obtain
Data volume, quantizing factor or additional information, and selection meets the I frame to impose a condition accordingly, then will meet the video to impose a condition
Frame is decoded to generate video breviary.Video breviary generation method proposed by the present invention is not necessarily to carry out full dose solution to video frame
Code, it will be able to quickly and relatively accurately select the video frame of informative to generate video breviary, while reduce to system
Computing resource consumption requirements.
3.4 for there are the videos of additional information
In one case, there are additional informations for video, while main body (such as personage) motion amplitude is little in video
Video, such as can be to occur more live streaming class video now.Be broadcast live class video main broadcaster in live streaming whole process with sitting posture face
To camera lens, do not moved significantly generally just like movement class video usually.Video content collecting is uploaded to server by main broadcaster
20, multiple viewing terminals 30 synchronously or asynchronously obtain the live content from server 20 and are watched.
The motion vector difference being broadcast live between the frame and frame of class video is little, but there may be additional letters for live streaming class video
Breath, the additional information can be liveness index, such as viewer thumbs up, comments on, and the content that can measure corresponding frame is
It is no important.These contents such as thumb up, comment on usually as independent file, are located at from the video file after coding different
In file.
In net cast, other than image and audio, it is also possible to which there are what viewing terminal 30 was sent to thumb up, comment on
Additional information.The additional information relevant to video content that server 20 can also issue viewing terminal 30 according to time shaft,
It is corresponding with video file to integrate.There is the region for individually storing the additional information in server.For example, for
To the video content between 2s, the viewing terminal 30 that the value of participation is seen live has carried out thumbing up operation 1s.This thumbs up operation and uploads
To server 20, and by server 20 according to time shaft it is corresponding to 1s into the video content between 2s.Since video is straight
Weighting interactivity is broadcast, the information that thumbs up of viewing terminal 30 is sent to other viewing terminals, therefore, net cast simultaneously by server
Terminal and each viewing terminal, which can obtain, thumbs up information.With when subsequent video plays back, in the viewing terminal to viewing playback
When 30 playing video file, the operation that thumbs up for the viewing terminal 30 that the value of participation is seen live can be sent to viewing playback simultaneously
Viewing terminal.
In video playback, the video content that terminal 10 uploads is provided due to being stored with video in server, and
According to time shaft integration and these additional informations relevant to video content in server 20, whenever subsequent viewing playback
When viewing terminal 30 watches the video content, when playing to 1s to content between 2s, viewing terminal 30 can be seen
Operation is thumbed up to other viewing terminals 30.Comment information with it is similar or identical the case where thumbing up information, details are not described herein.
When generating video breviary to such video, said additional information can be corresponded to often using time shaft
One video frame obtains the data volume and the corresponding additional information amount of each video frame of each video frame, according to superimposed view
The total amount of data of frequency frame determines the video frame for meeting setting condition, for example, can be ranked up to video frame, that is, sets item
Part is that data volume is greater than the 7th designated value, therefore can select top N from the video frame of sequence.Alternatively, can be according only to every
The corresponding additional information amount of one video frame is ranked up each video frame.It later, can be according to ranking results from multiple videos
The forward partial video frame of selected and sorted result is decoded in frame, and also with H264, H265 agreement, video code flow is restored
Video breviary is generated as RTP RTMP format, and using decoded video frame.
For example, above-mentioned " additional information is corresponded to each video frame according to time shaft " can be accomplished in that
Firstly, obtaining the additional information in designated time period；Secondly, when the additional information in this period is fifty-fifty corresponded to this section
Between each frame in.Above-mentioned " in designated time period " can voluntarily be specified by developer, and designated time period is shorter, then accurately
Property is higher.It is commented on for example, receive viewer 12 of the video in 1s to 2s praise with 12 articles, is calculating each video frame
When corresponding additional information amount, can praise conversion for 12 becomes 12 score values, and 12 comment conversions become 12 score values, then in 1s
The score value of the additional information of video in 2s is 24 points.Each video frame is averagely corresponded to by this 24 points, then 1s to
Each video frame is scored at 1 point in 2s.
By it is above-mentioned it is found that fourth embodiment of the invention propose it is a kind of there are the video of additional information (such as live streaming class
Video) video breviary generate method using the corresponding additional information amount of each video frame, pass through in this embodiment
Such as the modes such as sort, compare obtain the video frame for meeting and imposing a condition and are decoded, decoded without video frame full dose
Under the premise of improve the probability that important video frame is selected.The computing resource consumption requirements to system are reduced, are generated less than original
The video breviary of the video data volume facilitates user's quick obtaining video frequency abstract.
In conclusion different video types may correspond to different related informations, for ease of understanding, following table 2 is listed
The type of partial video file and the corresponding relationship of related information.
Table 2
Figure 10 show the flow chart of the video breviary generation method of one embodiment of the invention, as shown in Figure 10, the video
Breviary generation method may include steps of:
S101 provides the video file including multiple video frames；
In this step, video shown in fig. 5 provides terminal 10 and video code flow is uploaded to server 20.
Such as in live scene, video provides terminal 10 and acquires original video data stream and be uploaded to server 20.
Original video data stream is each pixel RGB of each video frame or the picture element matrix of yuv data composition, is mentioned in video
For terminal 10, original data stream is converted into specific format, and by transmission modes such as networks, is uploaded to server 20.Clothes
Business device 20 obtains the video file and carries out subsequent operation.
In one embodiment, video file provides terminal 10 in video and is encoded and uploads, and coding is, for example, by video text
Part is converted into the formats such as H264, H265, Vp8, Vp9, and is transmitted.However, in step s101, not limiting video text
Whether part is by coding.
S102 obtains the one or more associations information of the video frame；
It in this step, can be from the data volume for wherein obtaining each video frame, according to each for the video file
The data volume of video frame measures the abundant information degree of the frame video, judges that the video frame is when generating video breviary with this
No is key video sequence frame.
For example, executing subject, for example, server can compare coding protocol --- the association of such as H264/H265
View, the data volume of each video frame in the video is obtained by the fast resolving to video code flow.It, can be in conjunction with figure as above-mentioned
In 4
This two row of 29=pic_width-in-mbs-minus1 and 29=pic_height-in_map_units-minus1
Data calculate the resolution ratio of video frame, to calculate the data volume of the frame video frame with reference to resolution ratio.Therefore, step
S103, that is, the step of determining the data volume of the video frame for example may include: the resolution ratio of the video frame with reference to the coding,
Determine the data volume of the video frame.But as those skilled in the art can be specific, video frame is calculated with reference to resolution ratio
Data volume be only one of data volume for obtaining video frame method, the present invention do not limit.
S103 determines the video frame for meeting and imposing a condition using one or more of related informations；
In one embodiment, above-mentioned related information for example may include the quantizing factor of video frame, video frame data volume,
Motion vector or the corresponding additional information of the video frame.
S104 generates video breviary using the video frame for meeting and imposing a condition.
In this step, it can use the video frame that above-mentioned satisfaction imposes a condition and generate video breviary.The step for
In, such as can be by integrating these video frames according to the sequence of time shaft, and be decoded, utilize decoded view
Frequency frame generates video breviary.
Above-mentioned video breviary can be the independent video after breviary, for example, selected and decoded video frame is former view
The video frame of 100ms, 500ms and 800ms in frequency file, then in the video breviary generated, above-mentioned video frame is located at
100ms, 200ms and 300ms.Thus user can obtain video outline or abstract, quickly and easily confirm the interior of video
Hold.In addition, above-mentioned video breviary is also possible to the combination of the video frame arranged according to time shaft.In user in original video text
When dragging progress bar on part, when dragging to corresponding time point, the picture of these video frames can be accordingly shown.Such as
When wherein a frame corresponds to the 200ms in original video in video breviary, then a node is established in the 200ms of original video, when
When user drags progress bar to this node, shown picture out is this decoded frame.
In one embodiment, above-mentioned video frame is the video frame of coding, step S104 utilizes the view for meeting setting condition
Frequency frame, generate video breviary the step of may include:
It is decoded to the video frame to impose a condition is met, generates video breviary using decoded video frame.
Figure 11 show second embodiment.In the embodiment shown in fig. 11, when the related information of step S103 includes view
When the data volume of frequency frame, the data volume of video frame may include image data amount and audio number according to the difference of packaged content
According to amount.For some video files not comprising audio, the data volume of video frame can also be individually for image data amount.For one
The video file of the additional informations such as subtitle has been embedded a bit, and the data volume of video frame may include the data of image, audio and subtitle
The summation of amount.
In this step, the e.g. data volume that imposes a condition is greater than the first designated value.When the data volume of video frame meets
When this setting condition, the video frame can choose, as video frame to be decoded.Setting condition for example can also be data volume
The video frame for the top N that sorts can be ranked up video frame according to the size sequence from high to low of data volume herein, and select
The video frame for selecting sequence top N, as video frame to be decoded, for generating video breviary.The video breviary is for example for giving birth to
At net cast.That is, step S103 for example may include following sub-step:
S1031 is ranked up the multiple video frame according to the size of the data volume；
S1032, at least one forward video frame of selected and sorted meet the video frame to impose a condition as data volume.
In one embodiment, it in order to generate frame number according to the equally distributed video breviary of time shaft, can will encode
Video file afterwards is divided into multiple subfiles according to temporal information, then is ranked up for the data volume of each subfile.
Then in step S102, that is, before the step of obtaining the one or more associations information of the video frame, the method
Further include:
Video file is divided into multiple subfiles according to temporal information by S100；
For each subfile, following steps are executed:
Obtain the one or more associations information of the video frame；
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Using the video frame for meeting and imposing a condition, video breviary is generated.
Above-mentioned the step of executing for each subfile, is same or similar with step S102 to S104, and details are not described herein.
It can be seen from the above, the embodiment of the present application passes through the data for obtaining each video frame when related information is data volume
Amount selects the video frame for meeting and imposing a condition according to data volume, generates video breviary using the video frame for the condition that meets.
It is decoded the mode with content analysis compared to for all videos frame, the embodiment of the present invention is met by selection
The video frame of setting condition can reduce the calculation amount of system；Compared to the side that interception partial video paragraph generates video breviary
Formula, the embodiment of the present invention can quickly and accurately select the video frame of informative raw by the data volume of acquisition video frame
It is exported at video breviary, avoids mistakenly leaving out the important content of video during interception video, and then lead to not generate
Relatively comprehensive video breviary.
In another embodiment, the related information of video frame includes the quantizing factor of video frame；Figure 11 a show association letter
Schematic flow chart of the present invention when breath is quantizing factor.In the present embodiment, step S100, step S101, step S104 with
A upper embodiment is identical, and step S102 can be with are as follows: obtains the quantizing factor of the video frame.And step S103 is using one
Or multiple related informations, determine that the step of meeting the video frame to impose a condition may include:
S2031 is ranked up the multiple video frame according to the size of the quantizing factor；
S2032 selects at least one the forward video frame that sorts, and the video frame to impose a condition is met as quantizing factor.
In above-mentioned steps, in conjunction with shown in earlier figures 3, the 4th data QP of the Macroblock layer is quantization
The factor, for characterizing the compression degree of video frame.Server can obtain the quantization of each video frame from each video file
The factor.As above-mentioned, quantizing factor is used to initial data carrying out data resolution decline, and quantizing factor is bigger, then video frame
Compression degree is higher, and effective information is fewer；Quantizing factor is smaller, then the compression degree of video frame is lower, and effective information is more.
The compression ratio that the frame video is characterized since quantizing factor is ascending, the e.g. quantizing factor that imposes a condition is less than the
The video frame of two designated values determines that quantizing factor meets by comparing the quantizing factor and the second designated value of each video frame
The video frame of the setting condition；Alternatively, by the ascending sequence of quantizing factor, M video frame sorting forward.In a kind of this feelings
Under condition, it can use the ascending sequence of quantizing factor and multiple video frames be ranked up, determine the forward M video that sorts
Frame.
It can be seen from the above, the embodiment of the present application passes through the amount for obtaining each video frame when related information is quantizing factor
Change the factor, the video frame for meeting and imposing a condition is selected according to quantizing factor, generates video using the video frame for the condition that meets and contract
Slightly.
It is decoded the mode with content analysis compared to for all videos frame, the embodiment of the present invention is met by selection
The video frame of setting condition can reduce the calculation amount of system；Compared to the side that interception partial video paragraph generates video breviary
Formula, the embodiment of the present invention can quickly and accurately select the video frame of informative by the quantizing factor of acquisition video frame
The output of video breviary is generated, avoids mistakenly leaving out the important content of video during interception video, and then lead to not give birth to
At relatively comprehensive video breviary.
Figure 12 show the flow chart of the video breviary generation method of another embodiment of the present invention, as shown in figure 12, the view
Frequency breviary generation method may include steps of:
S301 provides the video file including multiple video frames；
The step for it is identical as above-mentioned steps S101, details are not described herein.
S302 determines that the frame type of the video frame, the frame type include I frame, P frame and B frame；
It in this step, can be from the Slice data in the Slice layer under NAL unit shown in Fig. 3
The type of the primary acquisition video frame of the Macroblock layer of third position, the i.e. video frame are I frame or P frame/B
Frame.
S303 is determined and is met the first I-frame video frame to impose a condition；
S304 is determined and is met the second P frame video frame and B frame video frame to impose a condition；
In above-mentioned two step, there is different attributes in view of I frame and P frame, B frame video frame, can be set different
It imposes a condition, I frame and P frame, B frame video frame is selected respectively.
For example, I-frame video frame key message is abundant, compression ratio is lower, is not related to motion vector, therefore can be directed to I frame
Video frame obtains the video frame for meeting and imposing a condition according to its data volume or quantizing factor；For example, the step for S303
In, I-frame video frame can be ranked up according to the sequence of data volume from high to low, can also according to quantizing factor by it is small to
Big sequence is ranked up I-frame video frame.
Including multiple motion vectors in P frame and B frame can be according to fortune after these motion vectors are normalized
Moving vector scoring carries out descending sequence.Wherein the scoring of motion vector can carry out normalizing by the motion vector of the frame
Change processing to obtain.In step s 304, can score descending sequence according to above-mentioned motion vector, to obtained multiple
P frame video frame and the sequence of B frame video frame.It may include two ways to the sequence of P frame/frame, one is be ranked up simultaneously to P frame
B frame is ranked up, another kind is to carry out whole sequence to P frame and B frame, and ignore difference therein.
For example, can be carried out in the following way to the processing that motion vector is normalized:
Calculate the total movement vector magnitude summation of each video frame；
Calculate the average value of the total movement vector magnitude of each video frame；Or
Calculate the energy distributions of the total movement vector magnitude of each video frame.
The average value of motion vector amplitude can be averaged by the amplitude of each motion vector to the video frame
It calculates and obtains, and energy distributions can pass through the coordinate form of each vector --- (x, y) calculates its energy value (x2+
y2)2To obtain energy value, then accumulation calculating or average computation are carried out, acquisition can measure Distribution Value.
Therefore, the above-mentioned first data volume for imposing a condition for example, I-frame video frame is greater than a certain setting value, or according to
The small Mr. Yu of quantizing factor of top n (perhaps preceding 1/n) I-frame video frame or I-frame video frame after the descending sequence of data volume
One setting value either has small top n (or preceding 1/n) I-frame video frame etc. to after big sequence, the present invention according to quantizing factor
It is not specially limited.
Above-mentioned second set the motion vector scoring that a condition is, for example, P frame video frame and B frame video frame be greater than it is a certain
Setting value, top n (or preceding 1/n) P frame video frame and B frame video after descending sequence of either scoring according to motion vector
Frame etc., the present invention are not specially limited.
In one embodiment, the step of step S303, the i.e. determining I-frame video frame for meeting the first setting condition, is for example
Include:
S3031 obtains the data volume of the I-frame video frame with reference to the resolution ratio of the video frame of the coding.
In this sub-step, the resolution ratio of the I-frame video frame after can use coding obtains the data volume of the video frame.
S3032 is ranked up the multiple I-frame video frame according to the size of data volume；
S3033, at least one forward I-frame video frame of selected and sorted meet the I-frame video to impose a condition as data volume
Frame.
In some cases, the data volume of I-frame video frame not only includes image data amount, and including and audio data
One of amount and caption data amount or combination, then step S3031 could alternatively be:
S3031 ', with reference to the image data amount and the corresponding audio data of the I-frame video frame of the I-frame video frame
Amount and caption data amount in one of or combination, determine the data volume of the I-frame video frame.
In one embodiment, the step of step S303, the i.e. determining I-frame video frame for meeting the first setting condition, is for example
Include:
S3034 is ranked up the multiple I-frame video frame according to the sequence of quantizing factor from small to large；
S3035, at least one forward I-frame video frame of selected and sorted meet the I frame view to impose a condition as quantizing factor
Frequency frame.
In view of the aforementioned method for having been described above and obtaining quantizing factor, details are not described herein.
S305 generates video breviary using identified I-frame video frame and P frame and B frame video frame；
In this step, video breviary is generated using decoded video frame.In this step, such as can pass through
These video frames are integrated according to the sequence of time shaft, and generate video breviary.Above-mentioned video breviary is for example for video
Live streaming.
In one embodiment, it in order to generate frame number according to the equally distributed video breviary of time shaft, can will encode
Video file afterwards is divided into multiple subfiles according to temporal information, then obtains its data volume for the I frame of each subfile,
And its motion vector is obtained for the P frame and B frame of each subfile, I frame and P frame/B frame are ranked up respectively.
That is, in step S301, before the frame type for determining the video frame, this method further include:
The video file is divided into multiple subfiles according to temporal information by S300, for each subfile, is executed
Following steps:
Determine that the frame type of the video frame, the frame type include I frame, P frame and B frame；
It determines and meets the first I-frame video frame to impose a condition；
It determines and meets the second P frame video frame and B frame video frame to impose a condition；
Using identified I-frame video frame and P frame and B frame video frame, video breviary is generated.
It can be seen from the above, the video breviary generation method that the embodiment of the present application proposes at least has the advantages that
The embodiment of the present application is directed to the different characteristics of I-frame video frame and P frame video frame and B frame video frame, according to difference
Setting condition selection I-frame video frame P frame video frame, B frame video frame generate video breviary, be based on I-frame video frame and P frame video
Frame, the feature of B frame video frame respectively, can more accurately extract the important video in each type video frame.
In addition, being decoded the mode with content analysis compared to for all videos frame, the embodiment of the present invention passes through choosing
The video frame for meeting and imposing a condition is selected, can reduce system to the decoded calculation amount of video frame full dose；It is regarded compared to interception part
Frequency range is born into the mode of video breviary, and the embodiment of the present invention can quickly and accurately select the video frame of informative, raw
It is exported at video breviary, avoids mistakenly leaving out the important content of video during interception video, and then lead to not generate
Relatively comprehensive video breviary.
Figure 13 show the flow chart of the video breviary generation method of another embodiment of the present invention, as shown in figure 13, the view
Frequency breviary generation method may include steps of:
S401 provides the video file including multiple video frames；
The step for it is identical as step S101, details are not described herein.
S402 determines the additional information amount of the corresponding additional information of the video frame；
Additional information, which for example may include that each video frame is corresponding, thumbs up information and comment information.Thumbing up information is, for example,
Content and/or number are thumbed up, comment information is, for example, to comment on content and/or number.Alternatively, additional information may include the video
The corresponding plug-in subtitle of file.The additional information correspondence shown as in temporal information corresponding with video frame, when can be by one section
Between the corresponding additional information of length averagely arrive each video frame of this time, obtain the additional information amount of each frame；If certain
One video frame is corresponding to thumb up number and/or number of reviews or subtitle amount is larger, it is believed that additional information amount is larger, corresponding
Video frame it is important.
In this step, the corresponding additional information amount of video frame can be obtained in the following way:
The processing that is weighted for thumbing up number and number of reviews in a period of time is obtained total amount by first way,
In thumb up number the weight of weight and number of reviews can be with self-setting；Above-mentioned total amount is evenly distributed to this period of time again
Each video frame, obtain the additional information amount of each video frame；
The information that thumbs up in a period of time can be converted to data volume, and comment content for example may be used by the second way
To be converted to data volume according to number of words, the two, which is added, obtains additional information amount.A byte can be converted to by thumbing up information for example,
It is added again with comment content, obtains total amount；Above-mentioned total amount is evenly distributed to each video frame of this period of time again, is obtained every
The additional information amount of one video frame；
The third mode can obtain the corresponding caption information of each frame video frame when additional information is caption information
Data volume, as additional information amount；Likewise it is possible to thumb up letter for the data volume of caption information is corresponding with each video frame
Breath is added with the data volume of comment information, obtains the additional information amount of each frame video.
S403 determines that additional information amount meets the video frame to impose a condition；
In this step, impose a condition for example, additional information amount be greater than a certain setting value video frame, or for by
It is located at the video frame of top N (or preceding 1/n) according to the sequence of additional information amount.Herein can according to additional information amount to video frame into
Row sequence, the video frame of part is selected according to ranking results.
S404 generates video breviary using the video frame for meeting and imposing a condition.
In this step, it can use the video frame that above-mentioned satisfaction imposes a condition and generate video breviary.Such as it can lead to
It crosses and integrates these video frames according to the sequence of time shaft, and generate video breviary.Above-mentioned video breviary is for example for giving birth to
At net cast.These video frames can be for example the video frame of coding, and step S404 for example can be with are as follows: to it is described be more than setting
The video frame of condition is decoded, and generates video breviary using decoded video frame.
Details are not described herein for the generating mode of above-mentioned video breviary.
In one embodiment, it in order to generate frame number according to the equally distributed video breviary of time shaft, can will encode
Video file afterwards is divided into multiple subfiles according to temporal information, then is ranked up for the data volume of each subfile.
Before step S402, i.e., the data volume of the described determination video frame, the method for the present embodiment proposition further include:
The video file is divided into multiple subfiles according to temporal information by S400, for each subfile, is executed
Following steps:
Determine the corresponding additional information amount of the video frame；
Determine that additional information amount meets the video frame to impose a condition；
Using the video frame for meeting and imposing a condition, video breviary is generated.
It can be seen from the above, the video breviary generation method that the embodiment of the present application fourth embodiment proposes at least have it is following excellent
Point:
The additional information of some type of video is utilized in the embodiment of the present application, corresponding attached by obtaining each video frame
Add information content, the video frame for meeting and imposing a condition is selected according to additional information amount, generates view using the video frame for the condition that meets
Frequency breviary.
It is decoded the mode with content analysis compared to for all videos frame, the embodiment of the present invention is met by selection
The video frame of setting condition can reduce the calculation amount of system；Compared to the side that interception partial video paragraph generates video breviary
Formula, the embodiment of the present invention can quickly and accurately select the video of informative by the additional information amount of acquisition video frame
Frame generates the output of video breviary, avoids mistakenly leaving out the important content of video during interception video, and then lead to not
Generate relatively comprehensive video breviary.
In particular, since live streaming class video duration is longer, carrying out full dose for video files such as live streaming class videos
Decoding Analysis will expend too many resource, therefore present invention utilizes the reference letter of the information as selecting video frame such as thumb up, comment on
Breath can generate video breviary with quick obtaining to important video frame.
Figure 14 show the flow chart of the generation method of the video breviary of another embodiment of the present invention.As shown in figure 14, originally
The generation method for the video breviary that embodiment proposes includes following steps:
S501 provides the video file including multiple video frames；
The step for it is identical as S101, details are not described herein.
S502 obtains the one or more associations information of the video frame；
In this step, wherein the related information for example may include the number of the quantizing factor of video frame, video frame
According to amount, motion vector scoring or the corresponding additional information amount of the video frame；It is uploaded when video file provides terminal 10 from video
To server 20, the related information of each video frame can be obtained from the video file uploaded.In related information,
The quantizing factor of video frame, the data volume of video frame can be obtained by mode above-mentioned, the corresponding additional letter of each video frame
Breath amount can be extracted from the specific storage region of server according to time shaft and be obtained, these additional informations, such as certain
One time point thumbed up information and comment information, can be uploaded to server 20 from viewing terminal 30, and deposit according to temporal information
Storage is in above-mentioned specific region.The data volume of video frame is for example including image data amount above-mentioned and amount of audio data and video
Both embedded caption data amounts of code stream at least one.The additional information of video frame, which includes that each video frame is corresponding, thumbs up information
And/or comment information.When subtitle is plug-in subtitle, the additional information of video frame further includes caption information.The additional letter of video
Breath amount is according to one of aforementioned various ways or the calculated additional information amount of combination.
S503 determines the video frame for meeting and imposing a condition using one or more of related informations；
In this step, these related informations extracted can be normalized according to preset rule
Processing, such as specified weight can be set, to the quantizing factor of above-mentioned video frame, the data volume of video frame, video frame pair
The additional information answered is normalized.Due to above-mentioned quantizing factor, data volume, the unit of additional information amount and numerical value model
Enclose different, can use the method for asking quotient, respectively by these information and the unity quantization factors of setting, unit data quantity,
Unit additional information is divided by, its scoring is obtained, and the modes such as addition is recycled to obtain normalized related information scoring.Specifically,
May include following sub-step:
S503a obtains the corresponding weight of each related information；
S503b obtains the scoring that the video frame is directed to each related information；
S503c, scoring and corresponding weight to the related information are weighted processing, obtain the related information and comment
Point.
In this step, the corresponding additional information amount of the data volume of video frame, video frame is bigger, then the information weight of the frame
Want degree higher, but quantizing factor is bigger, illustrates that the information content of the frame is lower, then significance level is lower.Therefore it is carrying out
When normalized, the negative value of quantizing factor can be added with other dimensions.
It is worth noting that, normalized can be accomplished in several ways, it is above-mentioned be only lift one of example into
Row explanation, is not specially limited.
It is described setting condition for example can for related information scoring be greater than a certain designated value video frame, be also possible to according to
The video frame for the sequence top N (or preceding 1/n) that related information scoring obtains.After normalized, association letter can use
Breath scoring determines the video frame for meeting and imposing a condition.
S504 generates video breviary using the video frame for meeting and imposing a condition；
In this step, it can use the video frame that above-mentioned satisfaction imposes a condition and generate video breviary.The step for
In, such as can be by integrating these video frames according to the sequence of time shaft, and be decoded, utilize decoded view
Frequency frame generates video breviary.
In one embodiment, in step S502, the step of obtaining the one or more associations information of the video frame it
Before, the method also includes:
S500a establishes related information library；
In this step, it can establish related information library, this related information library is for example including various passes shown in table 2
Join information, the corresponding additional information of data volume, video frame of quantizing factor, video frame for example including video frame, motion vector
Scoring.
S500b obtains the type of video file；
The type of video file may include: movement/game class video file, live streaming class video file, comprising audio
Video file and video file comprising subtitle.The type of video file transmission terminal 10 can be in uploaded videos from video
The label of video file setting obtains, alternatively, server can be according to the content (sound carried in the video file after coding
Pleasure, subtitle etc.) judge automatically the type of video file.
That is, in this step, the S500b, the step of obtaining the type of video file may include:
S5001b obtains the type of video file in the label subsidiary from the video file；Or
S5002b obtains the type of the video file from the video file after coding.
S500c, according to the type of video file, selection corresponds to the related information of the type from related information library
In this step, the corresponding related information of video genre of the video file is selected.When the video file
Belong to more than one video genres, the corresponding significance level information of these types can be summarized, i.e., the step may include:
Summarize the corresponding related information of type of each video file, the related information as the video file.
In one embodiment, step S502, that is, before the one or more associations information for obtaining the video frame, the side
Method further include:
The video file is divided into multiple subfiles according to temporal information by S500d, for each subfile, is executed
Following steps:
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Determine that related information scoring meets the video frame to impose a condition；
Using the video frame for meeting and imposing a condition, video breviary is generated；
Wherein, the related information includes the quantizing factor of video frame, the data volume of video frame, motion vector or the view
The corresponding additional information of frequency frame.
The generation method for the video breviary that the embodiment of the present application proposes, selects satisfaction using the related information of video frame and sets
The video frame of fixed condition generates video breviary, which includes one or more for measuring the significance level of video frame
Dimension can select video frame to generate video breviary more fully hereinafter from multiple angles, select crucial view by this mode
Frame generates more close to the video breviary of video content.
It is decoded the mode with content analysis compared to for all videos frame, the embodiment of the present invention is met by selection
The video frame of setting condition can reduce system to the decoded calculation amount of video frame full dose；Compared to interception partial video paragraph
The mode of video breviary is generated, the embodiment of the present invention can quickly and accurately select the video frame of informative, generate video
Breviary output avoids mistakenly leaving out the important content of video during interception video, and then leads to not generate relatively complete
The video breviary in face.
In addition, method provided in this embodiment can also be ranked up respectively for I frame and P frame/B frame video frame, selecting
When selecting P frame/B frame video frame, motion vector scoring is introduced, is carried out the characteristics of P frame/B frame video frame itself can be combined more smart
True selection, to generate the video breviary for being better able to reflecting video centre point.
The embodiment of the present application also proposes a kind of net cast method, is applied to video-see terminal, the net cast method
Include the following steps:
Step S1001 sends live streaming access request；
Step S1002 receives live video file, and the live video file is according to above-mentioned video breviary generation side
The video file that method generates.
In the present embodiment, as shown in connection with fig. 5, in step S1001, video-see terminal 30 can send live streaming access
Live video file and video breviary generated are then passed through transmission of network to video observing by request to server-side 20, server-side 20
See terminal 30, video-see terminal 30 receives the live video file and video breviary, and is watched.The video breviary is root
According to the video file that video breviary generation method above-mentioned generates, which can be is generated by video breviary above-mentioned
What method pre-generated, or in-time generatin during live streaming.The present invention is not specially limited.
The video breviary generation method that the embodiment of the present invention proposes provides terminal 10 in server-side 20 or video can be with
It realizes.On aforementioned base, the embodiment of the present invention also proposes a kind of generation method of video breviary, is applied to video and provides eventually
End, the generation method of the video breviary include:
S1101 provides the video file including multiple video frames；
S1102 obtains the one or more associations information of the video frame；
S1103 determines the video frame for meeting and imposing a condition using one or more of related informations；
S1104 generates video breviary using the video frame for meeting and imposing a condition；
The video breviary is uploaded to server-side by S1105.
Above-mentioned steps S1101 to step S1104 can be same or similar to step S104 with abovementioned steps S101, herein not
It repeats again.The difference is that step S1101 to step S1104 can video provide terminal (such as Fig. 5 video provide
Terminal 10) it executes.Above-mentioned multiple video files can be the video file for carrying out net cast, video contracting generated
It is slightly used for net cast or video playback, the abstract and key content of video are grasped for viewer.In specific operation, video
The video data stream that terminal 10 acquires original live video is provided, and obtains related information, the association for each video frame
Information is, for example, data volume, quantizing factor, additional information, motion vector of each video frame etc., can using these related informations
To determine the video frame for meeting and imposing a condition, video breviary is generated for providing terminal in video.In step S1105, give birth to
At video breviary from video provide terminal be uploaded to server-side 20, for video-see terminal 30 download watch.The video breviary
It can be sent to video-see terminal 30 simultaneously or successively with the video file of live streaming.
Figure 15 show a kind of block diagram of the generating means of video breviary of one embodiment of the invention.As shown in figure 15,
The generating means of the video breviary include:
Video file provides module 601, for providing the video file including multiple video frames；
Related information obtains module 602, for obtaining the one or more associations information of the video frame；
Video frame determining module 603 determines the view for meeting and imposing a condition for utilizing one or more of related informations
Frequency frame；
Video breviary generation module 604 generates video breviary for meeting the video frame to impose a condition using described.
In an embodiment of video breviary generating means of the present invention, the related information include video frame quantization because
Son, the data volume of video frame, motion vector or the corresponding additional information of the video frame.
In an embodiment of video breviary generating means of the present invention, quantizing factor, data volume and the institute of the video frame
Motion vector is stated to obtain from the video file；The corresponding additional information of the video frame includes the corresponding word of the video frame
Curtain information, thumb up information and comment information at least one.
In an embodiment of video breviary generating means of the present invention, the data volume of the video frame includes the video frame
Image data amount.
In an embodiment of video breviary generating means of the present invention, the data volume of the video frame further includes the video
At least one of the corresponding amount of audio data of frame and caption data amount.
In an embodiment of video breviary generating means of the present invention, the video frame determining module includes:
Normalized submodule is closed for multiple related informations of the video frame to be normalized
Join information scoring；
Submodule is determined, for determining that related information scoring meets the video frame to impose a condition.
In an embodiment of video breviary generating means of the present invention, described device further include:
Module is established in related information library, for establishing related information library；
Video file type obtains module, for obtaining the type of video file；
Type selecting module, for the type according to video file, selection corresponds to the type from related information library
Related information.
In an embodiment of video breviary generating means of the present invention, the video breviary is returned for net cast and video
It puts.
In an embodiment of video breviary generating means of the present invention, the video frame is the video frame of coding；
The video breviary generation module is specifically used for:
It is decoded to the video frame to impose a condition is met, generates video breviary using decoded video frame.
In an embodiment of video breviary generating means of the present invention, described impose a condition is one of following:
The related information scoring is greater than the video frame of the first designated value；
The video frame of the sequence top N of acquisition is ranked up according to related information scoring；Or
Sorting for acquisition, which is ranked up, according to related information scoring is located at the video of the preceding 1/n of the video frame sum
Frame.
In an embodiment of video breviary generating means of the present invention, described device further include:
Video file division module, for according to temporal information, the video file to be divided into multiple subfiles；
The related information obtains the one or more associations that module is used to obtain the video frame of each subfile
Information；
The video frame determining module is used to believe using the one or more associations of the video frame of each subfile
Breath determines the video frame for meeting and imposing a condition；
The video breviary generation module, for meeting the video frame to impose a condition using described, for each Ziwen
Part generates video breviary.
In an embodiment of video breviary generating means of the present invention, the related information is the data volume of video frame；
Described impose a condition includes one of following:
The data volume of video frame is greater than the video frame of the second designated value；
Data volume according to video frame carries out the video frame for the sequence top N that descending sequence obtains；Or
Data volume according to video frame carries out the preceding 1/n that the sequence that descending sequence obtains is located at the video frame sum
Video frame.
In an embodiment of video breviary generating means of the present invention, the video frame determining module is used for:
With reference to the resolution ratio of the video frame, the data volume of the video frame is determined.
In an embodiment of video breviary generating means of the present invention, the video frame determining module is used for:
With reference to the image data amount and the corresponding amount of audio data of the video frame and caption data amount of the video frame
In one of or combination, determine the data volume of the video frame.
In an embodiment of video breviary generating means of the present invention, the related information is the quantizing factor of video frame；
Described impose a condition includes one of following:
The quantizing factor of video frame is less than the video frame of third designated value；
Quantizing factor according to video frame carries out the video frame for the sequence top N that ascending sequence obtains；Or
Before the sequence that quantizing factor according to video frame carries out ascending sequence acquisition is located at the video frame sum
The video frame of 1/n.
In an embodiment of video breviary generating means of the present invention, the related information is corresponding additional from video frame
The additional information amount of information acquisition；
Described impose a condition includes one of following:
The corresponding additional information amount of video frame is greater than the video frame of the 4th designated value；
The video frame for the sequence top N that descending sequence obtains is carried out according to the corresponding additional information amount of video frame；Or
Person
It is total that the sequence that the corresponding additional information amount of foundation video frame carries out descending sequence acquisition is located at the video frame
The video frame of several preceding 1/n.
In an embodiment of video breviary generating means of the present invention, the additional information includes that the video frame is corresponding
Caption information, thumb up information and comment information at least one.
In an embodiment of video breviary generating means of the present invention, the information that thumbs up includes thumbing up content and/or point
Praise number；The comment information includes commenting on content and/or number of reviews,
The additional information amount obtains in the following way:
Number is thumbed up and number of reviews is normalized and obtains the additional information amount for video frame is corresponding；
This is normalized in the data volume of the corresponding data volume for thumbing up content of video frame and comment content
Additional information amount；Or
Using the data volume of the corresponding caption information of video frame as the additional information amount；Or
By the data volume of the corresponding caption information of video frame, thumbs up the data volume of content and comment on the data volume progress of content
Normalized obtains the additional information amount.
In an embodiment of video breviary generating means of the present invention, described device further include:
Frame determination type module, for determining that the frame type of the video frame, the frame type include I frame, P frame and B frame；
The related information obtains module and is used for:
For I-frame video frame, the corresponding related information of I-frame video frame is obtained；And
For P frame video frame and B frame video frame, P frame video frame and the corresponding related information of B frame video frame are obtained.
In an embodiment of video breviary generating means of the present invention, the corresponding related information of the I-frame video frame includes I
The quantizing factor of frame video frame, the data volume of video frame or the corresponding additional information of the I-frame video frame；
The P frame video frame and the corresponding related information of B frame video frame include the quantization of P frame video frame and B frame video frame
The factor, the data volume of video frame, motion vector or the P frame video frame and the corresponding additional information of B frame video frame.
In an embodiment of video breviary generating means of the present invention, the video frame determining module includes:
Second normalized submodule, for multiple related informations of the I-frame video frame to be normalized,
Obtain the scoring of the first related information；
Third normalized submodule, for carrying out multiple related informations of the P frame video frame and B frame video frame
Normalized obtains the scoring of the second related information；
Second video frame determines submodule, for determining that the scoring of the first related information meets the first I frame view to impose a condition
Frequency frame；
Third video frame determines submodule, for determining that the scoring of the second related information meets the second P frame view to impose a condition
Frequency frame and B frame video frame.
In an embodiment of video breviary generating means of the present invention, the P frame video frame and the corresponding pass of B frame video frame
Joining information includes motion vector, and the video frame determining module includes:
Motion vector scoring acquisition submodule, for obtaining motion vector scoring, the movement according to the motion vector
Vector scoring includes amplitude summation, amplitude average value or the energy distributions of motion vector；
Video frame acquisition submodule, for obtaining the P frame video frame for meeting preset condition according to motion vector scoring
Or B frame video frame.
In an embodiment of video breviary generating means of the present invention, the video breviary generation module includes:
Second video frame selects submodule, for meeting the first setting from the scoring of the first related information according to pre-defined rule
The I-frame video frame of selected section in the I-frame video frame of condition, and meet the second P to impose a condition from the scoring of the second related information
The P frame video frame and B frame video frame of selected section in frame video frame and B frame video frame；
Second video breviary generates submodule, for utilizing I-frame video frame, P frame video frame and the B frame video frame selected
Generate video breviary.
In an embodiment of video breviary generating means of the present invention, the data volume of the video frame is with reference to the coding
Video frame resolution ratio obtain.
In an embodiment of video breviary generating means of the present invention, the data volume of the video frame includes the figure of video frame
As data volume, and including in the corresponding amount of audio data of video frame and caption data amount one of or combination.
Figure 16 show a kind of block diagram of net cast device of another embodiment of the present invention, and as shown in figure 16, this is straight
Broadcasting device includes:
Access request sending module 701, for sending live streaming access request；
Live file receiving module 702, for receiving live video file, the live video file is according to upper video
The video file that breviary generating means generate.
Figure 17 show a kind of block diagram of the generating means of video breviary of another embodiment of the present invention, the video breviary
Generating means be applied to video provide terminal, comprising:
Second video file provides module 801, for providing the video file including multiple video frames；
Second related information obtains module 802, for obtaining the one or more associations information of the video frame；
Second video frame determining module 803 determines for utilizing one or more of related informations and meets setting condition
Video frame；
Second video breviary generation module 804 generates video contracting for meeting the video frame to impose a condition using described
Slightly；
Uploading module 805, for the video breviary to be uploaded to server-side.
In an embodiment of video breviary generating means of the present invention, the video breviary is returned for net cast and video
It puts.
In an embodiment of video breviary generating means of the present invention, the related information include video frame quantization because
Son, the data volume of video frame, motion vector or the corresponding additional information of the video frame.
For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
Embodiment of the disclosure can be implemented as using any suitable hardware, firmware, software, or and any combination thereof into
The system of the desired configuration of row.Figure 18, which is schematically shown, can be used for realizing showing for each embodiment described in the disclosure
Example property system (or device) 900.
For one embodiment, Figure 18 shows exemplary system 900, the system have one or more processors 902,
It is coupled to the system control module (chipset) 904 of at least one of (one or more) processor 902, is coupled to and be
The system storage 906 for control module 904 of uniting is coupled to the nonvolatile memory (NVM) of system control module 904/deposit
Storage equipment 908 is coupled to one or more input-output apparatus 910 of system control module 904, and is coupled to and is
The network interface 912 for control module 906 of uniting.
Processor 902 may include one or more single or multiple core processors, processor 902 may include general processor or
Any combination of application specific processor (such as graphics processor, application processor, Baseband processor etc.).In some embodiments,
System 900 can be as the browser described in the embodiment of the present application.
In some embodiments, system 900 may include with instruction one or more computer-readable mediums (for example,
System storage 906 or NVM/ store equipment 908) and mutually merge with the one or more computer-readable medium and be configured as
Execute instruction the one or more processors 902 to realize module thereby executing movement described in the disclosure.
For one embodiment, system control module 404 may include any suitable interface controller, with to (one or
It is multiple) at least one of processor 902 and/or any suitable equipment or component that communicate with system control module 904 mentions
For any suitable interface.
System control module 904 may include Memory Controller module, to provide interface to system storage 906.Storage
Device controller module can be hardware module, software module and/or firmware module.
System storage 906 can be used for for example, load of system 900 and storing data and/or instruction.For a reality
Example is applied, system storage 906 may include any suitable volatile memory, for example, DRAM appropriate.In some embodiments
In, system storage 906 may include four Synchronous Dynamic Random Access Memory of Double Data Rate type (DDR4SDRAM).
For one embodiment, system control module 904 may include one or more i/o controllers, with to
NVM/ stores equipment 908 and (one or more) input-output apparatus 910 provides interface.
For example, NVM/ storage equipment 908 can be used for storing data and/or instruction.NVM/ storage equipment 908 may include appointing
It anticipates nonvolatile memory appropriate (for example, flash memory) and/or to may include that any suitable (one or more) is non-volatile deposit
Equipment is stored up (for example, one or more hard disk drives (HDD), one or more CD (CD) drivers and/or one or more
Digital versatile disc (DVD) driver).
NVM/ storage equipment 908 may include a part for the equipment being physically mounted on as system 900
Storage resource or its can by the equipment access without a part as the equipment.For example, NVM/ storage equipment 908 can
It is accessed by network via (one or more) input-output apparatus 910.
(one or more) input-output apparatus 910 can be provided for system 900 interface with other any equipment appropriate
Communication, input-output apparatus 910 may include communication component, audio component, sensor module etc..Network interface 912 can be
System 400 provides interfaces with by one or more network communications, system 900 can according to one or more wireless network standards and/
Or arbitrary standards in agreement and/or agreement are carried out wireless communication with the one or more components of wireless network, such as are accessed
Wireless network based on communication standard, such as WiFi, 2G or 3G or their combination carry out wireless communication.
For one embodiment, at least one of (one or more) processor 902 can be with system control module 904
The logic of one or more controllers (for example, Memory Controller module) is packaged together.For one embodiment, (one
Or multiple) at least one of processor 902 can be encapsulated in the logic of one or more controllers of system control module 404
Together to form system in package (SiP).For one embodiment, at least one of (one or more) processor 902 can
It is integrated on same mold with the logic of one or more controllers of system control module 904.For one embodiment, (one
It is a or multiple) at least one of processor 902 can be integrated with the logic of one or more controllers of system control module 904
To form system on chip (SoC) on same mold.
In various embodiments, system 900 can be, but not limited to be: browser, work station, desk-top calculating equipment or movement
It calculates equipment (for example, lap-top computing devices, handheld computing device, tablet computer, net book etc.).In various embodiments,
System 900 can have more or fewer components and/or different frameworks.For example, in some embodiments, system 900 includes
One or more video cameras, keyboard, liquid crystal display (LCD) screen (including touch screen displays), nonvolatile memory port,
Mutiple antennas, graphic chips, specific integrated circuit (ASIC) and loudspeaker.
Wherein, if display includes touch panel, display screen may be implemented as touch screen displays, be used by oneself with receiving
The input signal at family.Touch panel includes one or more touch sensors to sense the hand on touch, slide, and touch panel
Gesture.The touch sensor can not only sense the boundary of a touch or slide action, but also detect and the touch or sliding
Operate relevant duration and pressure.
The embodiment of the present application also provides a kind of non-volatile readable storage medium, be stored in the storage medium one or
Multiple modules (programs) when the one or more module is used in terminal device, can make the terminal device execute
The instruction (instructions) of various method steps in the embodiment of the present application.
Provide a kind of computer equipment in one example, including memory, processor and storage are on a memory simultaneously
The computer program that can be run on a processor, which is characterized in that the processor is realized such as when executing the computer program
The method of the embodiment of the present application.
A kind of computer readable storage medium is additionally provided in one example, is stored thereon with computer program, it is special
Sign is, one or more methods such as the embodiment of the present application are realized when which is executed by processor.
Although some embodiments are various substitutions, and/or equivalent implementation for the purpose of illustrating and describing
Scheme calculates to reach same purpose and implement the realization for exemplifying and describing, and does not depart from the practical range of the application.This Shen
It please be intended to cover any modification or variation of the embodiment being discussed herein.It is, therefore, apparent that embodiment described herein only by right
It is required that being limited with their equivalent.
The foregoing is merely the preferred embodiments of the embodiment of the present application, all not to limit the embodiment of the present application
Within the spirit and principle of the embodiment of the present application, any modification, equivalent substitution, improvement and etc. done should be included in this Shen
Please embodiment protection within the scope of.
Claims (34)
1. a kind of generation method of video breviary, which is characterized in that the described method includes:
Video file including multiple video frames is provided；
Obtain the one or more associations information of the video frame；
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Using the video frame for meeting and imposing a condition, video breviary is generated.
2. the method as described in claim 1, which is characterized in that the related information includes the quantizing factor of video frame, video
Data volume, motion vector or the corresponding additional information of the video frame of frame.
3. method according to claim 2, which is characterized in that quantizing factor, data volume and the movement of the video frame
Vector is obtained from the video file；The corresponding additional information of the video frame includes the corresponding subtitle letter of the video frame
Cease, thumb up information and comment information at least one.
4. method according to claim 2, which is characterized in that the data volume of the video frame includes the picture number of the video frame
According to amount.
5. method as claimed in claim 3, which is characterized in that the data volume of the video frame further includes that the video frame is corresponding
Amount of audio data and caption data amount at least one.
6. the method as described in claim 1, which is characterized in that utilize one or more of related informations, determine that satisfaction is set
The video frame of fixed condition includes:
Multiple related informations of the video frame are normalized, related information scoring is obtained；
Determine that related information scoring meets the video frame to impose a condition.
7. the method as described in claim 1, which is characterized in that the one or more associations information for obtaining the video frame it
Before, the method also includes:
Establish related information library；
Obtain the type of video file；
According to the type of video file, selection corresponds to the related information of the type from related information library.
8. the method as described in claim 1, which is characterized in that the video breviary is used for net cast and video playback.
9. the method as described in claim 1, which is characterized in that
The video frame is the video frame of coding；
It is described to utilize the video frame for meeting setting condition, generation video breviary, comprising:
It is decoded to the video frame to impose a condition is met, generates video breviary using decoded video frame.
10. method as claimed in claim 5, which is characterized in that described impose a condition is one of following:
The related information scoring is greater than the video frame of the first designated value；
The video frame of the sequence top N of acquisition is ranked up according to related information scoring；Or
Sorting for acquisition, which is ranked up, according to related information scoring is located at the video frame of the preceding 1/n of the video frame sum.
11. method according to claim 8, which is characterized in that
Before the one or more associations information for obtaining the video frame, the method also includes:
According to temporal information, the video file is divided into multiple subfiles；
For each subfile, following steps are executed:
Obtain the one or more associations information of the video frame；
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Using the video frame for meeting and imposing a condition, video breviary is generated.
12. the method as described in claim 1, which is characterized in that the related information is the data volume of video frame；
Described impose a condition includes one of following:
The data volume of video frame is greater than the video frame of the second designated value；
Data volume according to video frame carries out the video frame for the sequence top N that descending sequence obtains；Or
Data volume according to video frame carries out the view that the sequence that descending sequence obtains is located at the preceding 1/n of the video frame sum
Frequency frame.
13. method as claimed in claim 11, which is characterized in that it is described to utilize one or more of related informations, it determines
Meeting the video frame to impose a condition includes:
With reference to the resolution ratio of the video frame, the data volume of the video frame is determined.
14. method as claimed in claim 11, which is characterized in that it is described to utilize one or more of related informations, it determines
Meeting the video frame to impose a condition includes:
In image data amount and the corresponding amount of audio data of the video frame and caption data amount with reference to the video frame
One of them is combined, and determines the data volume of the video frame.
15. the method as described in claim 1, which is characterized in that the related information is the quantizing factor of video frame；
Described impose a condition includes one of following:
The quantizing factor of video frame is less than the video frame of third designated value；
Quantizing factor according to video frame carries out the video frame for the sequence top N that ascending sequence obtains；Or
The sequence that quantizing factor according to video frame carries out ascending sequence acquisition is located at the preceding 1/n's of the video frame sum
Video frame.
16. the method as described in claim 1, which is characterized in that the related information is from the corresponding additional information of video frame
The additional information amount of acquisition；
Described impose a condition includes one of following:
The corresponding additional information amount of video frame is greater than the video frame of the 4th designated value；
The video frame for the sequence top N that descending sequence obtains is carried out according to the corresponding additional information amount of video frame；Or
The sequence that descending sequence obtains, which is carried out, according to the corresponding additional information amount of video frame is located at the video frame sum
The video frame of preceding 1/n.
17. method as claimed in claim 15, which is characterized in that the additional information includes the corresponding subtitle of the video frame
Information, thumb up information and comment information at least one.
18. the method described in claim 16, which is characterized in that the information that thumbs up is including thumbing up content and/or thumbing up number
Mesh；The comment information includes commenting on content and/or number of reviews,
The additional information amount obtains in the following way:
Number is thumbed up and number of reviews is normalized and obtains the additional information amount for video frame is corresponding；
Acquisition is normalized in the data volume of the corresponding data volume for thumbing up content of video frame and comment content, and this is added
Information content；Or
Using the data volume of the corresponding caption information of video frame as the additional information amount；Or
By the data volume of the corresponding caption information of video frame, thumbs up the data volume of content and comment on the data volume progress normalizing of content
Change processing, obtains the additional information amount.
19. the method as described in claim 1, which is characterized in that after the offer is including the video file of multiple video frames,
Further include:
Determine that the frame type of the video frame, the frame type include I frame, P frame and B frame；
The one or more associations information for obtaining the video frame, comprising:
For I-frame video frame, the corresponding related information of I-frame video frame is obtained；And
For P frame video frame and B frame video frame, P frame video frame and the corresponding related information of B frame video frame are obtained.
20. method as claimed in claim 18, which is characterized in that
The corresponding related information of the I-frame video frame includes the quantizing factor of I-frame video frame, the data volume of video frame or the I
The corresponding additional information of frame video frame；
The P frame video frame and the corresponding related information of B frame video frame include P frame video frame and B frame video frame quantizing factor,
Data volume, motion vector or the P frame video frame and the corresponding additional information of B frame video frame of video frame.
21. method as claimed in claim 19, which is characterized in that
It is described to utilize one or more of related informations, determine that the video frame for meeting and imposing a condition includes:
Multiple related informations of the I-frame video frame are normalized, the scoring of the first related information is obtained；
Multiple related informations of the P frame video frame and B frame video frame are normalized, the second related information is obtained and comments
Point；
Determine that the scoring of the first related information meets the first I-frame video frame to impose a condition；
Determine that the scoring of the second related information meets the second P frame video frame and B frame video frame to impose a condition.
22. method as claimed in claim 18, which is characterized in that the P frame video frame is corresponding with B frame video frame to be associated with letter
Breath includes motion vector, described to utilize one or more of related informations, determines the video frame for meeting and imposing a condition, comprising:
Obtain motion vector scoring according to the motion vector, motion vector scoring include motion vector amplitude summation,
Amplitude average value or energy distributions；
The P frame video frame or B frame video frame for meeting preset condition are obtained according to motion vector scoring.
23. method as claimed in claim 20, which is characterized in that it is described using the video frame for meeting and imposing a condition, it is raw
At video breviary, comprising:
According to pre-defined rule, the I frame of selected section in the I-frame video frame that first imposes a condition is met from the scoring of the first related information
Video frame, and meet selected section in the P frame video frame and B frame video frame that second imposes a condition from the scoring of the second related information
P frame video frame and B frame video frame；
Video breviary is generated using I-frame video frame, P frame video frame and the B frame video frame selected.
24. method as claimed in claim 19, which is characterized in that the data volume of the video frame is the view with reference to the coding
What the resolution ratio of frequency frame obtained.
25. method as claimed in claim 19, which is characterized in that the data volume of the video frame includes the picture number of video frame
According to amount, and including in the corresponding amount of audio data of video frame and caption data amount one of or combination.
26. a kind of net cast method characterized by comprising
Send live streaming access request；
Live video file is received, the live video file is according to claim 1 any one of -24 video breviary generation
The video file that method generates.
27. a kind of generation method of video breviary is applied to video and provides terminal, which is characterized in that the life of the video breviary
Include: at method
Video file including multiple video frames is provided；
Obtain the one or more associations information of the video frame；
Using one or more of related informations, the video frame for meeting and imposing a condition is determined；
Using the video frame for meeting and imposing a condition, video breviary is generated；
The video breviary is uploaded to server-side.
28. method as claimed in claim 26, which is characterized in that the video breviary is used for net cast and video playback.
29. method as claimed in claim 26, which is characterized in that the related information includes the quantizing factor of video frame, view
Data volume, motion vector or the corresponding additional information of the video frame of frequency frame.
30. a kind of generating means of video breviary characterized by comprising
Video file provides module, for providing the video file including multiple video frames；
Related information obtains module, for obtaining the one or more associations information of the video frame；
Video frame determining module determines the video frame for meeting and imposing a condition for utilizing one or more of related informations；
Video breviary generation module generates video breviary for meeting the video frame to impose a condition using described.
31. a kind of net cast device characterized by comprising
Access request sending module, for sending live streaming access request；
Live file receiving module, for receiving live video file, the live video file is according to claim 29-40
Any one of video breviary generating means generate video file.
32. a kind of generating means of video breviary are applied to video and provide terminal, which is characterized in that the life of the video breviary
Include: at device
Second video file provides module, for providing the video file including multiple video frames；
Second related information obtains module, for obtaining the one or more associations information of the video frame；
Second video frame determining module determines the video for meeting and imposing a condition for utilizing one or more of related informations
Frame；
Second video breviary generation module generates video breviary for meeting the video frame to impose a condition using described；
Uploading module, for the video breviary to be uploaded to server-side.
33. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor is realized when executing the computer program such as any one of claim 1-28
The method.
34. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The method as described in any one of claim 1-28 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710909929.XA CN109587581A (en) | 2017-09-29 | 2017-09-29 | Video breviary generation method and video breviary generating means |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710909929.XA CN109587581A (en) | 2017-09-29 | 2017-09-29 | Video breviary generation method and video breviary generating means |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109587581A true CN109587581A (en) | 2019-04-05 |
Family
ID=65919088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710909929.XA Pending CN109587581A (en) | 2017-09-29 | 2017-09-29 | Video breviary generation method and video breviary generating means |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109587581A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110213614A (en) * | 2019-05-08 | 2019-09-06 | 北京字节跳动网络技术有限公司 | The method and apparatus of key frame are extracted from video file |
| CN110366002A (en) * | 2019-06-14 | 2019-10-22 | 北京字节跳动网络技术有限公司 | Video file synthetic method, system, medium and electronic equipment |
| CN110996183A (en) * | 2019-07-12 | 2020-04-10 | 北京达佳互联信息技术有限公司 | Video abstract generation method, device, terminal and storage medium |
| CN113365104A (en) * | 2021-06-04 | 2021-09-07 | 中国建设银行股份有限公司 | Video concentration method and device |
| CN113596355A (en) * | 2020-04-14 | 2021-11-02 | 瑞昱半导体股份有限公司 | Caption image generating device and method |
| CN113923504A (en) * | 2021-12-02 | 2022-01-11 | 阿里巴巴达摩院(杭州)科技有限公司 | Video preview animation generation method and device |
| CN114120166A (en) * | 2021-10-14 | 2022-03-01 | 北京百度网讯科技有限公司 | Video question and answer method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101346719A (en) * | 2005-12-23 | 2009-01-14 | 高通股份有限公司 | Selecting key frames from video frames |
| CN101453649A (en) * | 2008-12-30 | 2009-06-10 | 浙江大学 | Key frame extracting method for compression domain video stream |
| CN105992074A (en) * | 2015-01-27 | 2016-10-05 | 张军 | Key frame and color histogram-based network video format file abstract extraction method |
| CN106506448A (en) * | 2016-09-26 | 2017-03-15 | 北京小米移动软件有限公司 | Live display packing, device and terminal |
| CN106792150A (en) * | 2016-12-20 | 2017-05-31 | 深圳市茁壮网络股份有限公司 | A kind of poster generation method and device |
-
2017
- 2017-09-29 CN CN201710909929.XA patent/CN109587581A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101346719A (en) * | 2005-12-23 | 2009-01-14 | 高通股份有限公司 | Selecting key frames from video frames |
| CN101453649A (en) * | 2008-12-30 | 2009-06-10 | 浙江大学 | Key frame extracting method for compression domain video stream |
| CN105992074A (en) * | 2015-01-27 | 2016-10-05 | 张军 | Key frame and color histogram-based network video format file abstract extraction method |
| CN106506448A (en) * | 2016-09-26 | 2017-03-15 | 北京小米移动软件有限公司 | Live display packing, device and terminal |
| CN106792150A (en) * | 2016-12-20 | 2017-05-31 | 深圳市茁壮网络股份有限公司 | A kind of poster generation method and device |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110213614B (en) * | 2019-05-08 | 2021-11-02 | 北京字节跳动网络技术有限公司 | Method and device for extracting key frame from video file |
| CN110213614A (en) * | 2019-05-08 | 2019-09-06 | 北京字节跳动网络技术有限公司 | The method and apparatus of key frame are extracted from video file |
| CN110366002B (en) * | 2019-06-14 | 2022-03-11 | 北京字节跳动网络技术有限公司 | Video file synthesis method, system, medium and electronic device |
| CN110366002A (en) * | 2019-06-14 | 2019-10-22 | 北京字节跳动网络技术有限公司 | Video file synthetic method, system, medium and electronic equipment |
| CN110996183A (en) * | 2019-07-12 | 2020-04-10 | 北京达佳互联信息技术有限公司 | Video abstract generation method, device, terminal and storage medium |
| CN113596355B (en) * | 2020-04-14 | 2024-04-16 | 瑞昱半导体股份有限公司 | Subtitle image generating apparatus and method |
| CN113596355A (en) * | 2020-04-14 | 2021-11-02 | 瑞昱半导体股份有限公司 | Caption image generating device and method |
| CN113365104B (en) * | 2021-06-04 | 2022-09-09 | 中国建设银行股份有限公司 | Video concentration method and device |
| CN113365104A (en) * | 2021-06-04 | 2021-09-07 | 中国建设银行股份有限公司 | Video concentration method and device |
| CN114120166A (en) * | 2021-10-14 | 2022-03-01 | 北京百度网讯科技有限公司 | Video question and answer method and device, electronic equipment and storage medium |
| CN114120166B (en) * | 2021-10-14 | 2023-09-22 | 北京百度网讯科技有限公司 | Video question-answering method and device, electronic equipment and storage medium |
| US12327084B2 (en) | 2021-10-14 | 2025-06-10 | Beijing Baidu Netcom Science Technology Co., Ltd. | Video question answering method, electronic device and storage medium |
| CN113923504A (en) * | 2021-12-02 | 2022-01-11 | 阿里巴巴达摩院(杭州)科技有限公司 | Video preview animation generation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109587581A (en) | Video breviary generation method and video breviary generating means | |
| US20220321968A1 (en) | System and method of displaying content based on locational activity | |
| US20180220194A1 (en) | System and method of displaying content and related social media data | |
| JP6441302B2 (en) | Generating video with multiple viewpoints | |
| CN103841389B (en) | A kind of video broadcasting method and player | |
| US20150156557A1 (en) | Display apparatus, method of displaying image thereof, and computer-readable recording medium | |
| CN109819272B (en) | Video sending method, video sending device, computer readable storage medium and electronic equipment | |
| CN114040230A (en) | Video code rate determining method and device, electronic equipment and storage medium thereof | |
| KR101905638B1 (en) | Device and method for playing video | |
| KR20130112162A (en) | Video display terminal and method for displaying a plurality of video thumbnail simultaneously | |
| CN103402091B (en) | Cloud desktop picture classification nad code method | |
| CN104956683A (en) | Adaptive noise reduction engine for streaming video | |
| CN101998051A (en) | Image display control device, imaging device provided with the image display control device, image processing device | |
| US20190379917A1 (en) | Image distribution method and image display method | |
| CN101668206A (en) | Multichannel video decoding display method and system based on H.264 | |
| CN109874024A (en) | A kind of barrage processing method, system and storage medium based on dynamic video poster | |
| CN116916071B (en) | Video screen display method, system, device, electronic device and storage medium | |
| CN117649624A (en) | Video image quality evaluation method, device, computer equipment and storage medium | |
| KR20160131827A (en) | System for cloud streaming service, method of image cloud streaming service using alpha level of color bit and apparatus for the same | |
| CN117768650A (en) | Image block chroma prediction method and device, electronic equipment and storage medium | |
| WO2023130896A1 (en) | Media data processing method and apparatus, computer device and storage medium | |
| CN113453041A (en) | Data processing method, device, equipment and storage medium | |
| CN110677728A (en) | Method, device and equipment for playing video and storage medium | |
| US9454992B2 (en) | Method and system to play linear video in variable time frames | |
| CN117579843B (en) | Video coding processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190405 |