CN116634192A - A method for automatic video editing - Google Patents

Abstract

The invention provides an automatic video editing method which is characterized by comprising the following steps: a step of receiving an original video; a step of intensity recognition of the original video; a step of performing highlight content recognition on the original video; selecting music according to the intensity recognition result; and fusing the highlight content and the music. The video automatic editing method can automatically complete daily editing of the video, and greatly improves the working efficiency of video publishers.

Description

A method for automatic video editing

technical field

The method of the invention relates to the field of video editing, in particular to a method for automatically editing videos.

Background technique

With the development of grid technology, applications based on video content are becoming more and more widely used. Most of the original videos that have been shot cannot be used directly. Especially in the case of short video applications, the duration of each video is strictly limited, and In the video to be processed, what the publisher needs to focus on, or what the viewer actually cares about is only a small part of the video (called the highlight moment), so the video publisher needs to edit, make music, and add filters before publishing Wait for the editing work. At present, the above-mentioned editing work can only be done manually, which leads to low efficiency.

Contents of the invention

In view of the shortcomings of the prior art described above, the present invention provides an automatic video editing method, which is characterized in that it includes: a step of receiving an original video; a step of identifying the intensity of the original video; performing a highlight on the original video The steps of content recognition; the step of selecting music according to the intensity recognition result; the step of fusing the highlight content with the music.

Preferably, in the above automatic video editing method, the step of receiving the original video further includes a step of detecting whether the original video meets the constraints.

Preferably, in the above automatic video editing method, the step of identifying the highlight content of the original video further includes: identifying a list of all segments with highlight characteristics according to the vertical category to which the original video belongs; The segment list is filtered to obtain the highlight content list.

Preferably, in the above automatic video editing method, in the step of filtering the list of clips to obtain the list of clips, only the clips in the list of clips whose duration is greater than or equal to the preset threshold T ₁ are kept, and the highlight content The total duration of the list is not greater than the preset second threshold T ₂ .

Preferably, in the above automatic video editing method, if the length of the segment list is greater than 1 and the length of the highlight content list is less than 1, the segments in the first segment list are concatenated as the highlight content list.

Preferably, in the above automatic video editing method, the step of selecting music according to the strength recognition result includes: marking candidate music; selecting music according to the video strength and the mark.

Preferably, in the above automatic video editing method, the step of fusing the highlight content and the music includes: defining a slot for the music; filling the highlight content into the slot.

Preferably, in the above automatic video editing method, the step of marking the candidate music includes marking two dimensions of music speed and emotion.

Preferably, in the above-mentioned automatic video editing method, the step of selecting music according to the video strength and the mark includes the steps of returning a recommended music list to the user according to the video strength, and determining the music according to the user selection.

Preferably, in the above-mentioned automatic video editing method, in the step of defining a slot for the music, if the music is music without lyrics, the interval between every 3 downbeats is defined as a slot; if the If the music department has lyric music, the music interval corresponding to each lyric is defined as a slot.

Preferably, in the above-mentioned automatic video editing method, if the music has lyric music, the prelude, interlude and ending in the music are processed in the following manner: when the duration of the paragraph is less than or equal to the preset threshold t, directly Merge the paragraphs into adjacent slots; when the paragraphs are greater than the threshold t and smaller than the step size p, then set the paragraphs as a single slot, and if the duration of the paragraphs is greater than or equal to t and less than 2p, then add A draw-in slot, take the strong beat point near the equally divided point of the paragraph as the demarcation point, and so on; wherein, the paragraph refers to the prelude, interlude and coda in the music; the threshold t is 3 seconds; the value of the step p is 9 seconds.

Preferably, in the above-mentioned automatic video editing method, when there are multiple original videos, the step of filling the highlight content into the card slot includes: when "current highlight content segment duration-1s-current card slot duration" When <1s, cut the current highlight content segment, first cut off the end 1s of the current highlight content segment, then cut the current highlight content segment to the point close to the beat, and set the starting point of the next slot Move to this place; when "1s ≤ current highlight content segment length – 1s - current card slot duration < 7s", the current highlight content segment is not processed, and the next segment of highlight content segment is read and filled; when "7s ≤ When the duration of the current highlight content segment – 1s – the duration of the current slot”, use the current highlight content segment to continue filling the next slot, and so on, and the next highlight content segment fills the starting point = the current highlight content segment is at the end point +3s; if the current highlight content segment fails to fill the next card slot, cut the current highlight content segment, first cut off the end 1s of the current highlight content segment, then cut the current highlight content segment to the nearest beat point, and The starting point of the next slot is moved there; the condition for filling the next slot is: "current material duration - 1s - current slot duration - 3s - next slot duration ≥ 1s".

Preferably, the above-mentioned automatic video editing method further includes the step of adding a transition effect to the video at the slot joint.

Preferably, the above-mentioned automatic video editing method also includes the step of adding a filter to the highlight content

Description of drawings

FIG. 1 is a flowchart of the automatic video editing method of the present invention.

Detailed ways

Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention.

The flow chart of the automatic video editing method of the present invention is shown in FIG. 1 . Step S1 represents receiving the original video, and this step usually includes the step of checking whether the video meets the constraints, which include the duration, frame rate, bit rate, and resolution of the video. If the constraints are not met, a prompt can be given to the user, or preprocessing can be performed. For example, if the time is too long, the user can be prompted; if the resolution is too high, compression can be performed.

Step S2 represents performing intensity recognition on the video. The intensity of the video is an index that indicates the degree of change in the video signal. The greater the movement range of the characters in the video, the faster the displacement of the subject object, and the more severe the change of the scene or lens, it means that there are more mutation components in the video signal. , the greater the intensity, otherwise the lower the intensity of the video. Among them, machine learning technologies such as neural networks can be directly used to identify the characters, subjects, and their range of motion and displacement speed in the video, which is not the invention point of the present invention, so it will not be described in detail. In this example, the video intensity is schematically divided into three levels: high, medium, and low. The intensity of a video reflects the emotion and style of the video to a large extent, so the grading of the intensity of the video can also be regarded as a classification of the style of the video.

Step S3 represents the identification of highlight content in the video. The highlight content in the video is usually the content that the publisher wants to highlight to the viewers, and it is also the content that the viewers care most about. The highlight recognition strategy of the present invention includes:

S31: According to the vertical category to which the video belongs, a list A of all clips with highlight characteristics is identified. Among them, the highlight features can be defined based on prior knowledge. For example, for a vertical category of basketball videos, the highlight features include a complete action of shooting and the complete trajectory of the ball; for a vertical category of beauty makeup videos, the highlight features include makeup The change of makeup before and after makeup, as well as the close-up and interaction of cosmetics.

S32: Preliminarily screen the fragments obtained in S31 to obtain a list B of fragments with highlight features (ie, highlight content fragments). One of the screening criteria may be duration. In this example, a minimum duration threshold T ₁ =5 seconds is defined, that is, only fragments with a duration greater than or equal to T ₁ are retained. Preferably, the maximum duration of the list list B does not exceed the threshold T ₂ =30 seconds. More preferably, if the length of list A is greater than or equal to 1, and the length of list B is less than 1, the highlight content fragments are spliced and filled with the fragments in list A.

Step S4 represents the step of selecting the soundtrack, that is, selecting the matching music for the video material. The music can be selected based on the speed, style and video intensity of the music. In this case, the method is as follows:

· S41: mark the candidate music. The marks can be divided into multiple dimensions. For example, the speed of music can be marked as fast, medium, and slow; the mood of music can be marked as positive, medium, and negative.

· S42: Select music according to video intensity and music tags. For example, a video with high video strength matches music with fast speed and positive emotion. If the result of the match is multiple pieces of music, one is randomly determined. However, in order to improve the user satisfaction of the final film, in this example, an interactive interface is provided for the user, that is, a music recommendation list is obtained and returned to the user through multiple cross-matching methods based on the video intensity and music tags, and then the user finally selects Matching music. It is not difficult to understand that: the more dimensions of music are marked, the greater the amount of calculation (time-consuming) matching will be, and the higher the accuracy of matching will be. Therefore, it is necessary to find a balance in practice.

Step S5 represents the step of fusing the selected music with the video material. In this example, the specific steps of fusion include:

· S51: Define card slots for alternative music. Music can be divided into music with lyrics and music without lyrics. For music without lyrics, the interval between every 3 downbeats is defined as a slot. For music with lyrics, the music interval corresponding to each lyrics is used as a card slot. For the prelude (that is, the interval between the beginning of the music and the beginning of the lyrics), the interlude (the interval between the lyrics and the lyrics) and the ending (the interval between the end of the lyrics and the end of the music), due to the common characteristics ——It belongs to pure music without lyrics, so it can be processed directly as music without lyrics, or it can be processed as follows. For the convenience of description, the prelude, interlude and ending are collectively referred to as paragraphs below: when the length of a paragraph is less than or equal to the preset When the threshold t (in this example, the value is 3 seconds), the paragraphs are directly merged into the adjacent card slot; when the paragraph is greater than the threshold t but smaller than the step size p (in this example, it is set to 9 seconds), the paragraphs are set separately as a card Slot, if the length of the paragraph is greater than or equal to t+p but less than t+2p, then add a slot, take the strong beat point near the even point of the paragraph as the dividing point, and so on.

· S52: filling the highlight content clips into the music card slot. When the number of elements in the highlight content fragment list is greater than 1, it can be filled in the following way:

When the "current highlight content segment duration - 1s - current slot duration"< 1s, the current highlight content segment is trimmed, first cutting off the end 1s of the highlight content segment, and then cutting the highlight content segment to the adjacent beat point, and The starting point of the next card slot is moved there;

When "1s ≤ the duration of the current highlight content segment – 1s - the duration of the current slot <7s", the current highlight content segment is not processed, and the next segment of highlight content segment is read and filled;

When "7s ≤ duration of the current highlight content segment – 1s - duration of the current slot", use the current highlight content segment to continue filling the next slot, and so on, and the filling starting point of the next highlight content segment = the current highlight content segment at End point + 3s; if the current highlight content segment fails to fill the next slot, trim the current highlight content segment, first cut off the end 1s of the current highlight content segment, and then cut the current highlight content segment to the nearest beat point, and move the starting point of the next card slot there.

Among them, the condition for filling the next slot is: "current material duration - 1s - current slot duration - 3s - next slot duration ≥ 1s".

Preferably, in order to improve user experience, the present invention may further include step S6: adding a transition effect to the video. The point of adding the transition effect is selected at the joint of the card slot. Among them, the transition refers to the animation effect when the video camera is switched, such as dissolving, flashing black, advancing and so on.

More preferably, the present invention further includes step S7: adding a filter effect to the video. You can select a unified filter for all highlight content according to the video intensity, or add different filters for different highlight content segments according to the vertical category of the highlight content segment.

In the foregoing embodiments, a single video is processed as an example, but the method of the present invention is also applicable to multi-video situations, that is, multiple original videos are received at the same time and output as a complete video. For the case of processing multiple videos, it is enough to perform intensity recognition and highlight recognition on each video separately, and the method is similar to that of processing a single video, and will not be described in detail.

Since the dynamic video is obtained by continuously playing a series of static videos with slight differences, the method of the present invention is also applicable to splicing multiple static pictures into video material. In this scenario, the steps of matching music can be optimized as follows: no need to distinguish whether there are lyrics in the music, only divide according to the music beat, and divide the interval between every two strong beats into a slot; no need to distinguish music Prelude, Interlude, Ending.

To sum up, the automatic video editing method of the present invention can automatically complete daily video editing and greatly improve the work efficiency of video publishers.

The above-mentioned embodiments only illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Any person familiar with this technology can modify or change the above embodiments, but as long as these changes are still within the scope of the claims, they all belong to the idea of using the present invention.

Claims (14)

1. A method for automatically editing video, comprising:

a step of receiving an original video;

a step of intensity recognition of the original video;

a step of performing highlight content recognition on the original video;

selecting music according to the intensity recognition result;

and fusing the highlight content and the music.

2. The method according to claim 1, wherein the step of receiving the original video further comprises the step of detecting whether the original video meets a constraint condition.

3. The method for automatically editing video according to claim 1, wherein the step of recognizing the highlight content of the original video further comprises:

identifying all fragment lists with highlight characteristics according to the vertical class to which the original video belongs;

and screening the fragment list to obtain the highlight content list.

4. The method for automatically editing video according to claim 3, wherein in the step of filtering the clip list to obtain the clip list, only a duration of at least a preset threshold T in the clip list is maintained ₁ And the total duration of the highlight content list is not more than a preset second threshold T ₂ 。

5. The method according to claim 4, wherein if the length of the clip list is greater than 1 and the length of the highlight content list is less than 1, clips in the first clip list are spliced as the highlight content list.

6. The automatic video editing method according to claim 1, wherein the selecting music according to the intensity recognition result comprises:

marking the alternative music;

music is selected based on the video intensity and the indicia.

7. The automatic video editing method according to claim 1, wherein the step of fusing the highlight content and the music comprises:

defining a card slot for the music;

and filling the high-light content into the clamping groove.

8. The method of automatic video editing according to claim 6, wherein the step of marking the candidate music includes marking both a tempo and an emotion dimension of the music.

9. The method of automatic video editing according to claim 6, wherein the step of selecting music according to the video intensity and the mark comprises the step of returning a list of recommended music to the user according to the video intensity and determining the music according to the user selection.

10. The method of automatic video editing according to claim 7, wherein in the step of defining a slot for the music, if the music is lyrics-free music, an interval between every 3 beats is defined as a slot; if the music is lyric music, the music interval corresponding to each lyric is defined as a clamping groove.

11. The video automatic editing method according to claim 10, wherein if the musical system has lyric music, the pre-music, the inter-music and the end-music in the music are processed as follows: when the paragraph duration is less than or equal to a preset threshold t, directly merging the paragraphs into adjacent clamping grooves; when the paragraph is larger than a threshold t and smaller than a step length p, the paragraph is independently set into a clamping groove, if the time length of the paragraph is larger than or equal to t and smaller than 2p, a clamping groove is additionally arranged, a strong beat point near the paragraph equal point is taken as a demarcation point, and the like;

wherein the paragraphs refer to the pre-forms, the interludes and the tail forms in the music; the threshold t is 3 seconds; the step p takes a value of 9 seconds.

12. The method of automatic video editing according to claim 7, wherein when the original video is plural, the step of filling the highlight content into the card slot comprises:

when the duration of the current highlight content segment is-1 s-the duration of the current clamping groove is <1s, cutting the current highlight content segment, firstly cutting the end 1s of the current highlight content segment, then cutting the current highlight content segment to a point close to the beat point, and moving the starting point of the next clamping groove to the point;

when the duration of the current highlight content segment is less than or equal to 1s and less than or equal to the duration of the current clamping groove is less than 7s, the current highlight content segment is not processed, and the next highlight content segment is read and filled;

when the time length of the current highlight content segment is less than or equal to 7s and less than or equal to the time length of the current clamping groove, namely-1 s, the current highlight content segment is used for continuously filling the next clamping groove, and the like, and the next highlight content segment is filled with a starting point=the current highlight content segment is at an ending point +3s; if the current highlight content segment cannot fill the next clamping groove, cutting the current highlight content segment, firstly cutting the tail 1s of the current highlight content segment, cutting the current highlight content segment to the adjacent beat point, and moving the starting point of the next clamping groove to the position;

the condition for filling the next clamping groove is as follows: the time length of the current material is-1 s, the time length of the current clamping groove is-3 s, and the time length of the next clamping groove is more than or equal to 1 s.

13. The method of automatic video editing according to claim 7, further comprising the step of adding a transition effect to the video at the card slot splice.

14. The method of automatic video editing according to claim 1, further comprising the step of adding a filter to said highlight content.