CN107968921B - Video generation method, device and electronic device - Google Patents

Abstract

The present invention proposes a video generation method, device and electronic equipment. The method includes: obtaining the selected audio and the standard actions corresponding to each time node in the audio, playing the audio, and collecting each video frame during the audio playing process. , when the audio is played to each time node, the corresponding standard action is displayed. In each video frame, the image of the standard action displayed synchronously during collection is added. At the end of the audio playback, based on the audio and each video frame, a target video. During the process of playing the selected audio, video frames are collected, images of displayed standard actions are added to the video collection interface, and at the end of the video, the target video is generated, which solves the problem in the existing technology of user substitution when playing games. The sense is not strong and the corresponding video cannot be automatically generated.

Description

Video generation method, device and electronic device

technical field

The present invention relates to the technical field of mobile terminals, and in particular, to a video generation method, apparatus and electronic device.

Background technique

With the development of video capture technology and applications, users can simultaneously play games on the video capture interface of the application, realizing the interaction between the user and the application.

At present, for example, in the existing application, a dance game can be played at the same time as the live broadcast. The user operates according to the instructions of the arrow floating across the screen. After the action is matched, the cartoon character on the screen will dance. This kind of user controls the cartoon image. In this way, the user does not have a strong sense of substitution, and the video of synchronous game play cannot be recorded, and the user's sense of participation is low.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

To this end, the first object of the present invention is to propose a video generation method, so as to synchronously capture video picture frames in the process of playing audio, add images of standard actions displayed in the video capture interface, and when the video ends, generate The target video solves the technical problem in the prior art that when a user plays a somatosensory game, the sense of substitution is not strong, and a corresponding video cannot be automatically generated.

The second object of the present invention is to provide a video generating apparatus.

The third object of the present invention is to provide an electronic device.

A fourth object of the present invention is to propose a non-transitory computer-readable storage medium.

In order to achieve the above purpose, the embodiment of the first aspect of the present invention proposes a video generation method, including:

Obtain the selected audio, and the standard actions corresponding to each time node in the audio;

Playing the audio, and collecting each video frame in the process of playing the audio;

When the audio is played to each time node, the corresponding standard action is displayed;

In each video frame, add the image of the standard action displayed synchronously during acquisition;

When the audio playback ends, a target video is generated according to the audio and each video frame.

Optionally, as a first possible implementation manner of the first aspect, the method further includes:

Perform human motion recognition on the video frame captured synchronously when the standard motion is displayed;

generating action evaluation information of the human action according to the degree of difference between the standard action and the human action;

The action evaluation information of the human action is added to the synchronously collected video frame.

Optionally, as a second possible implementation manner of the first aspect, the motion evaluation information includes: a human motion score and an animation effect corresponding to an interval to which the human motion score belongs.

Optionally, as a third possible implementation manner of the first aspect, before generating the action evaluation information of the human action according to the degree of difference between the standard action and the human action, the method further includes:

It is determined that the standard motion matches the human motion.

Optionally, as a fourth possible implementation manner of the first aspect, after determining that the standard action matches the human body action, the method further includes:

End the presentation process of the standard action.

Optionally, as a fifth possible implementation manner of the first aspect, when displaying the standard action, a plurality of video picture frames are collected synchronously, and each video picture frame has a corresponding piece of action evaluation information. The action evaluation information of the human action is added to the synchronously collected video frame, including:

Screen the generated multiple action evaluation information, and retain the action evaluation information with the highest evaluation;

Adding the action evaluation information with the highest evaluation to at least one video picture frame in the plurality of video picture frames collected synchronously; wherein, the at least one video picture frame displays the action evaluation with the highest evaluation Information corresponds to human actions.

Optionally, as a sixth possible implementation manner of the first aspect, after the audio playback ends, the method further includes:

According to the action evaluation information of each human action, the total score interface is generated;

The total score interface is displayed.

Optionally, as a seventh possible implementation manner of the first aspect, after generating the target video, the method further includes:

The target video is published.

In a video generation method according to an embodiment of the present invention, the selected audio and the standard actions corresponding to each time node in the audio are acquired, the audio is played, and each video frame is collected during the audio playback, and the audio is played to each At the time node, the corresponding standard action is displayed. In each video frame, the image of the standard action displayed synchronously during acquisition is added. When the audio playback ends, the target video is generated according to the audio and each video frame. In the process of playing the selected audio, the video frame is collected, the image of the displayed standard action is added to the video collection interface, and the target video is generated when the video ends, which solves the problem of the user playing a somatosensory game in the prior art. The sense of substitution is not strong, and the technical problem that the corresponding video cannot be automatically generated.

In order to achieve the above-mentioned purpose, the embodiment of the second aspect of the present invention provides a video generation device, including:

an acquisition module for acquiring the selected audio and the standard actions corresponding to each time node in the audio;

A playing module, used for playing the audio, and collecting each video frame in the process of playing the audio;

a first display module, configured to display the corresponding standard action when the audio is played to each time node;

The first adding module is used to add, in each video frame, images of standard actions that are displayed synchronously during collection;

The first generating module is configured to generate a target video according to the audio and each video frame when the audio playback ends.

Optionally, as a first possible implementation manner of the second aspect, the apparatus further includes:

A recognition module, configured to perform human motion recognition on the video frame captured synchronously when the standard motion is displayed;

a generating module, configured to generate action evaluation information of the human action according to the degree of difference between the standard action and the human action;

The second adding module is configured to add the action evaluation information of the human action to the video frame captured synchronously.

Optionally, as a second possible implementation manner of the second aspect, the motion evaluation information includes: a human motion score and an animation effect corresponding to an interval to which the human motion score belongs.

Optionally, as a third possible implementation manner of the second aspect, the apparatus further includes:

A determination module, configured to determine that the standard action matches the human action.

Optionally, as a fourth possible implementation manner of the second aspect, the apparatus further includes:

The ending module is used to end the presentation process of the standard action.

Optionally, as a fifth possible implementation manner of the second aspect, when the identification module displays the standard action, a plurality of video picture frames are synchronously collected, and the generation module generates for each video picture frame. A corresponding action evaluation information, the second adding module includes:

a screening unit, used for screening the generated multiple action evaluation information, and retaining the action evaluation information with the highest evaluation;

The adding unit is configured to add the action evaluation information with the highest evaluation to at least one video picture frame in the plurality of video picture frames collected synchronously; wherein, the at least one video picture frame displays the The action evaluation information with the highest evaluation corresponds to the human action.

Optionally, as a sixth possible implementation manner of the second aspect, the apparatus further includes:

The second display module is configured to generate a total score interface according to the action evaluation information of each human action; and display the total score interface.

Optionally, as a seventh possible implementation manner of the second aspect, the apparatus further includes:

A publishing module, configured to publish the target video.

In a video generation device according to an embodiment of the present invention, the acquisition module is used to acquire the selected audio and the standard actions corresponding to each time node in the audio, and the playback module is used to play the audio, and collect each video image during the audio playback process. frame, the first display module is used to display the corresponding standard action when the audio is played to each time node, the first addition module is used to add the image of the standard action displayed synchronously during acquisition in each video frame, the first The generating module is used for generating the target video according to the audio and each video frame when the audio playback ends. In the process of playing the selected audio, the video frame is collected, the image of the displayed standard action is added to the video collection interface, and the target video is generated when the video ends, which solves the problem of the user playing a somatosensory game in the prior art. The sense of substitution is not strong, and the technical problem that the corresponding video cannot be automatically generated.

In order to achieve the above object, the third aspect of the present invention provides an electronic device, including: a casing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is arranged inside the space enclosed by the casing, and the processor and The memory is arranged on the circuit board; the power supply circuit is used to supply power to each circuit or device of the above-mentioned electronic equipment; the memory is used to store the executable program code; the processor runs and executes the executable program code by reading the executable program code stored in the memory A program corresponding to the program code is used to execute the video generation method described in the first aspect.

In order to achieve the above object, the embodiment of the fourth aspect of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the video generation method as described in the first aspect is implemented .

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic flowchart of a method for generating a video according to an embodiment of the present invention;

2 is a schematic flowchart of another video generation method provided by an embodiment of the present invention;

3 is a schematic diagram of the ratio of limbs to height in the human anatomy provided by the present embodiment;

4A is a schematic structural diagram of a standard action provided by an embodiment of the present invention;

4B is a schematic structural diagram of an actual action provided by an embodiment of the present invention;

5 is a schematic diagram showing a total score interface display provided by an embodiment of the present invention;

6 is a schematic diagram of a picture effect in a target video provided by the present embodiment;

7 is a schematic structural diagram of a video generation apparatus according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of another video generation apparatus provided by an embodiment of the present invention; and

FIG. 9 is a schematic structural diagram of an embodiment of an electronic device of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

The following describes the video generation method, apparatus, and electronic device according to the embodiments of the present invention with reference to the accompanying drawings.

The electronic device in this embodiment may specifically be a mobile phone. Those skilled in the art may know that the electronic device may also be other mobile terminals, and the solutions provided in this embodiment can be referred to to identify human body movements.

In the following embodiments, the video generation method is explained by taking the electronic device as a mobile phone as an example.

FIG. 1 is a schematic flowchart of a video generation method provided by an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

Step 101: Acquire the selected audio and standard actions corresponding to each time node in the audio.

Specifically, open the mobile phone application and enter the video capture interface. As a possible implementation method, you can enter the audio selection interface before entering the video capture interface. You can use the drop-down menu to let the user click the favorite audio, audio Each time node has a corresponding standard action, select the audio through the confirm button, and enter the video capture interface.

Step 102 , play audio, and collect each video frame during the process of playing the audio.

Specifically, the video capture interface is entered, the audio starts to be played, and at the same time, each video frame is captured during the audio playback.

In step 103, when the audio is played to each time node, the corresponding standard action is displayed.

Specifically, when the audio is played to each time node, the standard action of the corresponding time node will be displayed, and the user will perform the same action according to the displayed standard action, and at the same time, the video frame of the user's human action will be collected.

Step 104 , in each video frame, add an image of the standard action displayed synchronously during the capture.

Specifically, when the corresponding standard action is displayed at each time node of the audio playback, the video frame of the synchronously collected human action contains multiple frames. As a possible implementation, the time point at which the standard action is displayed can be taken as the time. As a benchmark, N frames of images containing human actions are collected backwards. For the value of N, those skilled in the art can determine the value of N according to the actual application, and add images of standard actions displayed synchronously during collection to the N frames.

Step 105, when the audio playback ends, generate a target video according to the audio and each video frame.

Specifically, the target video is generated from the audio and each video frame captured during the audio playback.

As a possible implementation method, in the generated target video, the villain using the character image is scrolled and displayed on the screen to display standard actions. When the actions performed by the user match the standard actions, the villain disappears, and at the same time, Displays the action's score and the corresponding animation effect.

After the target video is generated, the user can publish the target video to the community or to social networking sites, which improves the user's experience.

In a video generation method according to an embodiment of the present invention, the selected audio and the standard actions corresponding to each time node in the audio are acquired, the audio is played, and each video frame is collected during the audio playback, and the audio is played to each At the time node, the corresponding standard action is displayed. In each video frame, the image of the standard action displayed synchronously during acquisition is added. When the audio playback ends, the target video is generated according to the audio and each video frame. In the process of playing the selected audio, the video frame is collected, the image of the displayed standard action is added to the video collection interface, and the target video is generated when the video ends, which solves the problem of the user playing a somatosensory game in the prior art. The sense of substitution is not strong, and the technical problem that the corresponding video cannot be automatically generated.

In order to clearly illustrate the previous embodiment, this embodiment provides another video generation method. FIG. 2 is a schematic flowchart of another video generation method provided by the embodiment of the present invention. As shown in FIG. 2 , the method includes:

Step 201: Acquire the selected audio and standard actions corresponding to each time node in the audio.

Step 202, playing audio, and starting to collect each video frame.

Reference may be made to steps 101 to 102 in the previous embodiment, which will not be repeated here.

Step 203: When the audio is played to each time node, the corresponding standard action is displayed along the preset track.

Specifically, when the corresponding time node is played, that is, the corresponding standard action is displayed on the video capture interface of the camera device. As a possible implementation, the corresponding standard action can be displayed in the form of a floating frame in the video capture interface. As another possible implementation, the corresponding standard actions can be scrolled and displayed in the form of a bullet screen in the video capture interface.

Step 204 , when displaying the corresponding standard actions, perform human motion recognition on the synchronously collected video frame.

As a possible implementation manner of performing human motion recognition on the video frame, each joint of the human body can be identified in the video frame, and the human motion can be recognized according to the identified human joints.

There are many ways to recognize the joints of the human body. As a possible implementation way of this embodiment, according to the depth information carried in the video frame of the human action, the human body and the background in each frame can be separated, and then the human body can be identified. Every joint of the human body. In order to make the video picture frame carry the depth information, the camera device for collecting the human body video picture frame may be a camera device that can collect depth information, and through the acquired depth information, identify the human body part in the image, such as dual cameras, depth camera ( Red-Green-Blue Depth)RGBD, which can obtain depth information while imaging, and can also obtain depth information through structured light/TOF lenses, which are not listed here. Specifically, as a possible implementation, according to the acquired depth information, combined with face recognition technology to identify the face area and position information in the image, so as to obtain the pixels contained in the face area and the corresponding depth information, Calculate the average value of the depth information corresponding to the face pixels. Further, since the human body and the human face are basically on the same imaging plane, the pixel points with the difference between the average value of the depth information corresponding to the pixel points in the human face and the average value of the depth information within the threshold range are identified as the human body, and the human body and the human face can be identified. Human body contour, so as to determine the depth information and position information of each pixel in the human body and the contour, and then the human body and the background can be separated. Furthermore, in order to facilitate the identification of the joints in the human body and eliminate the interference of the background, the image can be binarized so that the pixel value of the background is 0 and the pixel value of the human body is 1.

Further, according to the identified position information of the face and the human body, and according to the proportional relationship between the limbs and the height in human anatomy, the position information of each joint of the human body can be calculated. For example, FIG. 3 is a schematic diagram of the ratio of limbs to height in human anatomy provided in this embodiment, and FIG. 3 lists the proportional relationship of each joint in the limbs, and video frames can be determined according to the position information of the face and the human body The two-dimensional coordinate information (x, y) of the human neck joint can be obtained by obtaining the position information of the human neck joint. As shown in Figure 3, the difference between the height of the shoulder joint and the height of the neck joint is fixed. According to the coordinate information of the neck joint and the difference, the row where the shoulder joint is located can be determined. The value is 0, and the pixel value of the human body is 1. Therefore, the point with the corresponding pixel value of 1 at the left and right most edges of the row is the point corresponding to the shoulder joint, so as to determine the two-dimensional coordinate information of the left shoulder joint (x1, y1 ), the two-dimensional coordinate information (x2, y2) of the right shoulder joint.

According to the determined position information of the left shoulder joint, according to the standard distance between the left shoulder joint and the left elbow joint in Figure 3, draw a circle with the standard distance as the diameter. Since the pixel value of the background part is 0, when the left and When the pixel point on the right is located, the two-dimensional coordinate information (x3, y3) of the left elbow joint can be determined.

Similarly, the two-dimensional coordinate information of other joints of the human body can be further identified and determined. Each joint of the human body at least includes: neck joint, left shoulder joint, right shoulder joint, left elbow joint, right elbow joint, left wrist joint, right wrist joint, left knee joint Joints, left ankle joints, right knee joints, right ankle joints, etc. Since there are many joints, they are not listed here. The principles of the methods for identifying and determining the two-dimensional coordinates of other joints are the same, and will not be repeated here.

Step 205 , determine whether the standard action matches the human body action, if so, go to Step 206 , if not, go to Step 207 .

Optionally, connect two adjacent joints in each joint of the human body according to the identified human joints, obtain the connection line between the two adjacent joints, and calculate the distance between the connection line between the two adjacent joints and the preset reference direction. According to the difference between the actual angle and the standard angle, determine whether the human action matches the standard action. The standard angle is the connection between the two adjacent joints and the reference direction when the standard action is performed. The angle between, specifically, for each connection between two adjacent joints, calculate the difference between the corresponding standard angle and the actual angle, if the connection between each adjacent two joints is calculated If the difference is within the error range, it is determined that the human action matches the standard action; if there is at least one connection between two adjacent joints and the calculated difference is not within the error range, it is determined that the human action does not match the standard action.

For example, taking the standard action displayed at a time node of the audio and the actual human action performed by the user imitating the standard action as an example, the method for matching the standard action with the human action will be described. FIG. 4A is provided by an embodiment of the present invention. A schematic diagram of the structure of a standard action, the figure shows the standard action displayed at a certain time node, and the relevant joints involved in the standard action, each joint includes: left wrist joint, right wrist joint, left elbow joint, right elbow joint, left shoulder joint, Right shoulder joint, a total of 6 joints.

FIG. 4B is a schematic structural diagram of an actual action provided by an embodiment of the present invention. FIG. 4B shows an actual action performed by a user when the standard action in FIG. 4A is displayed. Identify each joint of the human body according to the collected human motion, obtain the connection line between two adjacent joints, and obtain the connection line 1 between the right wrist joint and the right elbow joint in FIG. 4B, and the connection line between the right elbow joint and the right shoulder joint. Line 2, line 3 between the right shoulder joint and the left shoulder joint, line 4 between the left shoulder joint and the left elbow joint, and line 5 between the left elbow joint and the right wrist joint, each of The action corresponding to the connection is called the decomposition action of the actual action made by the user, and all the decomposition actions constitute the actual action.

If the preset reference direction is the horizontal direction of the screen, the angle between the No. 1 connection and the horizontal direction of the screen is calculated to be 35 degrees, the angle between the No. 2 connection and the horizontal direction of the screen is 0 degrees, and the angle between the No. 3 connection and the horizontal direction of the screen is 0 degrees. The angle between the line and the horizontal direction of the screen is 0 degrees, the angle between the No. 4 connection and the horizontal direction of the screen is 0 degrees, and the angle between the No. 5 connection and the horizontal direction of the screen is 130 degrees.

Further, it is judged whether the movement of the human body matches the standard movement. Take the connection line 1 between the right wrist joint and the right elbow joint in FIG. 4B as an example to illustrate, the connection line 1 corresponds to the standard angle in FIG. 4A and is 45 degrees. In FIG. 4B The actual angle measured by the actual action is 35 degrees, and the difference is 10 degrees. According to the preset difference threshold, for example, 15 degrees, and the difference of 10 degrees is less than 15 degrees, the decomposition action corresponding to connection 1 can be determined. The decomposition actions in the standard actions are matched. Further, determine whether the decomposition actions corresponding to Link 2, Link 3, Link 4, and Link 5 match the decomposition actions in the standard actions. If all the decomposition actions match the standard actions If there is a match, the human body action matches the standard action. If any decomposition action does not match the corresponding decomposition action in the standard action, the human body action does not match the standard action.

Step 206, if there is a match, end the display process of the standard action.

Specifically, if it is determined that the human action and the standard action are matched, the picture used for displaying the standard action is removed from the screen, so as to end the process of displaying the standard action.

Step 207, if it does not match, continue to display the standard action until the standard action moves to the end position of the preset track.

Specifically, if the human action does not match the standard action, the standard action will continue to be displayed, and when the standard action moves to the end position of the preset trajectory, the picture used to display the standard action will be removed from the screen to end the standard action 's display.

Step 208 , generating and displaying the action evaluation information of the human action according to the degree of difference between the standard action and the human action.

Optionally, according to the recognized human action and the recognized corresponding joint when displaying the standard action, for each connection between two adjacent joints, the score of the connection is determined according to the corresponding difference and error range. coefficient, according to the scoring coefficient of the connection and the corresponding score of the connection, the actual score of the connection is generated, and then the evaluation information of the human action is generated. Specifically, according to the formula p=1-[2Δ/(a-b)], it can be The scoring coefficient p is calculated, where b is the lower limit of the error range, a is the upper limit of the error range, and Δ is the difference. Taking the connection line 1 between the right wrist joint and the right elbow joint in Figure 4B as an example, the corresponding difference is 10 degrees, for example, the upper limit of the error range of the difference is positive 50 degrees, and the lower limit of the error range is negative 50 degrees, according to the formula P=1-[2×10/(50-(-50))]=0.8, that is Link 1 has a scoring coefficient of 0.8. Similarly, the scoring coefficient of connection 2 can be calculated as 1, the scoring coefficient of connection 3 is 1, the scoring coefficient of connection 4 is 1, and the scoring coefficient of connection 5 is 0.9.

Further, the actual score of the connection is obtained by calculation, and then the evaluation information of the human action is generated. The evaluation information of the human action includes the human action score and the animation effect corresponding to the interval to which the human action score belongs. For example, the full score of the human action score is 100. , the animation effect corresponding to the interval above 90 points is "perfect prefect" flashing display; 80-90 points, the animation effect corresponding to the interval to which it belongs is "very good" flashing display. Specifically, the evaluation information of the connection includes the decomposition action score, and the decomposition action score is the product of the score coefficient of the connection and the score corresponding to the connection. As shown in FIG. 4B , the action has a total score of 100 points, and there are 5 items in total. Decomposing actions, the full score of each decomposing action is 20 points, and multiplying the full 20 points of the decomposing action corresponding to connection 1 by the corresponding scoring coefficient 0.8, the score of the decomposing action corresponding to connection 1 is obtained. 16 points, and thus the evaluation information of the link 1 is generated. In the same way, the score of the decomposition action corresponding to the link 2 included in the evaluation information of the link 2 is 20 points, and the score of the decomposition action corresponding to the link 3 included in the evaluation information of the link 3 is 20 points, The score of the decomposition action corresponding to Link 4 included in the evaluation information of Link 4 is 20 points, and the score of the decomposition action corresponding to Link 5 included in the evaluation information of Link 5 is 18 points. The score of the corresponding decomposition action is summed, that is, the score of the human action is 94 points, and the animation effect corresponding to the interval to which the score belongs is "prefect" flashing display, so as to obtain the evaluation information of the human action, and to The action evaluation information of the human action is displayed, and the user can understand the standard degree of the current action through the display, and motivate the user to make the action accurate.

It should be noted that, in the aforementioned step 207, there is no human action matching the standard action, and the score in the action evaluation information of the human action is 0 points. It is displayed in the steps, and only the better action evaluation information is displayed.

Further, by processing other multiple video frames of human actions according to the above method, evaluation information of human actions in different video frames can be obtained respectively.

Step 209, when the audio playback ends, generate a total score interface according to the action evaluation information of each human action, and display the total score interface.

Specifically, at the end of the audio playback, according to the action evaluation information of each human action, the human action scores contained therein are accumulated and summed to generate a total score score, and the animation effect corresponding to the interval to which the total score belongs, to generate a total score. Score interface, and display the total score interface.

As a possible implementation, the weight corresponding to each standard action in the audio can be preset. After determining the action evaluation information of each human action, the human action score of each human action can be multiplied by the corresponding weight. , to obtain the product value, so that the total score value is obtained by accumulating the product value, and then the corresponding score level is determined according to the interval to which the total score value belongs.

For example, when there are 100 time nodes in the audio, that is, 100 standard actions, the weight corresponding to each standard action can be set. For example, the corresponding weight of each standard action can be set to 0.01. When determining each human action After the action evaluation information is obtained, the product value can be obtained by multiplying the human action score of each human action by the corresponding weight, so that the total score value can be obtained by accumulating the product value. If the total score obtained is 87, it can be known that the interval to which it belongs is [80, 90). Therefore, the score level can be "C".

FIG. 5 is a schematic diagram showing a total score interface provided by an embodiment of the present invention. As shown in FIG. 5 , after the video ends, the user obtains a total score of 88 points, and the score level is “C”.

Step 210: Add the action evaluation information of the human action and the image of the standard action to the collected video frame, and generate a target video according to the audio and each video frame.

On the one hand, action evaluation information needs to be added to the video frame. Specifically, when displaying standard actions, multiple video frames are collected synchronously, and each video frame has a corresponding action evaluation information. The action evaluation information of the action is added to the video frame collected synchronously. As a possible implementation method, the generated action evaluation information is screened, the action evaluation information with the highest evaluation is retained, and the action evaluation information with the highest evaluation is saved. , which is added to at least one video frame of the multiple video frames collected synchronously. In the video frame with action evaluation information added, the human action corresponding to the action evaluation information with the highest evaluation should be displayed.

It should be noted that, in the aforementioned step 207, there is no human action matching the standard action, the score in the action evaluation information of the human action is 0, and the action evaluation information of the human action with a score of 0 is not added to the standard action. In the captured video picture frame, the picture can be made cleaner and the visual disturbance to the viewer is reduced.

On the other hand, it is necessary to add the image of the standard action to the video frame. Specifically, when displaying the standard action, there are multiple video frames that are collected synchronously, and each video frame has a corresponding action evaluation information. As a possible implementation manner, the method of comparing the score included in the action evaluation information of each video frame with the threshold score is used to determine the video frame to which the standard action needs to be added. If the comparison result is higher than the threshold score, in the corresponding video frame, the image of the standard action displayed synchronously when the video frame is collected is added.

FIG. 6 is a schematic diagram of the picture effect in the target video provided by the present embodiment. As shown in FIG. 6 , the user performs human body actions synchronously according to the standard actions performed by the villain scrolled and displayed on the screen. When the action is matched, the villain disappears. At the same time, according to the action evaluation information of the human action, the corresponding score and the animation effect corresponding to the score interval are displayed. The score is 94 points, and the corresponding animation effect is prefect. The standard actions are displayed in the image of the villain, which makes the user feel more involved.

Step 211 , when an operation of clicking the publishing control in the total score interface is detected, publish the target video.

Specifically, after the target video is generated, if the user clicks the publishing control in the total score interface, the target video can be published to the community, or the video can be shared, which improves the user's participation and experience.

It should be noted that, in this embodiment, step 209 and step 210 may be executed simultaneously. For example, the target video may be generated in the background while the total score page shown in FIG. 5 is displayed to improve the generation efficiency of the target video, thereby preventing users from long The process of waiting for time.

In a video generation method according to an embodiment of the present invention, the selected audio and the standard actions corresponding to each time node in the audio are acquired, the audio is played, and each video frame is collected during the audio playback, and the audio is played to each At the time node, the corresponding standard action is displayed. In each video frame, the image of the standard action displayed synchronously during acquisition is added. When the audio playback ends, the target video is generated according to the audio and each video frame. In the process of playing the selected audio, the video frame is collected, the image of the displayed standard action is added to the video collection interface, and the target video is generated when the video ends, which solves the problem of the user playing a somatosensory game in the prior art. The sense of substitution is not strong, and the technical problem that the corresponding video cannot be automatically generated.

In order to realize the above embodiments, the present invention also provides a video generation apparatus.

FIG. 7 is a schematic structural diagram of a video generating apparatus according to an embodiment of the present invention.

As shown in FIG. 7 , the apparatus includes: an acquiring module 71 , a playing module 72 , a first displaying module 73 , a first adding module 74 and a first generating module 75 .

The obtaining module 71 is configured to obtain the selected audio and the standard actions corresponding to each time node in the audio.

The playing module 72 is used for playing audio, and collecting each video frame during the process of playing the audio.

The first display module 73 is configured to display the corresponding standard action when the audio is played to each time node.

The first adding module 74 is configured to add, in each video frame, images of standard actions that are displayed synchronously during capture.

The first generating module 75 is configured to generate a target video according to the audio and each video frame when the audio playback ends.

It should be noted that the foregoing explanations of the method embodiment are also applicable to the apparatus of this embodiment, and details are not repeated here.

In a video generation device according to an embodiment of the present invention, the acquisition module is used to acquire the selected audio and the standard actions corresponding to each time node in the audio, and the playback module is used to play the audio, and collect each video image during the audio playback process. frame, the first display module is used to display the corresponding standard action when the audio is played to each time node, the first addition module is used to add the image of the standard action displayed synchronously during acquisition in each video frame, the first The generating module is used for generating the target video according to the audio and each video frame when the audio playback ends. In the process of playing the selected audio, the video frame is collected, the image of the displayed standard action is added to the video collection interface, and the target video is generated when the video ends, which solves the problem of the user playing a somatosensory game in the prior art. The sense of substitution is not strong, and the technical problem that the corresponding video cannot be automatically generated.

Based on the foregoing embodiment, the embodiment of the present invention also provides another possible implementation manner of the video generation apparatus. FIG. 8 is a schematic structural diagram of another video generation apparatus provided by the embodiment of the present invention. Above, the apparatus further includes: an identification module 76 , a determination module 77 , an end module 78 , a second generation module 79 , a second addition module 80 , a second display module 81 and a release module 82 .

The recognition module 76 is configured to perform human motion recognition on the video frame captured synchronously when displaying the standard motion.

The determining module 77 is configured to determine that the standard action matches the human action.

The end module 78 is used to end the presentation process of the standard action.

The second generating module 79 is configured to generate action evaluation information of the human action according to the degree of difference between the standard action and the human action.

The second adding module 80 is configured to add the action evaluation information of the human action to the synchronously collected video frame.

The second display module 81 is used for generating a total score interface according to the action evaluation information of each human action; and displaying the total score interface.

The publishing module 82 is used to publish the target video.

As a possible implementation manner, the second adding module 80 may further include: a screening unit 801 and an adding unit 802 .

The screening unit 801 is configured to screen the plurality of generated action evaluation information, and retain the action evaluation information with the highest evaluation.

The adding unit 802 is configured to add the action evaluation information with the highest evaluation to at least one video picture frame in the plurality of video picture frames collected synchronously, wherein at least one video picture frame displays the action evaluation information corresponding to the highest evaluation human action.

It should be noted that the foregoing explanations of the method embodiment are also applicable to the apparatus of this embodiment, and details are not repeated here.

In a video generation device according to an embodiment of the present invention, the acquisition module is used to acquire the selected audio and the standard actions corresponding to each time node in the audio, and the playback module is used to play the audio, and collect each video image during the audio playback process. frame, the first display module is used to display the corresponding standard action when the audio is played to each time node, the first addition module is used to add the image of the standard action displayed synchronously during acquisition in each video frame, the first The generating module is used for generating the target video according to the audio and each video frame when the audio playback ends. In the process of playing the selected audio, the video frame is collected, the image of the displayed standard action is added to the video collection interface, and the target video is generated when the video ends, which solves the problem of the user playing a somatosensory game in the prior art. The sense of substitution is not strong, and the technical problem that the corresponding video cannot be automatically generated.

An embodiment of the present invention further provides an electronic device, and the electronic device includes the device described in any of the foregoing embodiments.

FIG. 9 is a schematic structural diagram of an embodiment of an electronic device of the present invention, which can implement the flow of the embodiment shown in FIGS. 1-2 of the present invention. As shown in FIG. 9 , the above-mentioned electronic device may include: a casing 91 , a processor 92 , and a memory 93. The circuit board 94 and the power supply circuit 95, wherein the circuit board 94 is arranged in the space enclosed by the housing 91, and the processor 92 and the memory 93 are arranged on the circuit board 94; the power supply circuit 95 is used for the above-mentioned electronic equipment. Each circuit or device is powered; the memory 93 is used to store the executable program code; the processor 92 runs the program corresponding to the executable program code by reading the executable program code stored in the memory 93, so as to execute any of the foregoing embodiments The described video generation method.

For the specific execution process of the above steps by the processor 92 and the further steps performed by the processor 92 by running executable program codes, reference may be made to the descriptions of the embodiments shown in FIGS.

This electronic device exists in many forms, including but not limited to:

(1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions, and its main goal is to provide voice and data communication. Such terminals include: smart phones (eg iPhone), multimedia phones, functional phones, and low-end phones.

(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has the characteristics of mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, such as iPads.

(3) Portable entertainment equipment: This type of equipment can display and play multimedia content. Such devices include: audio and video players (eg iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

(4) Server: A device that provides computing services. The composition of the server includes a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but due to the need to provide highly reliable services, the processing capacity, stability , reliability, security, scalability, manageability and other aspects of high requirements.

(5) Other electronic devices with data interaction function.

To implement the above embodiments, the embodiments of the present invention further provide a non-transitory computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the video generation method described in the above method embodiments.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the invention includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present invention belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical scope disclosed by the present invention can easily think of changes or substitutions. All should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (14)

1. A video generation method, comprising the steps of:

acquiring a selected audio and standard actions corresponding to time nodes in the audio;

playing the audio, and collecting each video picture frame in the process of playing the audio;

displaying corresponding standard actions when the audio is played to each time node;

carrying out human body action recognition on the video picture frames synchronously acquired when the standard action is displayed;

determining whether the standard action is matched with the human body action, and if so, finishing the display process of the standard action; if not, continuing to display the standard action until the standard action moves to the end position of the preset track;

adding images of standard actions synchronously displayed during acquisition into each video picture frame;

and when the audio playing is finished, generating a target video according to the audio and each video picture frame.

2. The video generation method of claim 1, further comprising:

generating action evaluation information of the human body action according to the difference degree between the standard action and the human body action;

and adding the motion evaluation information of the human body motion to the synchronously acquired video picture frame.

3. The video generation method according to claim 2, wherein the action evaluation information includes: and the human body action score and the animation effect corresponding to the interval to which the human body action score belongs.

4. The video generation method according to claim 2, wherein when the standard motion is exhibited, a plurality of video frame frames are synchronously captured, each video frame has a corresponding piece of motion evaluation information, and the adding the motion evaluation information of the human motion to the synchronously captured video frame frames includes:

screening the generated plurality of action evaluation information, and reserving action evaluation information with the highest evaluation;

adding the action evaluation information with the highest evaluation to at least one video picture frame in the plurality of synchronously acquired video picture frames; and the at least one video picture frame shows that the action evaluation information with the highest evaluation corresponds to the human action.

5. The video generation method according to claim 2, further comprising, after the audio playback ends:

generating a total score interface according to the action evaluation information of each personal action;

and displaying the total score interface.

6. The video generation method according to any one of claims 1 to 5, further comprising, after the generating the target video:

and issuing the target video.

7. A video generation apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring the selected audio and the standard actions corresponding to the time nodes in the audio;

the playing module is used for playing the audio and collecting each video picture frame in the process of playing the audio;

the first display module is used for displaying the corresponding standard action when the audio is played to each time node;

the identification module is used for identifying human body actions aiming at video picture frames synchronously acquired when the standard actions are displayed;

the determining module is used for determining that the standard action is matched with the human body action; the ending module is used for ending the display process of the standard action if the standard action is matched with the standard action; if not, continuing to display the standard action until the standard action moves to the end position of the preset track;

the first adding module is used for adding images of standard actions synchronously displayed during acquisition in each video picture frame;

and the first generation module is used for generating a target video according to the audio and each video picture frame when the audio playing is finished.

8. The video generation apparatus of claim 7, wherein the apparatus further comprises:

the generating module is used for generating action evaluation information of the human body action according to the difference degree between the standard action and the human body action;

and the second adding module is used for adding the motion evaluation information of the human body motion to the synchronously acquired video picture frame.

9. The video generation apparatus according to claim 8, wherein the motion estimation information includes: and the human body action score and the animation effect corresponding to the interval to which the human body action score belongs.

10. The video generation apparatus according to claim 8, wherein the identification module synchronizes a plurality of captured video frame when the standard motion is exhibited, the generation module generates a corresponding piece of motion evaluation information for each video frame, and the second adding module includes:

a screening unit for screening the generated plurality of action evaluation information and reserving the action evaluation information with the highest evaluation;

the adding unit is used for adding the action evaluation information with the highest evaluation to at least one video picture frame in the plurality of synchronously acquired video picture frames; and the at least one video picture frame shows that the action evaluation information with the highest evaluation corresponds to the human action.

11. The video generation apparatus of claim 8, wherein the apparatus further comprises:

the second display module is used for generating a total score interface according to the action evaluation information of each personal action; and displaying the total score interface.

12. A video generation apparatus according to any one of claims 7 to 11, wherein the apparatus further comprises:

and the publishing module is used for publishing the target video.

13. An electronic device, comprising: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space enclosed by the shell, and the processor and the memory are arranged on the circuit board; a power supply circuit for supplying power to each circuit or device of the electronic apparatus; the memory is used for storing executable program codes; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the video generation method of any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the video generation method of any of claims 1-6.

Images