CN114630057B - Method and device for determining special effect video, electronic equipment and storage medium - Google Patents

Abstract

Embodiments of the present disclosure provide a method, device, electronic device, and storage medium for determining a special effect video. The method includes: in response to a special effect triggering operation, determining a blurred video frame corresponding to the current video frame to be processed; and determining the blurred video frame corresponding to the current to-be-processed video frame. Audio special effects that are consistent with the audio information of the current video frame to be processed; use the blurred video frame as the background image of the current special effect video frame, and use the audio special effects as the foreground image of the current special effect video frame; by Special effects video frame splicing processing of video frames to obtain the target special effects video. The technical solution of the embodiment of the present disclosure not only presents the user's voice in a visual form, enhancing the interest of the special effects video, but also blurs the to-be-processed video frames shot by the user to meet the user's personalized needs.

Description

Methods, devices, electronic equipment and storage media for determining special effects videos

Technical field

Embodiments of the present disclosure relate to the field of video processing technology, and in particular, to a method, device, electronic device, and storage medium for determining special effect video.

Background technique

With the development of network technology, more and more applications have entered users' lives, especially a series of software that can shoot short videos, which are deeply loved by users. For example, users can shoot videos through the application software and publish the videos to specific platforms or share them with other users.

However, in the existing technology, the video special effects provided by the application for users are not rich enough, and the video content shot by the user lacks interest. At the same time, the user's personalized needs are not considered during the video shooting process, thus reducing the user's satisfaction. Use experience.

Contents of the invention

The present disclosure provides a method, device, electronic device and storage medium for determining special effects video, which not only presents the user's voice in a visual form and enhances the interest of the special effects video, but also blurs the to-be-processed video frames shot by the user. processing to meet the individual needs of users.

In a first aspect, embodiments of the present disclosure provide a method for determining special effects video, including:

In response to the special effect triggering operation, determining a blurred video frame corresponding to the current video frame to be processed; and determining an audio special effect consistent with the audio information of the current video frame to be processed;

Use the blurred video frame as the background image of the current special effect video frame, and use the audio special effect as the foreground image of the current special effect video frame;

By splicing the special effects video frames of each video frame to be processed, the target special effects video is obtained.

In a second aspect, embodiments of the present disclosure also provide a device for determining special effects video, including:

A blurred video frame determination module, configured to determine a blurred video frame corresponding to the current video frame to be processed in response to a special effect triggering operation; and to determine an audio special effect that is consistent with the audio information of the current video frame to be processed;

A special effects video frame generation module, configured to use the blurred video frame as the background image of the current special effects video frame, and the audio special effects as the foreground image of the current special effects video frame;

The target special effects video generation module is used to obtain the target special effects video by splicing the special effects video frames of each to-be-processed video frame.

In a third aspect, embodiments of the present disclosure also provide an electronic device, where the electronic device includes:

one or more processors;

a storage device for storing one or more programs,

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method for determining special effect video as described in any one of the embodiments of the present disclosure.

In a fourth aspect, embodiments of the present disclosure also provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform the determination of special effects as described in any of the embodiments of the present disclosure. Video method.

The technical solution of the embodiment of the present disclosure determines, in response to the special effect triggering operation, the blurred video frame corresponding to the current video frame to be processed, and the audio special effects consistent with the audio information of the current video frame to be processed; using the blurred video frame as the background image, using the audio special effects as the foreground image, thereby constructing the current special effects video frame. Further, each special effects video frame is spliced to obtain the target special effects video, which not only presents the user's voice in a visual form, but also enhances the special effects video. At the same time, by blurring the video frames shot by users to be processed, it meets the personalized needs of users and improves the user experience in the process of making special effects videos.

Description of the drawings

The above and other features, advantages, and aspects of various embodiments of the present disclosure will become more apparent with reference to the following detailed description taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It is to be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

Figure 1 is a schematic flowchart of a method for determining special effects video provided in Embodiment 1 of the present disclosure;

Figure 2 is a schematic structural diagram of a device for determining special effects video provided in Embodiment 2 of the present disclosure;

FIG. 3 is a schematic structural diagram of an electronic device provided by Embodiment 3 of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, which rather are provided for A more thorough and complete understanding of this disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

It should be understood that various steps described in the method implementations of the present disclosure may be executed in different orders and/or in parallel. Furthermore, method embodiments may include additional steps and/or omit performance of illustrated steps. The scope of the present disclosure is not limited in this regard.

As used herein, the term "include" and its variations are open-ended, ie, "including but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as “first” and “second” mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units. Or interdependence. It should be noted that the modifications of "one" and "plurality" mentioned in this disclosure are illustrative and not restrictive. Those skilled in the art will understand that unless the context clearly indicates otherwise, it should be understood as "one or Multiple”.

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of these messages or information.

Before introducing the technical solution, the application scenarios of the embodiments of the present disclosure may be exemplified. For example, when users shoot videos through application software or make video calls with other users, they may want to make the captured video content more interesting. At the same time, some users may have personalized needs for the video footage (for example, Some users with social phobia do not want their appearance to be shown in the video), it can be understood that these users want to hide all or part of the content in the shot (for example, the user's own facial image) ), at this time, according to the technical solution of this embodiment, while blurring the video picture, the user's audio special effects can be superimposed on each video frame, so as to obtain special effects that have richer picture content and effectively meet the user's personalized needs. video.

Embodiment 1

Figure 1 is a schematic flowchart of a method for determining special effects videos provided in Embodiment 1 of the present disclosure. This embodiment of the present disclosure is suitable for generating more interesting special effects videos while meeting the personalized needs of users. This method can It is executed by a device that determines the special effect video. The device can be implemented in the form of software and/or hardware. Optionally, it can be implemented through an electronic device. The electronic device can be a mobile terminal, a PC or a server.

As shown in Figure 1, the method includes:

S110. In response to the special effect triggering operation, determine the blurred video frame corresponding to the current video frame to be processed; and determine the audio special effect consistent with the audio information of the current video frame to be processed.

Among them, the device for executing the special effects video processing method provided by the embodiment of the present disclosure can be integrated in application software that supports special effects video processing functions, and the software can be installed in an electronic device. Optionally, the electronic device can be a mobile terminal or PC version etc. The application software may be a type of software for image/video processing. The specific application software will not be described in detail here, as long as it can realize image/video processing. It can also be a specially developed application to add special effects and display the special effects in the software, or it can be integrated in the corresponding page. Users can process the special effects video through the page integrated in the PC.

In this embodiment, in the application software or application program that supports the special effects video processing function, a control for triggering the special effects can be developed in advance. When it is detected that the user triggers the control, the special effects triggering operation can be responded to.

In this embodiment, the current video frame to be processed may be a frame of image captured by an electronic device installed with application software when responding to a special effect triggering operation, or may be a frame of image in the currently played video, corresponding to , the blurred video frame is an image obtained by blurring the current video frame to be processed.

It can be understood that the blurred video frame will not display the content in the picture as clearly as the video frame to be processed, but will reduce the clarity of the picture, that is, make the picture have a blurred visual effect, thus hiding all the information in each frame. or partial information. For example, after processing multiple video frames to be processed that contain user facial information and obtaining corresponding blurred video frames, the user's appearance cannot be accurately identified only by blurring the video frames.

In the actual application process, there are many ways to determine the blurred video frame. Alternatively, add a blur filter to the current video frame to be processed to obtain a blurred video frame; or perform Gaussian blur processing on the current video frame to be processed to obtain Blur the video frame; or, input the current video frame to be processed into the blur processing model to obtain the blurred video frame; or, if the current video frame to be processed includes a target object, blur the target object to obtain the blurred video frame. The above methods are explained below respectively.

In the first way of determining blurred video frames, the blur filter can be a pre-developed filter in the application software. Specifically, after adding a blur filter to the video frame to be processed, the entire area of each frame of the image can be blurred, or the areas with higher definition or stronger contrast in each frame of the image can be blurred, thus obtaining The corresponding blurred video frame. Persons skilled in the art should understand that when only a part of an area in an image is blurred, it can be achieved by balancing the parameters related to pixels in different areas of the image, and the embodiments of the present disclosure will not be repeated here.

In the second way to determine blurred videos, Gaussian blur is also called Gaussian smoothing. Gaussian blur processing can reduce image noise and reduce the level of detail. The visual effect of the image generated by this blur technology is like passing through a frosted glass. Observe the image. Based on this, it can be understood that after the current video frame to be processed is subjected to Gaussian blur processing, the corresponding blurred video frame can also be obtained.

In the third way of determining blurred videos, the blur processing model can be a pre-trained neural network model and can be integrated in relevant application software, at least for generating blurred video frames. It can be understood that the input of the model is the current video frame to be processed, and the output of the model is the corresponding blurred video frame. Those skilled in the art should understand that the fuzzy processing model can be trained based on the corresponding training set and verification set. When the loss function of the fuzzy processing model converges, it indicates that the model can be integrated in the application after training. Specifically, The training process will not be described again in this embodiment.

In the fourth way to determine the blurred video, the target object can be set in the application in advance. For example, image data containing specific user facial information is input into the application as the target object. Furthermore, when the application responds to a special effect trigger operation , and after identifying the facial information of a specific user in the display interface, the current video frame to be processed can be automatically blurred to obtain the corresponding blurred video frame. It can be understood that in this mode, when the target object is not recognized in the display interface, each frame image will not be blurred.

In this embodiment, while acquiring the current video frame to be processed, in order to present the user's voice in a visual form in the special effects video, it is first necessary to acquire the user's audio information. For example, the user's voice information is collected through the microphone on the electronic device while shooting a video. It can be understood that the audio information at least includes audio features used to characterize audio content or audio characteristics. In practical applications, audio features can also include voiceprint spectrum features. Correspondingly, audio special effects can also be dynamic spectrums (such as dynamic ripples) displayed by electroacoustic singularities. When its accuracy reaches a certain value, The sound wave spectrum can at least characterize a specific user and the voice information issued by that user; at the same time, when the volume in the audio changes, the dynamic ripples in the voiceprint speech will also fluctuate. When the volume is higher, the dynamic ripples will fluctuate. It is also larger, correspondingly, when the volume is lower, the fluctuation of the dynamic ripple is also smaller. It should be noted that in order to unify the visual and auditory effects in the final generated special effects video, the voiceprint voice also needs to be consistent with the audio information of the current video frame to be processed.

Further, after the application parses the audio information and obtains the audio features, it can construct audio special effects corresponding to the audio features of the audio information. The audio special effects can be displayed in the display interface and used to represent the audio features. Cartoon textures, such as pre-constructed small animal cartoon textures, can also be ripples corresponding to the voiceprint spectrum. Those skilled in the art should understand that audio special effects can be pre-created according to actual needs, such as country-related characteristic patterns or more. Visualized note patterns, etc. are not specifically limited in the embodiments of the present disclosure.

In this embodiment, the display form of the audio special effects includes dynamic display and/or static display. Specifically, the dynamic display is to display the audio features based on animation, and the static display is to display the audio features statically on the display interface. For example, when the application dynamically displays the pre-built small animal cartoon texture, the texture will continuously jump up and down in the display interface as the audio characteristics change. When the application statically displays the ripples corresponding to the voiceprint spectrum. , the application will splice multiple segments of ripples into a whole segment and display it on the display interface based on the timestamp. It can be understood that the size of the display interface is usually limited, so what is displayed at any time is only the ripple corresponding to the voiceprint spectrum. Only when the application detects that the user selects a specific moment or performs a drag operation on the ripple display area, will the ripple corresponding to the specific moment or drag operation be displayed.

Optionally, the audio information corresponding to the current video frame to be processed is processed based on the voiceprint feature extraction model to obtain the voiceprint speech. Among them, the voiceprint feature extraction model can be a pre-trained model, and can also be integrated in the application. It can be understood that after the user's audio information is collected, it can be input into the voiceprint feature extraction model to obtain the corresponding voiceprint speech. Of course, in the actual application process, the voiceprint voice input by the voiceprint feature extraction model can be a sound wave spectrum with various animation effects. For example, the output sound wave spectrum can be presented in a variety of shapes or colors. Those skilled in the art should It is understood that the specific visual effect presented by the voiceprint voice can be selected according to the actual situation, and the embodiments of the present disclosure are not specifically limited here.

S120. Use the blurred video frame as the background image of the current special effect video frame, and use the audio special effect as the foreground image of the current special effect video frame.

In this embodiment, after the blurred video frame and the audio special effect are determined, the special effect video frame can be constructed based on the above information. Specifically, the special effects video frame includes a background image and a foreground image. The foreground image is superimposed on the background image and can block all or part of the background image, thereby making the constructed special effects video frame more layered. Next The process of determining the background image is explained.

Optionally, in the process of determining the background image, the overlay background image corresponding to the current video frame to be processed can be determined, and the target transparency corresponding to the overlay background image can be determined; the overlay background image can be superimposed on the blurred video frame based on the target transparency. on, as a background image.

The overlay background image may be an image preset by the user through the application, or an image automatically selected by the application based on the brightness, color and other information of the video frame to be processed. Specifically, it can be a solid-color image, such as a pure black or pure gray image. When the above-mentioned solid-color image is selected as the overlay background image, the visual effect of the final special effects video frame can be made softer. It can be understood that the automatically selected image is more suitable for the picture presented by the video frame to be processed.

In the process of practical application, the pixel mean can be determined based on the pixel value of each pixel in the current video frame to be processed; based on the pixel mean, the superimposed background image corresponding to the current video frame to be processed is determined. For example, the current video frame to be processed is analyzed to determine the pixel value of each pixel in the picture. Further, the pixels of all channels (R, G, B) are averaged to obtain the average brightness of the current video frame to be processed. Based on the pixel mean, the corresponding color image can be determined as the overlay background image. It can be understood that a solid color overlay background image is determined based on the pixel average.

In order to make the final generated special effects video present a better visual effect, while determining the overlay background image, it is also necessary to determine the target transparency of the image. Similarly, the user can pre-set the target transparency through the application, and the application can also determine the target transparency according to the to-be-processed The corresponding target transparency is dynamically selected based on the brightness, color and other information of the video frame, which will not be described in detail here. Finally, after the superimposed background image is superimposed on the blurred video frame according to the target transparency, the background image of the special effect video frame is obtained. It should also be noted that the overlaid background image is set to have a certain transparency.

For example, after determining the blurred video frame, the application can automatically select a gray image as the overlay background image based on the blurred video frame, and at the same time determine the target transparency of this gray image to be 50% based on parameters preset by the user. On this basis, the gray image can be adjusted to 50% transparency and then superimposed on the blurred video frame, and the superimposed image can be used as the background image.

In this embodiment, after the background image is determined, the audio special effects consistent with the time corresponding to the background information can be used as the foreground image and superimposed on the background image, thereby constructing a special effects video frame. Those skilled in the art should understand that when there are multiple video frames to be processed, the application can determine corresponding special effect video frames for each video frame to be processed based on the solution of this embodiment. The embodiments of this disclosure will not be repeated here.

S130. Obtain the target special effects video by splicing the special effects video frames of each video frame to be processed.

In this embodiment, after the corresponding special effects video frames are determined for the multiple video frames to be processed, the sequence corresponding to the multiple special effects video frames can be determined according to the timestamp carried by each video frame to be processed, so that according to the sequence Multiple special effects video frames are spliced to obtain the target special effects video. It can be understood that in the target special effects video, the content in each video frame to be processed will be displayed with a blurred visual effect. At the same time, the voiceprint voice of the corresponding moment will be displayed at a specific position on the upper layer of the screen, that is, the user's audio at the moment will be displayed. Presented in a visual form.

In this embodiment, since the video frame to be processed is blurred and only the voiceprint is displayed, although the user's personalized needs are met, it may affect the video viewing experience of other users. Therefore, in order to improve the visibility of the special effects video to other users To increase the attractiveness, you can also add a target character model to the special effects video. This process is explained below.

Specifically, the target character model is superimposed on the foreground image; the body movement information and/or facial expressions of the target object in the current video frame to be processed are obtained; the target character model is adjusted to match the body movements and/or facial expressions.

The target character model can be a preset static or dynamic 3D model, which can at least present an anthropomorphic image, for example, a virtual cartoon character. At the same time, the target character model can be superimposed in the foreground image, for example, the target character model is presented above or below the voiceprint voice. Those skilled in the art should understand that the target character model will also block all or part of the background image, thereby making the constructed special effects video frame more layered.

In this embodiment, when the target character model is a dynamic model, in order to further enhance the interest of the special effects video, the movements of the model in the special effects video can also be matched with the real movements of the user. For example, when it is determined that the user's arm is raised through key point recognition technology in the current video frame to be processed, the arm of the target character model also needs to be adaptively adjusted, that is, the arm of the virtual character is raised; During the processing of the video frame, if it is determined that the user's arm has been lowered, the avatar's arm needs to be adaptively adjusted to be lowered. Based on this, in the final generated special effects video, the target character model can make actions that are basically consistent with the user's actual actions. It should be noted that during the actual application process, the application can also capture the user's facial expression in the video frame to be processed, thereby adaptively adjusting the expression of the target character model. The embodiments of the present disclosure will not be repeated here.

It can be understood that in any frame of the special effects video, the body movements and/or facial expressions of the target character model match the body movements and/or facial expressions of the target object in the video frame to be processed.

In practical applications, on the one hand, special effects videos can be generated in real time according to the solution of this embodiment, for example, during video calls between multiple users. On the other hand, you can also post-process existing videos to generate special effects videos. The following is an explanation of the special effects video obtained through post-processing.

Specifically, if the target special effects video frame is generated in the video recording mode, text information corresponding to the audio information is determined; when playing the target special effects video, the text information is displayed in the target area of the playback interface.

Specifically, different from the video call mode between multiple users, the video recording mode refers to the mode in which users shoot videos independently. In this mode, users can shoot special effects videos based on the functions provided by the application, and control the generated special effects. The video undergoes operations such as further processing, storage or sharing.

When the target special effects video is generated in video recording mode, in order to facilitate users to share the special effects video and improve the intelligence of the application, the text information corresponding to the audio information can be determined based on the pre-trained speech recognition model and/or semantic recognition model, and The text information is displayed in the target area of the playback interface. In this process, in order to avoid the text information being blocked and thus affecting the user's viewing experience, the target area is located in the foreground image of each special effects video frame to be processed; at the same time, in order to make the text The information does not block the voiceprint voice in each frame of the special effects image. In order to avoid visual confusion between the two elements for viewing users, the text information corresponding to the audio information and the voiceprint voice need to be distinguished in the playback interface. For example, the text information is displayed in white and in a specific font at the bottom of the playback interface, and the voiceprint voice is superimposed in a horizontal shape and multiple colors on the center of the background image, thereby displaying it in the center of the playback interface.

Those skilled in the art should understand that the audio information corresponding to each frame in the special effects video may be different. Based on this, the text information displayed in the playback interface will also continue to change as the video is played. By displaying the text information corresponding to the audio information in the playback interface, the technical effect of automatically adding "subtitles" to the special effects video is realized, which avoids the problem that the audio information entered by the user is not clear, thus affecting the viewing experience of other users' special effects video.

It should be noted that when the corresponding text information is generated for the target special effects video, in order to avoid the audio information, text information and voiceprint voice elements in the special effects video from being out of sync, you can also adjust the audio information, text information and voice. Text and voice are displayed on the same frequency. For example, controls for adjusting the timestamps of the above three elements are developed in the application in advance. After the corresponding text information is determined for the audio information corresponding to each frame in the special effects video, if it is found in the generated special effects video When the text information in a certain frame or multiple frames is out of sync with the actual audio information, you can adjust the text information corresponding to a certain frame or multiple frames of special effects images by applying an adjustment operation to the control that controls the timestamp of the text information. It can be understood that when the voiceprint voice and the actual audio in a certain frame or multiple frames of the special effects video are out of synchronization, the timestamp can also be adjusted in the above manner, and the embodiments of the present disclosure will not be repeated here.

In this embodiment, the application may also display the target keyword associated with the audio information on the display interface. Optionally, update at least one target keyword on the display interface based on the historical audio information of each historical video frame to be processed before the current video frame to be processed.

Specifically, after determining the user's audio information, the application can parse the audio information based on a pre-trained speech processing model and extract keywords therein. Among them, the keywords can be one or more words that appear most frequently in the audio information, or they can also be words that appear in the audio information and are consistent with the content in the pre-built keyword thesaurus. Of course, in the actual application process , the keywords can also be current hot words appearing in the audio information, or professional vocabulary related to a certain field, etc. Those skilled in the art should understand that the keyword extraction rules can be set according to actual needs. Embodiments of the present disclosure There are no specific limitations here.

In this embodiment, after the application determines the keywords in the audio information, the keywords can be displayed in the display interface, where the displayed keywords are at least used to realize the positioning function of historical video frames. Specifically, when a triggering target keyword is detected, jump to the historical target special effects video frame corresponding to the target keyword and play it; wherein, the historical target special effects video frame is based on the special effects processing of the historical to-be-processed video frame. The resulting video frames.

It can be understood that when the application detects the user's trigger operation for any one of the keywords, the keyword is the target keyword. Furthermore, the application can automatically jump to the currently playing special effects video frame to the frame containing the key. In this way, users can quickly locate the special effects video frames corresponding to this content when they later recall the content associated with a certain keyword. For example, the word "technical solution" appears in the user's audio information. At the same time, the application has determined that the word is a keyword and displayed it in the display interface. On this basis, when the user subsequently plays back the words generated by the application, When viewing a special effects video, if you want to watch only the part of the special effects video that mentions "technical solution", you can click on the word "technical solution" displayed on the display interface. When the application detects the user's trigger operation, you can Automatically jump to the historical special effects video frame where the word "technical solution" first appears, and continue playing from that frame.

The technical solution of the embodiment of the present disclosure determines, in response to the special effect triggering operation, the blurred video frame corresponding to the current video frame to be processed, and the audio special effects consistent with the audio information of the current video frame to be processed; using the blurred video frame as the background image, using the audio special effects as the foreground image, thereby constructing the current special effects video frame. Further, each special effects video frame is spliced to obtain the target special effects video, which not only presents the user's voice in a visual form, but also enhances the special effects video. At the same time, by blurring the video frames shot by users to be processed, it meets the personalized needs of users and improves the user experience in the process of making special effects videos.

Embodiment 2

Figure 2 is a schematic structural diagram of a device for determining special effects video provided in Embodiment 2 of the present disclosure. As shown in Figure 2, the device includes: a blur video frame determination module 210, a special effect video frame generation module 220, and a target special effect video Generate module 230.

The blurred video frame determination module 210 is configured to determine the blurred video frame corresponding to the current video frame to be processed in response to the special effect triggering operation; and to determine the audio special effects consistent with the audio information of the current video frame to be processed.

The special effect video frame generation module 220 is configured to use the blurred video frame as the background image of the current special effect video frame, and the audio special effect as the foreground image of the current special effect video frame.

The target special effects video generation module 230 is used to obtain the target special effects video by splicing the special effect video frames of the video frames to be processed.

Based on the above technical solutions, the blurred video frame determination module 210 includes a blurred video frame determination unit and an audio special effects determination unit.

A blur video frame determination unit configured to add a blur filter to the current video frame to be processed to obtain the blurred video frame; or to perform Gaussian blur processing on the current video frame to be processed to obtain a blurred video frame; or, Input the current video frame to be processed into the blur processing model to obtain the blurred video frame; or, if the current video frame to be processed includes a target object, blur the target object to obtain the blurred video frame.

The audio special effects determination unit is used to process the audio information corresponding to the current video frame to be processed based on the voiceprint feature extraction model to obtain the audio special effects.

On the basis of the above technical solutions, the audio special effects correspond to the audio characteristics of the audio information; the audio characteristics include voiceprint spectrum characteristics; the display form of the audio special effects includes dynamic display and/or static display. The dynamic display is to display audio features based on animation, and the static display is to statically display the audio features on the display interface.

Based on the above technical solutions, the special effects video frame generation module 220 includes a superimposed background image determination unit and a background image determination unit.

An overlay background image determination unit is configured to determine an overlay background image corresponding to the currently to-be-processed video frame, and a target transparency corresponding to the overlay background image.

A background image determining unit configured to superimpose the superimposed background image on the blurred video frame according to the target transparency as the background image.

Optionally, the superimposed background image determination unit is also configured to determine the pixel mean value according to the pixel value of each pixel point in the current video frame to be processed; based on the pixel mean value, determine the corresponding pixel value of the current video frame to be processed. overlay background image.

Based on the above technical solutions, the device for determining special effects video also includes a target character model determination module.

The target character model determination module is used to superimpose the target character model in the foreground image; obtain the body movement information and/or facial expression of the target object in the current video frame to be processed; adjust the target character model, and the Body movements and/or facial expressions matched.

Based on the above technical solutions, the device for determining special effects video also includes a target keyword update module.

The target keyword update module is configured to update at least one target keyword on the display interface based on the historical audio information of each historical video frame to be processed before the current video frame to be processed.

Based on the above technical solutions, the device for determining special effect video also includes a jump module.

A jump module, used to jump to the historical target special effects video frame corresponding to the target keyword when a trigger target keyword is detected, and play it; wherein the historical target special effects video frame is based on the historical target special effects video frame. The video frame obtained after special effects processing of the video frame to be processed.

Based on the above technical solutions, the device for determining special effects video also includes a text information determination module.

Text information determination module, used to determine the text information corresponding to the audio information if the target special effects video frame is generated in the video recording mode; when playing the target special effects video, in the target area of the playback interface The text information is displayed in; wherein the target area is located in the foreground image of each special effect video frame to be processed.

Based on the above technical solutions, the device for determining special effect video also includes an adjustment module.

The adjustment module is used to adjust the audio information, text information and audio special effects to be displayed at the same frequency.

On the basis of the above technical solutions, the device for determining special effect video also includes a distinction display module.

The differential display module is used to differentially display text information and audio special effects corresponding to the audio information in the middle and lower parts of the playback interface.

The technical solution provided by this embodiment determines, in response to the special effect triggering operation, the blurred video frame corresponding to the current video frame to be processed, and the audio special effects consistent with the audio information of the current video frame to be processed; the blurred video frame is regarded as The background image uses the audio special effects as the foreground image to construct the current special effects video frame. Further, each special effects video frame is spliced to obtain the target special effects video. It not only presents the user's voice in a visual form, but also enhances the special effects. At the same time, by blurring the video frames shot by users to be processed, it meets the personalized needs of users and improves the user experience in the process of making special effects videos.

The device for determining special effects video provided by the embodiments of the present disclosure can execute the method for determining special effects video provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

It is worth noting that the various units and modules included in the above-mentioned devices are only divided according to functional logic, but are not limited to the above-mentioned divisions, as long as they can achieve the corresponding functions; in addition, the specific names of each functional unit are just In order to facilitate mutual differentiation, it is not used to limit the protection scope of the embodiments of the present disclosure.

Embodiment 3

FIG. 3 is a schematic structural diagram of an electronic device provided by Embodiment 3 of the present disclosure. Referring now to FIG. 3 , a schematic structural diagram of an electronic device (such as the terminal device or server in FIG. 3 ) 300 suitable for implementing embodiments of the present disclosure is shown. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (such as Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 3 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 3 , the electronic device 300 may include a processing device (such as a central processing unit, a pattern processor, etc.) 301 , which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 302 or from a storage device 306 . The program in the memory (RAM) 303 executes various appropriate actions and processes. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302 and the RAM 303 are connected to each other via a bus 304. An editing/output (I/O) interface 305 is also connected to bus 304.

Generally, the following devices may be connected to the I/O interface 305: an editing device 306 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration An output device 307 such as a computer; a storage device 308 including a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG. 3 illustrates electronic device 300 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such embodiments, the computer program may be downloaded and installed from the network via communication device 309, or from storage device 306, or from ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of these messages or information.

The electronic device provided by the embodiments of the present disclosure and the method for determining special effect video provided by the above embodiments belong to the same inventive concept. Technical details that are not described in detail in this embodiment can be referred to the above embodiments, and this embodiment has the same characteristics as the above embodiments. Same beneficial effects.

Embodiment 4

Embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method for determining special effect video provided in the above embodiments is implemented.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium. (e.g., communications network) interconnection. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device:

In response to the special effect triggering operation, determining a blurred video frame corresponding to the current video frame to be processed; and determining an audio special effect consistent with the audio information of the current video frame to be processed;

Use the blurred video frame as the background image of the current special effect video frame, and use the audio special effect as the foreground image of the current special effect video frame;

By splicing the special effects video frames of each video frame to be processed, the target special effects video is obtained.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. The name of the unit does not constitute a limitation on the unit itself under certain circumstances. For example, the first acquisition unit can also be described as "the unit that acquires at least two Internet Protocol addresses."

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

According to one or more embodiments of the present disclosure, [Example 1] provides a method for determining special effects videos, which method includes:

In response to the special effect triggering operation, determining a blurred video frame corresponding to the current video frame to be processed; and determining an audio special effect consistent with the audio information of the current video frame to be processed;

Use the blurred video frame as the background image of the current special effect video frame, and use the audio special effect as the foreground image of the current special effect video frame;

By splicing the special effects video frames of each video frame to be processed, the target special effects video is obtained.

According to one or more embodiments of the present disclosure, [Example 2] provides a method for determining special effects videos, which method further includes:

Optionally, add a blur filter to the current video frame to be processed to obtain the blurred video frame; or,

Perform Gaussian blur processing on the current video frame to be processed to obtain a blurred video frame; or,

Input the current video frame to be processed into the blur processing model to obtain the blurred video frame; or,

If the current video frame to be processed includes a target object, the target object is blurred to obtain a blurred video frame.

According to one or more embodiments of the present disclosure, [Example 3] provides a method for determining special effects videos, which method further includes:

Optionally, the audio information corresponding to the current video frame to be processed is processed based on a voiceprint feature extraction model to obtain the audio special effect.

According to one or more embodiments of the present disclosure, [Example 4] provides a method for determining special effects videos, which method further includes:

Optionally, the audio special effects correspond to the audio features of the audio information; the audio features include voiceprint spectrum features; the display form of the audio special effects includes dynamic display and/or static display, and the dynamic display is based on animation. Display audio features, and the static display means that the audio features are statically displayed on the display interface.

According to one or more embodiments of the present disclosure, [Example 5] provides a method for determining special effects videos, which method further includes:

Optionally, determine the superimposed background image corresponding to the current video frame to be processed, and the target transparency corresponding to the superimposed background image;

The superimposed background image is superimposed on the blurred video frame according to the target transparency as the background image.

According to one or more embodiments of the present disclosure, [Example 6] provides a method for determining special effects videos, which method further includes:

Optionally, determine the pixel mean value based on the pixel value of each pixel in the current video frame to be processed;

Based on the pixel mean, an overlay background image corresponding to the current video frame to be processed is determined.

According to one or more embodiments of the present disclosure, [Example 7] provides a method for determining special effects videos, which method further includes:

Optionally, superimpose a target character model in the foreground image;

Obtain body movement information and/or facial expressions of the target object in the current video frame to be processed;

The target character model is adjusted to match the body movements and/or the facial expressions.

According to one or more embodiments of the present disclosure, [Example 8] provides a method for determining special effects video, the method further includes:

Optionally, update at least one target keyword on the display interface based on the historical audio information of each historical video frame to be processed before the current video frame to be processed.

According to one or more embodiments of the present disclosure, [Example 9] provides a method for determining special effects video, the method further includes:

Optionally, when a triggering target keyword is detected, jump to the historical target special effects video frame corresponding to the target keyword and play it;

Wherein, the historical target special effect video frames are video frames obtained after special effects processing of historical video frames to be processed.

According to one or more embodiments of the present disclosure, [Example 10] provides a method for determining special effects video, the method further includes:

Optionally, if the target special effect video frame is generated in the video recording mode, determine the text information corresponding to the audio information;

When playing the target special effects video, display the text information in the target area of the playback interface;

Wherein, the target area is located in the foreground image of each special effect video frame to be processed.

According to one or more embodiments of the present disclosure, [Example 11] provides a method for determining special effects video, the method further includes:

Optionally, adjust the audio information, text information and audio special effects to be displayed at the same frequency.

According to one or more embodiments of the present disclosure, [Example 12] provides a method for determining special effects videos, and the method further includes:

Optionally, text information and audio special effects corresponding to the audio information are displayed differently in the middle and lower parts of the playback interface.

According to one or more embodiments of the present disclosure, [Example 13] provides a device for determining special effects video, the device includes:

A blurred video frame determination module, configured to determine a blurred video frame corresponding to the current video frame to be processed in response to a special effect triggering operation; and to determine an audio special effect that is consistent with the audio information of the current video frame to be processed;

A special effects video frame generation module, configured to use the blurred video frame as the background image of the current special effects video frame, and the audio special effects as the foreground image of the current special effects video frame;

The target special effects video generation module is used to obtain the target special effects video by splicing the special effects video frames of each to-be-processed video frame.

The above description is only a description of the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art should understand that the disclosure scope involved in the present disclosure is not limited to technical solutions composed of specific combinations of the above technical features, but should also cover solutions composed of the above technical features or without departing from the above disclosed concept. Other technical solutions formed by any combination of equivalent features. For example, a technical solution is formed by replacing the above features with technical features with similar functions disclosed in this disclosure (but not limited to).

Furthermore, although operations are depicted in a specific order, this should not be understood as requiring that these operations be performed in the specific order shown or performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (14)

1. A method of determining a special effect video, comprising:

responding to the special effect triggering operation, and determining a fuzzy video frame corresponding to the current video frame to be processed; determining an audio special effect consistent with the audio information of the current video frame to be processed;

The blurred video frame is used as a background image of the current special effect video frame, and the audio special effect is used as a foreground image of the current special effect video frame;

the target special effect video is obtained through special effect video frame splicing processing of each video frame to be processed;

the step of taking the blurred video frame as the background image of the current special effect video frame comprises the following steps:

determining a superimposed background image corresponding to the current video frame to be processed and a target transparency corresponding to the superimposed background image;

and superposing the superposed background image on the fuzzy video frame according to the target transparency to serve as the background image.

2. The method of claim 1, wherein the determining a blurred video frame corresponding to a current video frame to be processed comprises:

adding a fuzzy filter into the current video frame to be processed to obtain the fuzzy video frame; or alternatively, the first and second heat exchangers may be,

gaussian blur processing is carried out on the current video frame to be processed to obtain a blurred video frame; or alternatively, the first and second heat exchangers may be,

inputting the current video frame to be processed into a fuzzy processing model to obtain the fuzzy video frame; or alternatively, the first and second heat exchangers may be,

and if the current video frame to be processed comprises the target object, blurring the target object to obtain a blurred video frame.

3. The method of claim 1, wherein said determining an audio special effect consistent with audio information of the current video frame to be processed comprises:

and processing the audio information corresponding to the current video frame to be processed based on the voiceprint feature extraction model to obtain the audio special effect.

4. A method according to any one of claims 1-3, wherein the audio effects correspond to audio features of the audio information; the audio features include voiceprint spectral features; the display form of the audio special effect comprises dynamic display and/or static display, wherein the dynamic display is used for displaying audio characteristics based on animation, and the static display is used for displaying the audio characteristics on a display interface in a static mode.

5. The method of claim 1, wherein the determining the superimposed background image corresponding to the current video frame to be processed comprises:

determining a pixel mean value according to the pixel value of each pixel point in the current video frame to be processed;

and determining an overlapped background image corresponding to the current video frame to be processed based on the pixel mean value.

6. The method as recited in claim 1, further comprising:

Superimposing a target character model in the foreground image;

acquiring limb action information and/or facial expression of a target object in the current video frame to be processed;

the target character model is adjusted to match the limb movements and/or the facial expressions.

7. The method as recited in claim 1, further comprising:

updating at least one target keyword on a display interface according to the historical audio information of each historical video frame to be processed before the current video frame to be processed;

updating at least one target keyword on a display interface according to the historical audio information of each historical video frame to be processed before the current video frame to be processed, wherein the updating comprises the following steps:

after the audio information is determined, analyzing the audio information based on a pre-trained voice processing model, extracting keywords in the audio information, and displaying the keywords in a display interface, wherein the displayed keywords are at least used for realizing the positioning function of historical video frames.

8. The method as recited in claim 7, further comprising:

when triggering operation on a target keyword is detected, jumping to a historical target special effect video frame corresponding to the target keyword, and playing;

The historical target special effect video frame is based on a video frame obtained after special effect processing of a historical to-be-processed video frame.

9. The method as recited in claim 1, further comprising:

if the target special effect video frame is generated in a video recording mode, determining text information corresponding to the audio information;

displaying the text information in a target area of a playing interface when the target special effect video is played;

the target area is located in a foreground image of each special effect video frame to be processed.

10. The method as recited in claim 9, further comprising:

and adjusting the audio information, the text information and the audio special effect to be displayed in the same frequency.

11. The method as recited in claim 10, further comprising:

and displaying the text information and the audio special effect corresponding to the audio information in the playing interface in a lower distinguishing way.

12. An apparatus for determining a special effect video, comprising:

the fuzzy video frame determining module is used for responding to the special effect triggering operation and determining a fuzzy video frame corresponding to the current video frame to be processed; determining an audio special effect consistent with the audio information of the current video frame to be processed;

The special effect video frame generation module is used for taking the blurred video frame as a background image of the current special effect video frame and taking the audio special effect as a foreground image of the current special effect video frame;

the target special effect video generation module is used for obtaining target special effect video through the special effect video frame splicing processing of each video frame to be processed;

the special effect video frame generation module comprises a superimposed background image determination unit and a background image determination unit;

the superimposed background image determining unit is used for determining a superimposed background image corresponding to the current video frame to be processed and a target transparency corresponding to the superimposed background image;

the background image determining unit is used for superposing the superposed background image on the blurred video frame according to the target transparency to serve as the background image.

13. An electronic device, the electronic device comprising:

one or more processors;

storage means for storing one or more programs,

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of determining special effects video of any of claims 1-11.

14. A storage medium containing computer executable instructions for performing the method of determining special effects video of any one of claims 1-11 when executed by a computer processor.