CN114372172A - Method and device for generating video cover image, computer equipment and storage medium - Google Patents

Abstract

The application provides a method, a device, computer equipment and a storage medium for generating a video cover image, which can be applied to artificial intelligence, the traffic field or the map field and the like and are used for solving the problem of low accuracy of the generated video cover image. The method comprises the following steps: acquiring a video to be processed and an initial cover image of the video to be processed; generating a target cover image of the video to be processed based on the video to be processed and the initial cover image; wherein the target cover image is generated by adding a picture-in-picture image on the initial cover image, and the picture-in-picture image is matched with the content of the video to be processed. Therefore, the picture-in-picture image plays a role in emphasizing the content of the video to be processed on the basis of the initial cover image, and the accuracy of generating the video cover is improved.

Description

Method, device, computer device and storage medium for generating video cover image

technical field

The present application relates to the field of computer technology, and in particular, to a method, apparatus, computer device, and storage medium for generating a video cover image.

Background technique

With the continuous development of self-media technology, more and more network media platforms can provide network objects with the service of publishing videos and watching videos. On the network media platform, the network object can achieve the purpose of publishing the video by uploading the produced video. The published video can provide a video preview through the title and cover image of the video, so that network objects can click to enter the video of interest through the video preview, and watch the videos released by other network objects.

The cover image of a video is usually a video frame selected from the video. However, a video frame in the video can only represent a video scene, and cannot accurately represent the theme of the video, let alone highlight the key content expressed by the video frame. Therefore, the network object cannot accurately click to enter the video of interest through the cover image of the video, so that the network object needs to click to enter multiple videos for viewing before viewing the video of interest.

It can be seen that under the related art, the accuracy of the generated video cover image is low.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, apparatus, computer device, and storage medium for generating a video cover image, so as to solve the problem of low accuracy in generating a video cover image.

In a first aspect, a method for generating a video cover image is provided, including:

Obtain the video to be processed, and the initial cover image of the video to be processed;

generating a target cover image of the video to be processed based on the video to be processed and the initial cover image;

The target cover image is generated by adding a picture-in-picture image to the initial cover image, and the picture-in-picture image matches the content of the video to be processed.

In a second aspect, a device for generating a video cover image is provided, comprising:

Acquisition module: used to acquire the video to be processed and the initial cover image of the video to be processed;

Processing module: based on the video to be processed and the initial cover image, generate a target cover image of the video to be processed;

The target cover image is generated by adding a picture-in-picture image to the initial cover image, and the picture-in-picture image matches the content of the video to be processed.

Optionally, the processing module is specifically used for:

Based on the information extraction strategy, extract the key feature information contained in the video to be processed, wherein the key feature information is used to represent: the subject of the video to be processed, and the key objects contained;

Acquire each candidate material that matches the key feature information, and select a candidate material that satisfies the presentation conditions from the various candidate materials as the target material;

Using the target material as a picture-in-picture image of the initial cover image, synthesizing the target material and the initial cover image to generate the target cover image.

Optionally, the processing module is specifically used for:

obtaining each reference image, wherein the reference image is an image collected from a network resource;

Extracting the respective video frame features of each to-be-processed video frame contained in the video to be processed, and extracting the respective image features of the respective reference images;

Based on each video frame feature and each image feature, from each of the video frames to be processed and each of the reference images, each candidate material matching the key feature information is determined.

Optionally, the key feature information includes: word features, face features, and object features;

Then the processing module is specifically used for:

Determine the graphic similarity between the word feature and each video frame feature, and between the word feature and each image feature, respectively;

Respectively determine the facial similarity between the facial feature and each video frame feature, and between the facial feature and each image feature;

respectively determining the object similarity between the object feature and each video frame feature, and between the object feature and each image feature;

Based on the obtained image-text matching degree, each face matching degree and each object matching degree, each candidate material matching the key feature information is determined from each of the video frames to be processed and each of the reference images.

Optionally, the processing module adopts the following method to obtain the key feature information:

Obtain the release information and the subtitle file of the video to be processed, and extract the word features of the keywords contained in the release information and the subtitle file;

Acquiring key video frames in the video to be processed, and extracting the facial features of the face region included in the key video frames, and the object features of the object regions included, wherein the key video frames are the to-be-processed In the video, the video frame representing the video scene switching;

The obtained word features, face features and object features are used as the key feature information.

Optionally, the processing module is specifically used for:

Based on each video frame feature and each image feature, from each of the video frames to be processed and each of the reference images, determine each candidate image that matches the key feature information;

Determining the respective candidate images, each of which matches the image area with the key feature information;

Based on the respective image regions, the respective candidate images are cropped to obtain respective candidate materials.

Optionally, the processing module is specifically used for:

Based on the sharpness evaluation strategy, perform sharpness evaluation on the respective candidate materials, and determine the respective sharpness evaluation values of the respective candidate materials;

Based on the content quality evaluation strategy, perform content quality evaluation on each candidate material, and determine the respective content quality evaluation value of each candidate material;

A weighted sum of each clarity evaluation value and each content quality evaluation value is obtained, and the weighted sum conforms to the candidate material larger than the presentation threshold as the target material.

Optionally, the processing module is specifically used for:

Detecting the initial cover image, and determining the target object contained in the initial cover image;

dividing the initial cover image into a target area and a non-target area based on the position of the target object in the initial cover image;

Adjusting the size of the target material based on the shape and size of the non-target area;

The adjusted target material is overlaid on the non-target area in the initial cover image to generate the target cover image of the video to be processed.

In a third aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to the first aspect.

In a fourth aspect, a computer device is provided, comprising:

memory for storing program instructions;

The processor is configured to call the program instructions stored in the memory, and execute the method according to the first aspect according to the obtained program instructions.

In a fifth aspect, a computer-readable storage medium is provided, the storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to perform the method according to the first aspect or the description.

In this embodiment of the present application, the picture-in-picture image added to the initial cover image matches the content of the video to be processed, for example, is related to the theme of the video to be processed, or matches the key objects contained in the video to be processed, then obtain The target cover image can more accurately represent the content of the video to be processed and improve the accuracy of generating video cover.

Further, the picture-in-picture image can play a role in emphasizing the content of the video to be processed on the basis of the initial cover image, for example, it can play a role in emphasizing the subject of the video to be processed or the key objects contained in it, so as to avoid showing the initial image separately. When the cover image represents the unclear content, the accuracy of generating the video cover is further improved.

Description of drawings

1a is a schematic diagram of a method for generating a video cover image in the related art;

FIG. 1b is a schematic diagram 1 of a principle of a method for generating a video cover image provided by an embodiment of the application;

Fig. 1c is an application scenario of the method for generating a video cover image provided by an embodiment of the present application;

Fig. 2a is a schematic flowchart 1 of a method for generating a video cover image provided by an embodiment of the application;

2b is a second schematic flowchart of a method for generating a video cover image provided by an embodiment of the application;

3 is a schematic diagram 2 of a principle of a method for generating a video cover image provided by an embodiment of the present application;

4a is a schematic diagram three of a principle of a method for generating a video cover image provided by an embodiment of the application;

FIG. 4b is a fourth schematic diagram of a principle of a method for generating a video cover image provided by an embodiment of the application;

Fig. 4c is a schematic diagram five of a principle of a method for generating a video cover image provided by an embodiment of the present application;

FIG. 5 is a schematic sixth schematic diagram of a method for generating a video cover image provided by an embodiment of the present application;

6a is a seventh schematic diagram of a principle of a method for generating a video cover image provided by an embodiment of the present application;

FIG. 6b is a schematic eighth schematic diagram of a method for generating a video cover image provided by an embodiment of the present application;

6c is a schematic diagram 9 of a principle of a method for generating a video cover image provided by an embodiment of the present application;

FIG. 6d is a schematic schematic diagram ten of a method for generating a video cover image provided by an embodiment of the present application;

FIG. 6e is an eleventh schematic schematic diagram of a principle of a method for generating a video cover image provided by an embodiment of the application;

7 is a schematic structural diagram 1 of an apparatus for generating a video cover image provided by an embodiment of the present application;

FIG. 8 is a second schematic structural diagram of an apparatus for generating a video cover image according to an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application.

Some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

(1) Feeds:

Feeds are news sources, which are short forms of web feeds, news feeds, and syndicated feeds. Feeds are a data format through which a website disseminates the latest information to users, usually arranged in a timeline. A prerequisite for a user to be able to subscribe to a website is that the website provides a source of news.

(2) Short video:

Short video is a kind of Internet content dissemination method, which is generally a video dissemination content within 5 minutes of dissemination on new Internet media. With the popularization of mobile terminals and the speeding up of the Internet, the short-term, fast-paced and large-flow dissemination of content has gradually gained the favor of major platforms, fans and capital.

The embodiments of the present application relate to the field of artificial intelligence (Artificial Intelligence, AI), are designed based on machine learning (Machine Learning, ML) technology, and can be applied to fields such as cloud computing, smart transportation, assisted driving, or maps.

Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which studies the design principles and implementation methods of various machines, attempts to understand the essence of intelligence, and produces a new kind of artificial intelligence that can respond in a similar way to human intelligence. Intelligent machines enable machines to perceive, reason and make decisions.

Artificial intelligence is a comprehensive discipline that covers a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating interactive systems, and mechatronics. The software technologies of artificial intelligence mainly include computer vision technology, speech processing technology, natural language processing technology, machine learning/deep learning, autonomous driving, and intelligent transportation. With the development and progress of artificial intelligence, artificial intelligence can be researched and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, smart wearable devices, unmanned driving, automatic It is believed that with the further development of future technology, artificial intelligence will be applied in more fields and play an increasingly important role. value. The solutions provided by the embodiments of the present application relate to technologies such as deep learning of artificial intelligence and augmented reality, and are further described by the following embodiments.

Machine learning is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Reorganize the existing knowledge structure, so that the computer continuously improves its own performance.

Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence; the core of machine learning is deep learning, which is a technology for realizing machine learning. Machine learning usually includes deep learning, reinforcement learning, transfer learning, inductive learning, artificial neural network, teaching learning and other technologies, while deep learning includes Convolutional Neural Networks (CNN), deep belief networks, recurrent neural networks, Autoencoders, Generative Adversarial Networks, etc.

It should be noted that, in the embodiments of this application, related data such as user portraits, user historical operation records, etc. are involved. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain user permission or consent, and relevant data. The collection, use, and processing of data must comply with the relevant laws, regulations and standards of relevant countries and regions.

The following briefly introduces the application fields of the method for generating a video cover image provided by the embodiments of the present application.

With the continuous development of self-media technology, more and more network media platforms can provide network objects with the service of publishing videos and watching videos. The threshold for content production has been lowered, allowing the release of various content to grow exponentially. Content sources come from various content creation agencies, such as professionally-generated content (PGC) and user-generated content (UGC) from self-media and agencies.

On the network media platform, the network object can achieve the purpose of publishing the video by uploading the produced video. The published video can provide a video preview through the title and cover image of the video, so that network objects can click to enter the video of interest through the video preview, and watch the videos released by other network objects.

The most important factors in the viewing of content by web objects are the title, cover image and author of the content. Referring to Figure 1a, the cover image is the first impression of the video on the network object. When consuming video content, the quality of the cover image greatly affects the viewing desire of the network object. The quality of the cover image includes two aspects, one is the quality of the cover image, such as whether it is clear, whether it has flaws, etc., and the other is the information conveyed by the cover image, whether the content is effective, whether it fits the theme of the video, etc. Web objects upload video, to a certain extent, a creative process, delivering content through the video, and in the title of the video, it is generally described. A good cover image should be as relevant as possible to the title and content, or to the picture expressed in the video, because it usually most directly reflects the subject, such as a scene, a star, or a point highlighted and the gaps between those points. mutual relationship.

The cover image of a video is usually a video frame selected from the video. However, a video frame in the video can only represent a video scene, and cannot accurately represent the theme of the video, let alone highlight the key content expressed by the video frame. Therefore, the network object cannot accurately click to enter the video of interest through the cover image of the video, so that the network object needs to click to enter multiple videos for viewing before viewing the video of interest.

It can be seen that under the related art, the accuracy of the generated video cover image is low.

In order to solve the problem of low accuracy in generating a video cover image, the present application proposes a method for generating a video cover image. After acquiring the video to be processed and the initial cover image of the video to be processed, the method can generate a target cover image of the video to be processed based on the video to be processed and the initial cover image. Referring to FIG. 1b, the target cover image is generated by adding a picture-in-picture image to the initial cover image, and the picture-in-picture image matches the content of the video to be processed.

In this embodiment of the present application, the picture-in-picture image added to the initial cover image matches the content of the video to be processed, for example, is related to the theme of the video to be processed, or matches the key objects contained in the video to be processed, then obtain The target cover image can more accurately represent the content of the video to be processed and improve the accuracy of generating video cover.

Further, the picture-in-picture image can play a role in emphasizing the content of the video to be processed on the basis of the initial cover image, for example, it can play a role in emphasizing the subject of the video to be processed or the key objects contained in it, so as to avoid showing the initial image separately. When the cover image represents the unclear content, the accuracy of generating the video cover is further improved.

The following describes the application scenarios of the method for generating a video cover image provided by the present application.

Please refer to FIG. 1 c , which is a schematic diagram of an application scenario of the method for generating a video cover image provided by the present application. The application scenario includes a client 101 and a server 102 . The client 101 and the server 102 can communicate, and the communication method can be by using wired communication technology, for example, by connecting a network cable or serial cable; or by using wireless communication technology, for example, by using Bluetooth or wireless security. Wireless fidelity (wireless fidelity, WIFI) and other technologies to communicate, there is no specific limitation.

The client 101 generally refers to a device that can provide the video to be processed to the server 102 , such as a terminal device, a web page that the terminal device can access, or a third-party program that the terminal device can access. The terminal device may be an intelligent transportation device, a camera, a mobile phone, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, and the like. The server 102 generally refers to a device that can process the video to be processed, such as a terminal device or a server. Servers include but are not limited to cloud servers, local servers or associated third-party servers, etc. Both the client 101 and the server 102 can use cloud computing to reduce the occupation of local computing resources; similarly, cloud storage can also be used to reduce the occupation of local storage resources.

As an embodiment, the client 101 and the server 102 may be the same device, which is not specifically limited. In the embodiments of the present application, the client 101 and the server 102 are respectively different devices for description as an example.

Based on FIG. 1 c , the method for generating a video cover image provided by the embodiment of the present application will be described in detail by taking the client 101 as the client and the server 102 as the server as an example.

Please refer to FIG. 2a , which is a schematic flowchart of a method for generating a video cover image provided by an embodiment of the present application.

S21: Acquire the video to be processed and an initial cover image of the video to be processed.

The client can obtain the video to be processed and the initial cover image of the video to be processed. For example, in response to the target object's control operation on the display interface, the client loads the video to be processed and the initial cover image of the video to be processed. The control operation may be an operation of uploading a video, an operation of shooting a video, or an operation of entering an interface for browsing video thumbnails, etc., which are not specifically limited. For another example, the client may receive the video to be processed sent by other devices, as well as the initial cover image of the video to be processed, and the like.

For another example, after obtaining the video to be processed, the client can extract the initial cover image of the video to be processed based on the video to be processed. The key video frame with the largest number of video frames included between the video frames may also be a designated video frame selected from the video to be processed in response to a selection operation triggered by the target object, etc., which is not specifically limited.

S22, based on the video to be processed and the initial cover image, generate a target cover image of the video to be processed.

After acquiring the video to be processed and the initial cover image of the video to be processed, the client may generate a target cover image of the video to be processed based on the video to be processed and the initial cover image. The target cover image may be generated by the client determining a picture-in-picture image that matches the content of the video to be processed, and adding the picture-in-picture image to the initial cover image. The target cover image can also be the client sending the video to be processed and the initial cover image to the server, the server determines the picture-in-picture image that matches the content of the video to be processed, and adds the picture-in-picture image to the initial cover image to generate Yes, after the client receives the target cover image sent by the server, the client obtains the target cover image, etc. There is no specific limitation. The embodiments of the present application are introduced by taking the server sending the target cover image to the client as an example.

The process of determining the target cover image of the video to be processed by the server will be described in detail below. The process of determining the target cover image of the video to be processed by the client is similar, which will not be repeated here. Please refer to FIG. 2b.

S201, based on an information extraction strategy, extract key feature information contained in the video to be processed.

Based on the information extraction strategy, the key feature information contained in the video to be processed is extracted. The key feature information can be used to characterize the subject of the video to be processed, the key objects contained in the video to be processed, and the characteristics of the video to be processed. The subject, and the key objects that the video to process contains. The key feature information can include word features, face features, and object features.

There are many methods for extracting key feature information contained in the video to be processed, and three of them are used as examples to introduce below.

method one:

Obtain the release information and subtitle files of the video to be processed, and extract the word features of the keywords contained in the release information and subtitle files.

After obtaining the video to be processed, the server may obtain release information and subtitle files of the video to be processed. The publishing information of the to-be-processed video may include the publisher of the to-be-processed video, the release time, the title, the tag, the topic of participation, and the like. The server may determine the keywords contained in the release information of the video to be processed according to the title, tag, and topic of participation of the video to be processed. After obtaining the subtitle file of the video to be processed, the server may determine the keywords contained in the subtitle file of the video to be processed. Thus, keywords can characterize the subject of the video to be processed and the key objects contained in it to a certain extent.

The keyword can be the entity in the title, or the entity or sentence with the largest number in the subtitle file, or the subtitle corresponding to the video frame to be processed with the largest number of barrages, or the title, tag and participation. The names of people, animals, or intellectual property IPs contained in the topic, etc., can also be the names, animals, or intellectual property IPs with the largest number of people in the subtitle file.

The intellectual property IP can be the name of a TV series, a movie or an anime, etc. After obtaining the keywords contained in the release information of the video to be processed, the server may extract the word features of the keywords, and the word features uniquely characterize the keywords by quantifying the features of the keywords.

After obtaining the word feature of the keyword, the server may use the word feature as the key feature information contained in the video to be processed.

Method Two:

Acquire key video frames in the video to be processed, and extract the face features of the face region contained in the key video frames.

After the to-be-processed video is obtained, frame extraction can be performed on the to-be-processed video to obtain key video frames in the to-be-processed video, where the key video frames are video frames in the to-be-processed video that represent video scene switching. After obtaining the key video frame, the server may perform face detection processing on the key video frame to determine whether the key video frame contains a face area. The number of key video frames can be one or more. According to the key video frames containing the face region, the face features of the face region, such as face embedding, can be obtained.

After obtaining the face feature of the face region included in the key video frame, the server may use the face feature as the key feature information included in the video to be processed.

Method three:

Acquire key video frames in the video to be processed, and extract object features of object regions contained in the key video frames.

According to the introduction in Method 2, after obtaining the key video frame, the server can perform object detection processing on the key video frame to determine whether the key video frame contains an object area, and the object area is the area containing objects in the key video frame, such as dynamic Areas of objects, or areas containing static objects. The number of key video frames may be one or multiple, and according to the key video frames containing the object area, the object features of the object area can be obtained.

After obtaining the object feature of the object region included in the key video frame, the server may use the object feature as the key feature information included in the video to be processed.

As an embodiment, the server may use only one of the above methods, and use the obtained word feature, face feature or object feature as key feature information. The server may also use the above methods in combination, for example, taking word features and face features as key feature information; another example, taking face features and object features as key feature information; another example, taking word features and object features as key feature information Key feature information; for another example, word features, face features, and object features are used as key feature information.

S202: Obtain each candidate material that matches the key feature information, and select a candidate material that satisfies the presentation condition from each candidate material as a target material.

After obtaining the key feature information, the server may obtain each candidate material matching the key feature information. The server can determine each candidate material that matches the key feature information from each to-be-processed video frame contained in the video to be processed; it can also determine each reference image that matches the key feature information from each reference image collected from network resources. Each candidate material; each candidate material that matches the key feature information may also be determined from each to-be-processed video frame and each reference image, which is not specifically limited.

The following description will be given by taking as an example that the server determines each candidate material matching the key feature information from each video frame and each reference image.

The server can collect each reference image from the acquired network resources in real time, or collect each reference image in a preset period of time, and can also receive each reference image from the network resources sent by other devices, etc., which is not specifically limited. The reference image may be an image for which copyright content needs to be purchased, and the server obtains the reference image by purchasing the copyright content.

The server may extract respective video frame features of each to-be-processed video frame contained in the to-be-processed video, and extract respective image features of each reference image. There are many ways to extract video frame features and image features, for example, using CLIP model extraction; another example, using VGG16, Inception series models, ResNet and other classic models to extract; another example, using one-stage face detection network Retinaface , combined with the Asian face that often appears in the labeling task and the Arcface trained on the star face in the task scene, as well as the Resnet101 model extraction.

After obtaining each video frame feature and each image feature, the server may determine each candidate material matching the key feature information from each video frame to be processed and each reference image based on each video frame feature and each image feature. The server may, based on the to-be-processed video frames whose similarity between the video frame features and the key feature information is greater than the similarity threshold, and the to-be-processed video frames whose similarity between the image features and the key feature information is greater than the similarity threshold, determine the Each candidate material that matches the key feature information.

The server can also arrange each video frame to be processed based on the similarity between the video frame features and the key feature information in descending order of similarity, and based on the similarity between the image features and the key feature information, according to Arrange each reference image in descending order of similarity. The server determines each candidate material matching the key feature information based on the video frames to be processed and the reference images whose sequence numbers are arranged before the specified sequence numbers.

For example, taking the key feature information including word features, face features, and object features as an example, the server can determine the graphic similarity between the word feature and each video frame feature, as well as the graphic similarity between the word feature and each image feature. Spend. The similarity between pictures and texts can be calculated by using the CLIP model, so as to retrieve candidate materials that match the word features from each video frame to be processed and each reference image.

The server can also determine the face similarity between the face feature and each video frame feature, as well as the face similarity between the face feature and each image feature. Since the difference between the faces is relatively small, it can be The similarity of faces is determined by the combination of multiple models, and the retrieval accuracy is improved. For example, the one-stage face detection network Retinaface is used, and Arcface is trained by combining Asian faces that often appear in labeling tasks and star faces in task scenes. And the Resnet101 model determines the similarity of the face, and realizes the retrieval of candidate materials that match the face features from each video frame to be processed and each reference image.

The server may further determine the object similarity between the object feature and each video frame feature, and the object similarity between the object feature and each image feature. Because the differences between objects are relatively large, the traditional neural network model can be used to determine the similarity of objects, so as to retrieve candidate materials that match the characteristics of objects from each video frame to be processed and each reference image. Please refer to Figure 3, the server can use the object feature as a query condition, through the neural network model, determine the object similarity between the object feature and each video frame feature, and the object similarity between the object feature and each image feature, the similarity If the degree is greater than the threshold, the output is 1, and if the similarity is less than the threshold, the output is 0, and an output vector containing 0 or 1 is obtained. The output vector contains multiple element positions. The multiple element positions correspond to each video frame to be processed and each reference image. The element in the element position is 1, which represents the video frame feature of the corresponding video frame to be processed or the image feature of the reference image. , the similarity with the object feature is greater than the threshold; the element at the element position is 0, indicating that the video frame feature of the corresponding video frame to be processed or the image feature of the reference image, and the similarity between the object feature is not greater than the threshold. Therefore, the candidate material can be obtained based on the to-be-processed video frame or reference image corresponding to 1 in the output vector.

After the server obtains the similarity of each image and text, the similarity of each face, and the similarity of each object, it can select the similarity of each image and text, the similarity of each face and the similarity of each object based on the obtained similarity of each image and text, the similarity of each face and the similarity of each object. In the reference image, each candidate material that matches the key feature information is determined; it can also be determined from each video frame to be processed and each reference image based on the weighted sum of the corresponding graphic similarity, face similarity, and object similarity. Each candidate material that matches the key feature information. The server may use the image with the largest weighted sum as the candidate material, or may use several images with the largest weighted sum as the candidate material. For example, the Faiss library for clustering and similarity is used to provide efficient similarity search and clustering for dense vectors, support the search of billion-level vectors, and can very efficiently achieve vector retrieval and matching. For another example, the 01 vector is used to perform a correlation query, the similarity is marked as 1, and the dissimilarity is marked as 0, so that each candidate material matching the key feature information can be retrieved or matched.

As an example, in order to reduce the storage space occupied by each feature, dimensionality reduction processing can be performed on the video frame features and image features. For example, if the video frame features and image features are in the form of vectors, they can be converted from floating point vectors to 01Bit bits. 01 vector to achieve the purpose of dimensionality reduction and reduce the storage space occupied by each feature.

As an embodiment, the server may use candidate images matching the key feature information in each video frame to be processed and each reference image as candidate materials; it may also crop the candidate images matching the key feature information, and Cropped candidate images, as candidate materials, etc.

After the server determines each candidate image that matches the key feature information from each to-be-processed video frame and each reference image based on each video frame feature and each image feature, the server can determine each candidate image, each with the key feature information. matching image area. The server can use the target detection algorithm to determine the image area that matches the key feature information by using a rectangular frame; the server can also use the edge recognition algorithm to perform edge recognition on the target matching the key feature information, and determine the target according to the edge of the target. The key feature information is matched to the image area.

After each image area matching the key feature information is determined, the server may, based on each image area, perform cropping processing on each candidate image to obtain each candidate material. By removing unnecessary elements in the candidate image, the obtained candidate material can more accurately express the subject of the video to be processed and the key objects contained in the video to be processed.

After obtaining each candidate material matching the key feature information, the server may select a candidate material that satisfies the presentation condition from the various candidate materials as the target material. There are various methods for selecting candidate materials that meet the presentation conditions from each candidate material. For example, the server may perform a definition evaluation on each candidate material based on a definition evaluation strategy, and determine the respective definition evaluation value of each candidate material. The server may use candidate materials whose definition evaluation value is greater than the definition threshold as the target material; the server may also sort each candidate material based on the definition evaluation value, and use the candidate material before the specified serial number as the target material, etc.

For another example, based on the content quality evaluation strategy, the content quality evaluation is performed on each candidate material, and the respective content quality evaluation value of each candidate material is determined. The server may use the candidate materials whose content quality evaluation value is greater than the content quality threshold as the target material; the server may also sort each candidate material based on the content quality evaluation value, and use the candidate material before the specified serial number as the target material, etc. The content quality evaluation of each candidate material can evaluate the content related to advertising promotion, illegal content, etc. as a lower content quality evaluation value, so that among the target materials determined according to the content quality evaluation value, there is no content related to advertising promotion. Content, illegal content.

As an embodiment, after obtaining each definition evaluation value and each content quality evaluation value, the server may perform a weighted sum on the corresponding definition evaluation value and content quality evaluation value to obtain each weighted sum. After obtaining each weighted sum, the server can take the candidate material whose weighted sum is greater than the presentation threshold as the target material; the server can also sort each candidate material based on the weighted sum, and take the candidate material before the specified serial number as the target material .

S203, using the target material as a picture-in-picture image of the initial cover image, and synthesizing the target material and the initial cover image to generate a target cover image of the video to be processed.

After obtaining the target material, the server may use the target material as a picture-in-picture image of the initial cover image, synthesize the target material and the initial cover image, and generate the target cover image of the video to be processed. Please refer to FIG. 4a, which is the initial cover image of the video to be processed. The initial cover image only contains a scene where a woman is speaking, and cannot accurately represent the point of view of the video to be processed. According to the initial cover image, the network object cannot know the subject of the video to be processed or the key objects contained in it. It is very likely that the network object will not click to enter the pending video for viewing, thus missing the video of interest. It is also possible that the network object clicks to enter the video. Watch the video to be processed. After watching the video, it is known that the video to be processed is a video of no interest. Therefore, the network object needs to click to enter multiple videos for viewing before watching the video of interest.

After taking the target material as the picture-in-picture image of the initial cover image, synthesizing the target material and the initial cover image, and generating the target cover image of the video to be processed, please refer to Figure 4b, the target cover image contains a scene where a woman is talking, and a star A said that the target material does not look alike, then the target cover image can represent that in the to-be-processed video, the woman said something that does not match her appearance, and at the same time, it can also represent the participation of star A in the video. Therefore, the network object can accurately obtain the subject expressed by the to-be-processed video and the key objects contained in the to-be-processed video, so that the network object can more accurately obtain the video of interest for viewing through the target cover image.

As an example, using the target material as the picture-in-picture image of the initial cover image, there are various methods for synthesizing the target material and the initial cover image. The target material and the initial cover image; another example, overlay the target material on the position that does not contain the target object in the initial cover image, and synthesize the target material and the initial cover image; another example, obtain the target object contained in the initial cover image, The material and target objects are rearranged according to the pre-stored cover template, and the target material and the initial cover image are synthesized, and there are no specific restrictions.

The following is an example of the process of synthesizing the target material and the initial cover image by covering the target material on the position that does not contain the target object in the initial cover image.

The server may detect the initial cover image and determine the target object contained in the initial cover image. For example, the server uses the initial cover image as the input of the trained target detection model, and obtains the target object contained in the initial cover image output by the target detection model. The target object included in the initial cover image output by the target detection model may be marked with a rectangular frame, or marked with the edge of the target object, etc., which is not specifically limited.

The server may divide the initial cover image into a target area and a non-target area based on the position of the target object in the initial cover image. The target area is the area containing the target object in the initial cover image, which may be a rectangular area or an area enclosed by the edges of the target object. The non-target area is the area other than the target area in the original cover image.

The server can adjust the size of the target material based on the shape and size of the non-target area. The server can adjust the size of the target material with the target material filling the non-target area. For example, if the non-target area is a rectangular area and the shape of the target material is a circle, the long side of the rectangular area can be used as the diameter of the circle. , adjust the size of the target material.

If the content contained in the target material is the same object as the target object contained in the target area, the server may adjust the size of the target material by aiming to enlarge the target object by a specified multiple relative to the target object contained in the target area. Referring to Fig. 4c, the target material is the face area of a target object contained in the target area, then the target material can be enlarged to 2 times to highlight the facial expression of the target object. Since the scene is a funny scene, by highlighting the facial expressions of the target object, the purpose of enhancing the comedy effect and the sense of substitution can be achieved, so that the target cover image can accurately represent the funny theme of the video to be processed.

The server may also first determine the proportion of the target area to the initial cover image, and then adjust the size of the target material so that the proportion of the target material to the initial cover image is the same as the proportion of the target area to the initial cover image.

After adjusting the size of the target material, the adjusted target material is obtained. The server may overlay the adjusted target material on the non-target area in the initial cover image to generate the target cover image of the video to be processed.

As an embodiment, after determining the target object included in the initial cover image, the server may perform cropping processing on the initial cover image to obtain target object material including the target object. The server may generate a target cover image by combining the pre-stored cover template, and the obtained target material and target object material. In some cases, the background of the initial cover image is complex, which may easily cause confusion in the content of the obtained target cover image. Therefore, the target object in the initial cover image can be extracted, combined with a clear and concise cover template, and then the target material can be extracted. As a picture-in-picture image, superimposed on top of the cover template to generate the target cover image.

In the embodiments of the present application, various distributed systems can be used to invoke various services and work together to implement the method for generating a video cover image provided by the embodiments of the present application, please refer to FIG. 5 .

The network object can upload the video to be processed through the content provider to publish the video. Content providers include PGC, UGC, Multi-Channel Network (MCN), or professional user-generated content (Professional Generated Content+User Generated Content, PUGC) and other content production forms. The network object invokes the back-end Application Programming Interface (API) system through the mobile terminal or the client, uploads the video to be processed, and provides the server with local or captured video content or self-media articles or atlases written. The network object can upload the corresponding cover image while uploading the video to be processed, or can select the initial cover image from the video to be processed through the server.

The content provider first obtains the interface address of the upload server through communication with the upstream and downstream content interface services, then uploads the video to be processed, calls the content storage service, and stores the video to be processed in the content database. The upstream and downstream content interface services can also obtain the title, publisher, abstract, cover image, publishing time, etc. of the video to be processed from the content provider. When calling the content storage service, you can also store the meta information of the video to be processed, such as video file size, cover image link, bit rate, file format, title, release time, author, whether it is an original mark, and whether it is first published. in the content database. The upstream and downstream content interface services can submit the data stored in the content database to the dispatch center service, so that the dispatch center service can perform subsequent content processing and circulation.

Therefore, when a network object searches for a video, the content consumer can communicate with the content distribution export service to obtain index information corresponding to the searched video. By communicating with the content storage service, the streaming media file of the searched video corresponding to the index information is downloaded from the content database. Thus, streaming media files can be played through local players, or communicated with CDN services deployed at the edge to present graphic and text data.

The content consumer can report the behavior data, reading speed, completion rate, reading time, freeze, loading time, play clicks, etc. of network objects browsing videos to the server during the uploading and downloading process, so that the server can be based on the reported data. Network objects provide more user-friendly services.

The content consumer can browse videos through Feeds streaming, and provide a direct report and feedback portal for low-quality content. This portal is directly connected to the manual review system, and operators can confirm and review through the manual review system, which can be used as a follow-up upload for network objects The sample data for the quality filtering of the machine model during the video filtering process.

The dispatch center service mainly includes machine processing and the aforementioned manual review processing. Among them, machine processing can perform various quality judgments, such as low-quality content filtering; it can also mark content tags, such as content classification, topic information, etc.; Rearrangement, etc. The processing results obtained by the dispatch center service can be written into the content database.

The manual review process can be implemented by calling the manual review service through the manual review system. During the manual review process, the manual review system will read the information in the content database, and the results and status of the manual review will also be returned to the content database. Therefore, the content database also includes the classification of content during the manual review process, including first, second, and third-level classification and label information, for example, a video explaining a brand mobile phone, the first category is technology, the second category is smartphones, and the third category is smartphones. The class classification is the domestic mobile phone, and the label information is the brand, model, etc. The dispatch center service can specify different tonal enhancement template policies according to the label information of the first-level classification of each video in the content database.

The manual review service can be a WEB system that takes the results of machine filtering as input on the link, manually confirms and reviews the results of machine filtering, and writes the review results into the content database for records. The results are used to evaluate the actual effect of the machine filtering model online.

The scheduling center service can call the content reordering service when performing content reordering. The content reordering service mainly includes title removal, cover image removal, content text removal, and video fingerprint and audio fingerprint removal. The process of deduplication is usually to vectorize the title and text of the graphic content, using simmhash and BERT text vector. When deduplicating the image vector, the video fingerprint and the audio fingerprint are extracted from the video content to construct a vector, and then the distance between the vectors, such as the Euclidean distance, is calculated to determine whether it is repeated. Content sorting can reduce the amount of content review and ensure that there is only one copy of the same content in the recommended distribution pool, ensuring user experience.

The scheduling center service is mainly responsible for the entire scheduling process of video and graphic content flow. It receives uploaded videos to be processed through the upstream and downstream content interface services, and then obtains the meta information of the videos from the content database. The dispatch center service can call the picture-in-picture service when receiving the uploaded video to be processed, generate the target cover image for the video to be processed, and store it in the content database; Call the picture-in-picture service to generate the target cover image for the video to be processed, and present it through the content consumer.

The picture-in-picture service can call the picture material extraction service, determine the candidate images that match the key feature information based on the key feature information of the video to be processed, and call the selection and capture service of the cover image, where the cover image capture service mainly uses the original image size. and the cropped target size, use the person detection, subject detection, and OCR text recognition of the candidate image to take corresponding screenshots to obtain candidate materials.

After obtaining the candidate material, you can call the service of selecting and intercepting the cover image. The cover image selection service mainly filters and selects the candidate material according to basic quality characteristics such as clarity, aesthetics, unsuitable for pictures, mosaics, vulgar pornography, etc. Screen and remove some low-quality candidates that are not suitable for cover.

After selecting the target material that meets the presentation conditions based on each candidate material, the obtained target material can be stored in the enhanced picture material library, so as to be used for the cover image of the subsequent video. The enhanced image material library is used to save a candidate set for image enhancement, including the content of frame extraction of video content and the composition of purchased copyright content. After obtaining the target material, the template database can be called to select the target template. The picture-in-picture service can generate the target cover image of the video to be processed by synthesizing the target material and the initial cover image of the video to be processed based on the target template. The initial cover image may be the cover image uploaded by the content provider, or the cover image selected from each video frame of the video to be processed by invoking the selection and interception cover image service, which is not specifically limited.

The core synthesis principle of the template database is that the main elements of the screen should not be blocked. The main target detection can be used to determine several non-target areas, such as the left, right or upper or lower strategies, which can be determined by the actual situation. The template database can intelligently enhance service communication, provide policy presentation, and if there is text, it also includes font and style configuration policies for synthesized text.

The intelligent picture-in-picture service can communicate with the dispatch center service to complete the method for generating a video cover image provided by the embodiment of the present application. The dispatch center service includes extracting title keywords and entity words, then communicating with the screen material extraction service to complete the screening and matching of the target material, and finally generating the target cover image to achieve the output enhancement effect through the target material.

The following describes an example of the method for generating a video cover image provided by the embodiment of the present application, please refer to FIG. 6a.

After obtaining the video to be processed and the initial cover image of the video to be processed, the client can extract the key features contained in the video to be processed based on the video to be processed and the release information and subtitle file of the video to be processed, or the release information and subtitle file information. The key feature information may include one or more of word features, face features, or object features. For example, the key feature information is the features of entities such as words, names, and animals.

After obtaining one or more of word features, face features or object features, the client can determine each candidate image matching the key feature information from each reference image and each video frame to be processed, and apply Target recognition processing is performed on each candidate image, and the target in each candidate image is cropped to obtain each candidate material.

After obtaining each candidate material that matches the key feature information, the client can select a candidate material that satisfies the presentation condition from each candidate material as a target material. For example, based on the definition evaluation strategy and the content quality evaluation strategy, the definition evaluation and the content quality evaluation are performed on each candidate material, and the candidate material with relatively clear and high content quality is selected. After obtaining the selected candidate materials with relatively clear and high content quality, the client can also perform some other post-processing on these candidate materials to obtain the target materials. For example, according to the area of the initial cover image that does not contain the target object , adjust the size of these candidate materials to obtain the target material, so that when the target material is added as a picture-in-picture image to the initial cover image, it can not affect the content expressed by the initial cover image, and can further express the video to be processed The subject or key objects contained in the video are emphasized, and the subject or key subject contained in the video to be processed plays a role of emphasis.

The number of target materials can be one or more. The target material, as a picture-in-picture image, can be added to the initial cover image in various ways. It can be added randomly, or it can be filled in the initial cover image. The non-target area is added for the target, or multiple target materials can be added as a whole to the initial cover image, or the corresponding target material can be added to the position specified by the template by acquiring the pre-stored addition template, etc. No restrictions.

The following takes a short video as an example to introduce an example of the method for generating a video cover image provided by the embodiment of the present application.

For example, a short video introduces its point of view through the title, "The little raccoon eats grapes on the sofa and finds that there are no grapes in the bowl. I can laugh for a year." And usually located in an inconspicuous position, please refer to Figure 6b, a video frame contained in the short video can only express the scene of the little raccoon eating grapes on the sofa, and cannot intuitively and accurately convey the subject of the short video to the network object, namely A comparison of the reactions of raccoons when they ate grapes and when they finished eating grapes.

After obtaining the short video and determining the initial cover image of the short video, referring to FIG. 6c, the server can extract the key feature information of the short video based on the title of the short video and the video frames contained in the short video. The key feature information may include the word feature of the keyword, that is, the word feature of the entity "little raccoon" in the title, and the object feature of the key object, that is, the object feature of the object area of the little raccoon in the video frame.

The server can use the word feature of the keyword "little raccoon" and the object area of the key object "little raccoon" as the subject of the query, and in the video frames included in the short video, determine candidate images that match the keyword, and the key object. Each candidate image is obtained by matching the candidate images.

After obtaining each candidate image, please refer to FIG. 6d, the server can perform deduplication processing on each candidate image, and according to the image area of each candidate image that matches the keyword, or the image area that matches the key object, the Each candidate image is cropped to obtain each candidate material.

The server may perform a definition evaluation on each candidate material based on the definition evaluation strategy, determine the respective definition evaluation value of each candidate material, and based on the content quality evaluation strategy, perform content quality evaluation on each candidate material, and determine the respective quality of each candidate material. Content quality evaluation value. Finally, based on the weighted sum of the corresponding clarity evaluation value and the content quality evaluation value, the candidate material with the largest weighted sum is selected as the target material.

After the server obtains the initial cover image and the target material, referring to Figure 6e, the server can perform target detection processing on the initial cover image to determine the non-target area that does not contain the target object in the initial cover image, that is, the area where the raccoon is located in the initial cover image outside the area.

The server can overlay the target material on the non-target area, and combine the pre-stored cover template to synthesize the target material and the initial cover image to generate the target cover image.

Based on the same inventive concept, an embodiment of the present application provides an apparatus for generating a video cover image, which can implement the functions corresponding to the foregoing method for generating a video cover image. Referring to FIG. 7, the apparatus includes an acquisition module 701 and a processing module 702, wherein:

Obtaining module 701: used to obtain the video to be processed and the initial cover image of the video to be processed;

Processing module 702: Based on the video to be processed and the initial cover image, generate a target cover image of the video to be processed;

The target cover image is generated by adding a picture-in-picture image to the initial cover image, and the picture-in-picture image matches the content of the video to be processed.

In a possible embodiment, the processing module 702 is specifically configured to:

Based on the information extraction strategy, extract the key feature information contained in the video to be processed, wherein the key feature information is used to characterize: the subject of the video to be processed, and the key objects contained;

Obtain each candidate material that matches the key feature information, and select the candidate material that meets the presentation conditions from each candidate material as the target material;

The target material is used as the picture-in-picture image of the initial cover image, and the target material and the initial cover image are synthesized to generate the target cover image.

In a possible embodiment, the processing module 702 is specifically configured to:

obtaining each reference image, wherein the reference image is an image collected from network resources;

Extracting the respective video frame features of each to-be-processed video frame contained in the video to be processed, and extracting the respective image features of each reference image;

Based on each video frame feature and each image feature, each candidate material matching the key feature information is determined from each video frame to be processed and each reference image.

In a possible embodiment, the key feature information includes: word feature, face feature and object feature;

Then the processing module 702 is specifically used for:

Determine the graphic similarity between the word feature and each video frame feature, as well as between the word feature and each image feature;

Determine the face similarity between the face feature and each video frame feature, and between the face feature and each image feature;

Determine the object similarity between the object feature and each video frame feature, and between the object feature and each image feature;

Based on the obtained image-text matching degree, each face matching degree and each object matching degree, each candidate material matching the key feature information is determined from each to-be-processed video frame and each reference image.

In a possible embodiment, the processing module 702 uses the following method to obtain the key feature information:

Obtain the release information and subtitle files of the video to be processed, and extract the word features of the keywords contained in the release information and subtitle files;

Acquire key video frames in the video to be processed, and extract the face features of the face region contained in the key video frames and the object features of the object regions contained in the key video frames, wherein the key video frames are in the video to be processed and represent the video scene switching. video frame;

The obtained word features, face features and object features are used as key feature information.

In a possible embodiment, the processing module 702 is specifically configured to:

Based on each video frame feature and each image feature, from each to-be-processed video frame and each reference image, determine each candidate image that matches the key feature information;

Determine each candidate image separately, and each image area that matches the key feature information;

Based on each image area, each candidate image is cropped to obtain each candidate material.

In a possible embodiment, the processing module 702 is specifically configured to:

Based on the sharpness evaluation strategy, evaluate the sharpness of each candidate material, and determine the respective sharpness evaluation value of each candidate material;

Based on the content quality evaluation strategy, perform content quality evaluation on each candidate material, and determine the content quality evaluation value of each candidate material;

A weighted sum of each clarity evaluation value and each content quality evaluation value is obtained, and the weighted sum conforms to the candidate material greater than the presentation threshold as the target material.

In a possible embodiment, the processing module 702 is specifically configured to:

Detect the initial cover image, and determine the target object contained in the initial cover image;

Divide the initial cover image into target area and non-target area based on the position of the target object in the initial cover image;

Adjust the size of the target material based on the shape and size of the non-target area;

The adjusted target material is overlaid on the non-target area in the initial cover image to generate the target cover image of the video to be processed.

Please refer to FIG. 8 , the above-mentioned device for generating a video cover image can run on a computer device 800, and the current version and historical version of the data storage program and application software corresponding to the data storage program can be installed on the computer device 800, and the computer device 800 includes Processor 880 and memory 820. In some embodiments, the computer device 800 may include a display unit 840 including a display panel 841 for displaying an interface and the like for user interaction.

In a possible embodiment, the display panel 841 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD) or an organic light-emitting diode (OLED) (Organic Light-Emitting Diode).

The processor 880 is used to read the computer program, and then execute the method defined by the computer program, for example, the processor 880 reads the data storage program or file, etc., so as to run the data storage program on the computer device 800, and display the corresponding data on the display unit 840. interface. The processor 880 may include one or more general-purpose processors, and may further include one or more DSPs (Digital Signal Processors, digital signal processors), which are configured to perform related operations to implement the technical solutions provided by the embodiments of the present application.

The memory 820 generally includes internal memory and external memory. The internal memory can be random access memory (RAM), read only memory (ROM), and cache memory (CACHE). The external storage can be a hard disk, an optical disk, a USB disk, a floppy disk or a tape drive. The memory 820 is used to store computer programs and other data, the computer programs include application programs corresponding to each client, and the other data may include data generated after the operating system or the application program is executed, and the data includes system data (such as operating system data). configuration parameters) and user data. In this embodiment of the present application, the program instructions are stored in the memory 820, and the processor 880 executes the program instructions in the memory 820 to implement any method for generating a video cover image discussed in the preceding figures.

The above-mentioned display unit 840 is used for receiving input digital information, character information or contact touch operation/non-contact gesture, and generating signal input related to user settings and function control of the computer device 800 . Specifically, in this embodiment of the present application, the display unit 840 may include a display panel 841 . The display panel 841 is, for example, a touch screen, which can collect the user's touch operations on or near it (such as the user's operations on the display panel 841 or on the display panel 841 using a finger, a stylus, and any other suitable objects or accessories), and according to preset The specified program drives the corresponding connection device.

In a possible embodiment, the display panel 841 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation of the player, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and sends it to the touch controller. To the processor 880, and can receive the command sent by the processor 880 and execute it.

Among them, the display panel 841 can be realized by various types such as resistive type, capacitive type, infrared ray and surface acoustic wave. In addition to the display unit 840, in some embodiments, the computer device 800 may also include an input unit 830, and the input unit 830 may include an image input device 831 and other input devices 832, wherein the other input devices may include, but are not limited to, a physical keyboard, function keys One or more of (such as volume control buttons, switch buttons, etc.), trackball, mouse, joystick, etc.

In addition to the above, computer device 800 may also include power supply 890 for powering other modules, audio circuitry 860, near field communication module 870, and RF circuitry 810. Computer device 800 may also include one or more sensors 850, such as acceleration sensors, light sensors, pressure sensors, and the like. The audio circuit 860 specifically includes a speaker 861, a microphone 862, and the like. For example, the computer device 800 can collect the user's voice through the microphone 862, and perform corresponding operations.

As an embodiment, the number of processors 880 may be one or more, and the processors 880 and the memory 820 may be coupled or relatively independent.

As an embodiment, the processor 880 in FIG. 8 may be used to implement the functions of the acquiring module 701 and the processing module 702 in FIG. 7 .

As an embodiment, the processor 880 in FIG. 8 may be used to implement the functions corresponding to the server or the terminal device discussed above.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, execute Including the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk and other various A medium on which program code can be stored.

Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of software products in essence or in the parts that make contributions to the prior art, for example, embodied in a computer program product, and the computer program product is stored in a storage The medium includes several instructions for causing a computer device to perform all or part of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic disk or an optical disk and other mediums that can store program codes.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (12)

1. a method for generating a video cover image, is characterized in that, comprising: Obtain the video to be processed, and the initial cover image of the video to be processed; generating a target cover image of the video to be processed based on the video to be processed and the initial cover image; The target cover image is generated by adding a picture-in-picture image to the initial cover image, and the picture-in-picture image matches the content of the video to be processed. 2. The method according to claim 1, wherein generating a target cover image of the video to be processed based on the video to be processed and the initial cover image, comprising: Based on the information extraction strategy, extract the key feature information contained in the video to be processed, wherein the key feature information is used to represent: the subject of the video to be processed, and the key objects contained; Acquire each candidate material that matches the key feature information, and select a candidate material that satisfies the presentation conditions from the various candidate materials as the target material; Using the target material as a picture-in-picture image of the initial cover image, synthesizing the target material and the initial cover image to generate the target cover image. 3. The method according to claim 2, wherein acquiring each candidate material matching the key feature information comprises: obtaining each reference image, wherein the reference image is an image collected from a network resource; Extracting the respective video frame features of each to-be-processed video frame contained in the video to be processed, and extracting the respective image features of the respective reference images; Based on each video frame feature and each image feature, from each of the video frames to be processed and each of the reference images, each candidate material matching the key feature information is determined. 4. The method according to claim 3, wherein the key feature information comprises: word feature, face feature and object feature; Then, based on each video frame feature and each image feature, from the video frames to be processed and the reference images, determine each candidate material that matches the key feature information, including: Determine the graphic similarity between the word feature and each video frame feature, and between the word feature and each image feature, respectively; Respectively determine the facial similarity between the facial feature and each video frame feature, and between the facial feature and each image feature; respectively determining the object similarity between the object feature and each video frame feature, and between the object feature and each image feature; Based on the obtained image-text matching degree, each face matching degree and each object matching degree, each candidate material matching the key feature information is determined from each of the video frames to be processed and each of the reference images. 5. The method according to claim 4, wherein the key feature information is obtained by adopting the following method: Obtain the release information and the subtitle file of the video to be processed, and extract the word features of the keywords contained in the release information and the subtitle file; Acquiring key video frames in the video to be processed, and extracting the facial features of the face region included in the key video frames, and the object features of the object regions included, wherein the key video frames are the to-be-processed In the video, the video frame representing the video scene switching; The obtained word features, face features and object features are used as the key feature information. 6 . The method according to claim 3 , wherein, based on each video frame feature and each image feature, from the video frames to be processed and the reference images, it is determined that the key feature information matches the key feature information. 7 . candidate materials, including: Based on each video frame feature and each image feature, from each of the video frames to be processed and each of the reference images, determine each candidate image that matches the key feature information; Determining the respective candidate images, each of which matches the image area with the key feature information; Based on the respective image regions, the respective candidate images are cropped to obtain respective candidate materials. 7. The method according to claim 2, characterized in that, selecting candidate materials that meet the presentation conditions from the respective candidate materials as target materials, comprising: Based on the sharpness evaluation strategy, perform sharpness evaluation on the respective candidate materials, and determine the respective sharpness evaluation values of the respective candidate materials; Based on the content quality evaluation strategy, perform content quality evaluation on each candidate material, and determine the respective content quality evaluation value of each candidate material; A weighted sum of each clarity evaluation value and each content quality evaluation value is obtained, and a candidate material whose weighted sum is greater than the presentation threshold is used as the target material. 8 . The method according to claim 2 , wherein the target material is used as a picture-in-picture image of the initial cover image, and the target material and the initial cover image are synthesized to generate the target cover image. 9 . ,include: Detecting the initial cover image, and determining the target object contained in the initial cover image; dividing the initial cover image into a target area and a non-target area based on the position of the target object in the initial cover image; Adjusting the size of the target material based on the shape and size of the non-target area; The adjusted target material is overlaid on the non-target area in the initial cover image to generate the target cover image. 9. A device for generating a video cover image, comprising: Acquisition module: used to acquire the video to be processed and the initial cover image of the video to be processed; Processing module: based on the video to be processed and the initial cover image, generate a target cover image of the video to be processed; The target cover image is generated by adding a picture-in-picture image to the initial cover image, and the picture-in-picture image matches the content of the video to be processed. 10. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented. 11. A computer equipment, characterized in that, comprising: memory for storing program instructions; The processor is configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 8 according to the obtained program instructions. 12. A computer-readable storage medium, wherein the storage medium stores computer-executable instructions, the computer-executable instructions being used to cause a computer to execute the method according to any one of claims 1 to 8 .

Images