CN115114479A - Method and device for generating video label, storage medium and electronic device - Google Patents

Abstract

The invention discloses a method and device for generating a video label, a storage medium and an electronic device. Wherein, the method includes: acquiring a target video to be identified, wherein the target video carries a video description text for describing the target video; constructing a target candidate font library of the target video by using the video description text; extracting the video features of the target video and In the case of the description features of the video description text, the video features and the description features are iteratively calculated for N times to obtain M candidate character sequences, wherein the candidate character objects in each candidate character sequence include each candidate character in the target candidate character library. character and the confidence of matching the candidate characters; determine the target character from each candidate character sequence based on the confidence, and obtain M target characters; splicing the M target characters into target video tags matching the target video. The invention solves the technical problem of low video label accuracy determined by the prior method.

Description

Method and device for generating video label, storage medium and electronic device

technical field

The present invention relates to the field of computers, and in particular, to a method and apparatus for generating a video tag, a storage medium and an electronic device.

Background technique

With the development of network video platforms, network users are more and more accustomed to obtaining the latest network information by browsing videos. In the process of disseminating online videos on the Internet, video platforms usually match video tags for videos, and then implement video recommendation and video search functions based on video tags. For online videos, fresh and highly matched video tags can enable online users to discover new hot events at the first time, thereby increasing the click-through rate of new hot online videos.

The existing method for matching tags of online video, usually in the matching process, performs multiple classification and clustering operations on the extracted video features, and then determines the tag matching the video in the defined tag system.

It can be seen that the existing label determination method can only select the label matching the video in the preset label system, and for the video whose content is a fresh event, because the preset label library does not include the label corresponding to the fresh event. Fresh hashtags entries, so it is not possible to match fresh hashtags for fresh videos. As a result, in the process of providing video recommendation and video search functions based on tags, the platform cannot achieve accurate recommendation or provide accurate search results. That is to say, the existing method for determining video tags has the technical problem of low accuracy of the determined tags.

For the above problems, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a method and device for generating a video tag, a storage medium and an electronic device, so as to at least solve the technical problem of the low accuracy rate of the video tag determined by the existing method.

According to an aspect of the embodiments of the present invention, a method for generating a video tag is provided, including: acquiring a target video to be identified, wherein the target video carries video description text for describing the target video; using the video description The text constructs the target candidate character library of the above-mentioned target video, wherein, the above-mentioned target candidate character library includes a reference character set and a description character set, the above-mentioned reference character set includes a plurality of historical tag texts. Descriptive characters obtained by segmenting the video description text; in the case of extracting the video features of the above-mentioned target video and the description features of the above-mentioned video description text, the above-mentioned video features and the above-mentioned description features are iteratively calculated for N times to obtain M candidates Character sequence, wherein, the candidate character object in each of the above-mentioned candidate character sequence includes each candidate character in the above-mentioned target candidate character library and the confidence level of matching with the above-mentioned candidate character, and the above-mentioned confidence level is used to indicate the above-mentioned candidate character and the above-mentioned target video. The matching degree between the two, N and M are natural numbers greater than or equal to 1; the target character is determined from each of the above-mentioned candidate character sequences based on the above-mentioned confidence degree, and M above-mentioned target characters are obtained; M above-mentioned target characters are spliced into the above-mentioned target characters The target video tag for video matches.

According to another aspect of the embodiments of the present invention, there is also provided an apparatus for generating a video tag, including: an acquisition unit configured to acquire a target video to be identified, wherein the target video carries a video for describing the target video A description text; a construction unit, used to construct a target candidate character library of the above-mentioned target video by using the above-mentioned video description text, wherein, the above-mentioned target candidate character library includes a reference character set and a description character set, and the above-mentioned reference character set includes a plurality of historical label texts for word segmentation The obtained reference character, the above-mentioned description character set includes the description character obtained by word segmentation on the above-mentioned video description text; a computing unit is used to extract the video feature of the above-mentioned target video and the description feature of the above-mentioned video description text. The video features and the above-mentioned description features are calculated N times of iterations to obtain M candidate character sequences, wherein the candidate character objects in each of the above-mentioned candidate character sequences include each candidate character in the above-mentioned target candidate character library and the above-mentioned candidate character matching. confidence, the confidence is used to indicate the degree of matching between the candidate characters and the target video, and N and M are natural numbers greater than or equal to 1; a determination unit is used to determine from each of the candidate character sequences based on the confidence extracting target characters to obtain M above-mentioned target characters; a splicing unit for splicing the M above-mentioned target characters into target video tags matching the above-mentioned target videos.

According to yet another aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, where a computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the above-mentioned video tagging process when running. generate method.

According to yet another aspect of the embodiments of the present application, there is provided a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the video tag generation method as above.

According to yet another aspect of the embodiments of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the above-mentioned generation of the video tag through the computer program method.

In the embodiment of the present invention, the target video to be identified is acquired; a target candidate character library of the target video is constructed by using the video description text, wherein the target candidate character library includes a reference character set and a description character set, and the reference character set includes a plurality of historical tags. The reference characters obtained by word segmentation of the text, the description character set includes the description characters obtained by word segmentation of the video description text; in the case of extracting the video features of the target video and the description features of the video description text, the video features and description features N Iterative calculation to obtain M candidate character sequences; target characters are determined from each candidate character sequence based on confidence, and M target characters are obtained; then M target characters are spliced into target video tags matching the target video. That is to say, in this embodiment of the present application, by iteratively calculating the video features and the description text features of the target video, the target characters are output word by word from the description character set composed of the reference character set and the video description text, and then the target characters are output through the output The target characters make up the target video tag. Since the target candidate character library used for outputting characters in the embodiment of the present application combines the description character set, and the description character set includes fresh text content related to the video, the fresh characters can be output in combination with the description character set, and then according to the output freshness Character splicing obtains fresh labels, which improves the accuracy of output video labels, and solves the technical problem of low accuracy of video labels obtained by existing video label determination methods.

Description of drawings

The accompanying drawings described herein are used to provide a further understanding of the present invention, and constitute a part of the present application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the attached image:

1 is a schematic diagram of a hardware environment of a method for generating an optional video tag according to an embodiment of the present invention;

2 is a flowchart of an optional method for generating a video tag according to an embodiment of the present invention;

3 is a schematic diagram of an optional method for generating a video tag according to an embodiment of the present invention;

4 is a schematic diagram of another optional method for generating a video tag according to an embodiment of the present invention;

5 is a schematic diagram of an optional method for generating a video tag according to an embodiment of the present invention;

6 is a schematic diagram of another optional method for generating a video tag according to an embodiment of the present invention;

Fig. 7 is the flow chart of the generation method of another kind of optional video label according to an embodiment of the present invention;

8 is a schematic structural diagram of a device for generating an optional video tag according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an optional electronic device according to an embodiment of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, eg, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

According to an aspect of the embodiments of the present invention, a method for generating a video tag is provided. As an optional implementation manner, the above-mentioned method for generating a video tag may be applied, but not limited to, the method shown in FIG. 1 by the server 102 and the In the video tag generation system composed of the terminal device 104 . As shown in FIG. 1 , the server 102 is connected to the terminal device 104 through the network 110. The above-mentioned network may include, but is not limited to, a wired network and a wireless network, wherein the wired network includes: a local area network, a metropolitan area network, and a wide area network, and the wireless network Including: Bluetooth, WIFI and other networks that realize wireless communication. The above-mentioned terminal devices may include but are not limited to at least one of the following: mobile phones (such as Android mobile phones, iOS mobile phones, etc.), notebook computers, tablet computers, PDAs, MIDs (Mobile Internet Devices, mobile Internet Devices), PADs, desktop computers, smart TV, car equipment, etc. A client, such as a video sharing client, may be installed on the above-mentioned terminal device. The above-mentioned terminal equipment is also provided with a display, a processor and a memory. The display can be used to display the program interface of the video uploading application and display the video content of the uploading server. Processing, for example, compressing the obtained video file; the memory is used for storing the video to be uploaded. It can be understood that, after acquiring the target video to be uploaded in the above-mentioned terminal device 104, the terminal device 104 can send the above-mentioned target video to the server 102 through the network 110, and when the server 102 receives the target video, upload the target video according to the terminal device 104. The video generates a video tag prompt tag matching the target video, and the terminal device 104 can receive the video tag returned by the server 102 through the network 110 . The server 102 may be a single server, a server cluster composed of multiple servers, or a cloud server. The above server includes a database and a processing engine. Wherein, the above-mentioned database may include a historical label thesaurus for matching labels for videos and a pre-trained label generation model; the above-mentioned processing engine is used to generate target video labels according to the acquired target videos.

According to an aspect of the embodiments of the present invention, the above-mentioned video tag generation system may further perform the following steps: the terminal device 104 performs step S102 to obtain the target video to be identified; The target video; the server 102 executes steps S106 to S114 to obtain the target video to be identified and use the video description text to construct a target candidate character library of the target video, wherein the target candidate character library includes a reference character set and a description character set, and the reference character set includes a plurality of The reference characters obtained by word segmentation of the historical label text. The description character set includes the description characters obtained by word segmentation of the video description text; when the video features of the target video and the description features of the video description text are extracted, the video features and description features Carry out N times of iterative calculation to obtain M candidate character sequences, wherein the candidate character objects in each candidate character sequence include each candidate character in the target candidate character library and the confidence level of the candidate character matching, and the confidence level is used to indicate the candidate character. The matching degree between the character and the target video, N and M are natural numbers greater than or equal to 1; the target character is determined from each candidate character sequence based on the confidence, and M target characters are obtained; M target characters are spliced into the target The target video tag of the video matching; then step S116 is executed, the server 102 sends the target video tag to the terminal device 104 through the network 110 . It can be understood that, in the case where the terminal device 104 is a device with sufficient computing processing capability, the above steps S106 to S114 can also be performed in the terminal device 104 .

In the embodiment of the present invention, the target video to be identified is acquired; a target candidate character library of the target video is constructed by using the video description text, wherein the target candidate character library includes a reference character set and a description character set, and the reference character set includes a plurality of historical tags. The reference characters obtained by word segmentation of the text, the description character set includes the description characters obtained by word segmentation of the video description text; in the case of extracting the video features of the target video and the description features of the video description text, the video features and description features N Iterative calculation to obtain M candidate character sequences; target characters are determined from each candidate character sequence based on confidence, and M target characters are obtained; then M target characters are spliced into target video tags matching the target video. That is to say, in this embodiment of the present application, by iteratively calculating the video features and the description text features of the target video, the target characters are output word by word from the description character set composed of the reference character set and the video description text, and then the target characters are output through the output The target characters make up the target video tag. Since the target candidate character library used for outputting characters in the embodiment of the present application combines the description character set, and the description character set includes fresh text content related to the video, the fresh characters can be output in combination with the description character set, and then according to the output freshness Character splicing obtains fresh labels, which improves the accuracy of output video labels, and solves the technical problem of low accuracy of video labels obtained by existing video label determination methods.

The above is only an example, which is not limited in this embodiment.

As a kind of optional embodiment, as shown in Figure 2, the generation method of above-mentioned video label comprises the following steps:

S202, obtain the target video to be identified, wherein, the target video carries a video description text for describing the target video;

It should be noted that the above-mentioned video description information may include, but is not limited to, the video title information of the target video, the subtitle information of the target video, the label information of the target video, and the text information that appears in the picture in the target video. One or more. A textual information used to describe the target video content. The specific text information is not limited here.

S204, using the video description text to construct a target candidate character library of the target video;

The target candidate character library includes a reference character set and a description character set, the reference character set includes reference characters obtained by word segmentation of multiple historical tag texts, and the description character set includes description characters obtained by word segmentation of video description text.

The above method will be described below with a specific example. Assuming that in the operation of historically generating tags, it has been determined that the following tags can be used to classify videos: "game", "love", "tv series", and then based on the historically generated tags, it can be determined that the reference character set is "tourism, drama, love, affection, television, television, drama”;

Assuming that the acquired description text of the target video to be identified is a subtitle in the target video "The city hero game is so fun, let's play it!", and then based on the above description text, it can be determined that the description character set is "city, city , hero, hero, game, play, really, good, play, big, home, come, come, play, let's go".

Combining the above reference character set and the above description character set, the target candidate character library "game, game, love, affection, television, TV, drama, city, city, hero, hero, game, game, true, good, Play, big, home, come, come, play, let's go".

The above-mentioned character library includes the characters "Game, Game" in the description character set and the characters "Game, Game" in the reference character set. In an optional manner, the repeated characters in the above-mentioned character library can also be de-duplicated, so as to obtain the updated target candidate character library "Game, Game, Love, Love, TV, TV, Drama, City, City, Hero, hero, true, good, play, big, home, fast, come, play, let’s go”.

It can be understood that the above-mentioned method of constructing the target candidate font library is only an example, and in the specific implementation manner, the above-mentioned method of determining the target candidate font library can also be other similar methods. be limited.

S206, in the case of extracting the video features of the target video and the description features of the video description text, perform N times of iterative calculation on the video features and the description features to obtain M candidate character sequences, wherein, in each candidate character sequence, The candidate character object includes each candidate character in the target candidate character library and the confidence level of matching the candidate character;

It can be understood that the above confidence level is used to indicate the degree of matching between the candidate character and the target video, and N and M are natural numbers greater than or equal to 1.

It can be understood that the above-mentioned video features may be visual features used to characterize the style of video pictures, and may also be video content features used to characterize video picture content, and the content of the specific representations of the above-mentioned video features is not limited here; the above description The feature is the text feature corresponding to the video description text, which can be the title text feature, the subtitle text feature, or the picture text feature included in the video picture. The specific content of the above video description text feature is not limited here.

It can be understood that, in the above-mentioned iterative calculation process, the above-mentioned video features and description text features can be converted into corresponding vectors, so as to facilitate the iterative calculation. At the same time, the above iterative calculation indicates that the input value of the current iterative calculation is related to the output value of the previous iterative calculation, that is, in the above-mentioned embodiment of the present application, the subsequent result output by the iterative calculation is related to the previous result of the iterative calculation sex.

S208, determine target characters from each candidate character sequence based on the confidence, and obtain M target characters;

The above-mentioned candidate character sequences will be described below. It can be understood that the above candidate character sequence is composed of the confidence level of each character matching in the target candidate character library. Continue to take the above-mentioned candidate fonts as an example to explain .

Assuming that the first iterative calculation is performed on the video features and description features of the above video, the obtained first candidate character sequence is: "you (98%), play (90%), love (10%), love ( 10%), electricity (5%), TV (8%), drama (8%), city (58%), city (60%), English (88%), male (70%), true (60%) ), good (70%), play (60%), big (28%), home (5%), fast (18%), come (8%), play (68%), let’s (1%)” ;

Assume that the rule for outputting characters is to output the character with the highest confidence, and then it can be determined that the first character output by the current sequence is "you" according to the above-mentioned candidate sequence;

Assume that the video features and description features of the above videos are combined with the results of the previous iteration to perform the second iterative calculation. It is assumed that the obtained second candidate character sequence is: "you (70%), play (98%), love ( 10%), love (10%), electricity (5%), TV (8%), drama (8%), city (58%), city (60%), British (88%), male (70%) ), true (60%), good (70%), play (60%), big (28%), home (5%), fast (18%), come (8%), play (68%), bar(1%)";

Assume that the rule for outputting characters is to output the character with the highest confidence, and then it can be determined that the second character output by the current sequence is "play" according to the above-mentioned candidate sequence;

Perform multiple iterations according to the above method, assuming that the final output M characters are "you", "play", "city", "shi", "ying", "xiong", "good", "you", " play".

S210, splicing M target characters into target video tags matching the target video;

Continuing to describe the step with the above embodiment, the obtained M target characters are "you", "play", "city", "city", "ying", "xiong", "good", "game" In the case of "play" and "play", the above characters can be spliced in the order of output to obtain three labels "game", "city hero" and "good game".

In an optional manner, in the process of outputting the above-mentioned characters, the output identifier can also be used to prompt the splicing method of the target video tag. For example, the obtained M target characters are "you", "play", "CLS", "city", "city", "ying", "xiong", "CLS", "good", "you" , in the case of "play", the characters "you", "play" can be changed according to the interval identifier "CLS" between "you", "play" and "city", "city", "ying" and "xiong" "The "game" spliced into the first target tag, the "city hero" spliced into "city", "city", "ying", "xiong" as the second target tag, and the interval identifier "CLS" The "good game" formed by splicing "good", "you" and "play" after that is used as the third target tag.

In another optional manner, the obtained M target characters are disordered characters "you", "you", "play", "city", "ying", "city", "male", In the case of "good" and "game", according to semantic analysis, it can be determined that "game" and "play" are spliced into the first tag "game", and "game", "good" and "game" are spliced into the second tag The label "good game" and the splicing of "city", "ying", "city", and "hero" into the third label "city hero". It can be seen from this that in the case where the promotion video of a new game "City Hero" is online, through the above-mentioned implementation manner of the present application, in the case where "City Hero" is not included in the original history tag, Based on the fresh characters provided in the description character library, the "city hero" label is obtained through iterative calculation operations, thereby realizing the technical effect of automatically generating fresh labels and improving the efficiency and accuracy of label generation. At the same time, because in the above-mentioned embodiment of the present application, the tags are generated based on word-by-word output, the description text and the historical tag library do not originally include the tag of "good game". Label. This enables users to retrieve the above-mentioned videos even when searching for "good games", thereby increasing the click-through rate and viewing popularity of the above-mentioned videos.

The above method will be described below with reference to FIG. 3 for another specific embodiment.

As shown in Figure 3, a title image of a video is shown, and the title image includes the text content "Are you greedy today?" At the same time, the title of the video is "A wave of snacks necessary for chasing dramas, come @ He will make it for you", and the video includes subtitles "This tutorial is very easy".

For the video image shown in Figure 3, the above-mentioned related text can be used as the video description text information of the video, that is, the text content in the video image "Are you greedy today" and the text content of the video title "Chase drama" A wave of must-have snacks, please @he make it for you" and the subtitle content "This tutorial is very simple".

Next, extract the video feature of the video, which can be the image feature of the video cover image as shown in Figure 3; the video category feature "food" of the video can also be extracted as another video feature; Extract the video description text information The description feature is to extract the feature information corresponding to the above-mentioned video description text information. As a specific form, the above-mentioned features can be represented by vectors or matrices.

Further, according to the above-mentioned extracted visual features, category features and description features, the confidence level corresponding to each character in the candidate character library determined by the reference character library and the description character library is determined. Assuming that the historical tag text preset for generating reference characters does not include the tag "food tutorial", in the traditional tag extraction method, the tag can only be determined from the preset historical tag library, and the "food tutorial" cannot be obtained. "Tutorial" is a label with high video relevance, that is to say, there is a technical problem that the accuracy of the extracted label is low.

In this embodiment, the current candidate character library includes the reference character library obtained by the historical tag word segmentation, and the current video description text character library, that is, the text content in the above-mentioned video image "Are you greedy today?" As well as the text content of the video title "A wave of snacks necessary for chasing dramas, hurry @ he will make it for you" and the text content of the subtitle content "This tutorial is very simple" and other text characters, and obtain the historical tag database through the above method and the confidence level corresponding to each tag text of the candidate tag library composed of the above text content. It can be seen that since the text description content includes the key text "tutorial", and the classification feature of the video also reflects the key feature "food", the label of "food tutorial" can be output.

Assuming that the relevant features of the above video are iteratively calculated, and the output characters are "oil, fried, food, product, crisp, skin, flower, raw, beauty, food, teaching, course, zero, food", It can be determined that the target label is the final output label as shown in Figure 3, which is "fried food", "crispy peanut", "food tutorial" and "snack".

It can be seen that, through the above method of the present application, the tags that are not included in the historical tag library can be output based on the content of the description text of the video, thereby achieving the technical effect of improving the accuracy of the output video tags.

In the embodiment of the present invention, the target video to be identified is acquired; a target candidate character library of the target video is constructed by using the video description text, wherein the target candidate character library includes a reference character set and a description character set, and the reference character set includes a plurality of historical tags. The reference characters obtained by word segmentation of the text, the description character set includes the description characters obtained by word segmentation of the video description text; in the case of extracting the video features of the target video and the description features of the video description text, the video features and description features N Iterative calculation to obtain M candidate character sequences; target characters are determined from each candidate character sequence based on confidence, and M target characters are obtained; then M target characters are spliced into target video tags matching the target video. That is to say, in this embodiment of the present application, by iteratively calculating the video features and the description text features of the target video, the target characters are output word by word from the description character set composed of the reference character set and the video description text, and then the target characters are output through the output The target characters make up the target video tag. Since the target candidate character library used for outputting characters in the embodiment of the present application combines the description character set, and the description character set includes fresh text content related to the video, the fresh characters can be output in combination with the description character set, and then according to the output freshness Character splicing obtains fresh labels, which improves the accuracy of output video labels, and solves the technical problem of low accuracy of video labels obtained by existing video label determination methods.

As a kind of optional embodiment, the above-mentioned iterative calculation is carried out to the video feature and the description feature for N times, to obtain M candidate character sequences including:

S1, in the process of performing the i-th iterative calculation on the video features and the description features, in the vector transformation network, based on the visual features, the description features in the video features and the iterative reference vector obtained by the i-1th iteration calculation, the calculation is obtained. The ith intermediate latent vector set, wherein the ith intermediate latent vector set includes the ith visual latent vector matched with the visual feature, the ith descriptive latent vector matched with the description feature, and the ith iterative calculation is obtained The iterative reference vector of , i is a natural number greater than or equal to 1 and less than or equal to N;

S2, based on the iterative reference vector obtained by the ith iterative calculation, determine the respective matching confidences of each character in the target candidate font library, and determine the jth candidate character sequence according to the confidence, where j is greater than or equal to 0, and A natural number less than or equal to i;

S3, under the condition that the iteration reference vector obtained by the i-th iterative calculation does not reach the end condition, perform the i+1-th iteration calculation;

S4 , in the case that the iterative reference vector obtained by the ith iteration calculation reaches the end condition, determine the iteration reference vector obtained by the previous i iteration calculation as N iteration reference vectors.

It should be noted that the above vector transformation network may be a Transformer model. The typical structure of Transformer includes an encoder and a decoder. The two structures are usually collectively referred to as Transformer modules. The difference between the two structures is that the decoder can only see the information of the current position and the previous position during the calculation process. Models that can be used in this embodiment include BERT model (Bidirectional Encoder Representation from Transformers, bidirectional encoder identification model), GPT model (Generative Pre-Training, generate pre-training model), UniLM model (Unified Language Model, unified pre-training language) model) and the VPUniLM model. As a preferred way, in this embodiment, the VPUniLM model can be used as the above-mentioned vector transformation network.

It can be understood that the visual features of the target video may be image features obtained by extracting each frame of the target video, or may also be image features obtained by performing style extraction on the target video. The form of specific visual features used to characterize the picture content of the target video is not limited here.

It should be noted that, in the above model, while obtaining the visual features and descriptive features in the video features, it is also necessary to obtain the iterative reference vector obtained by the previous iterative calculation, so as to combine the above-mentioned visual features, description features and the previous iterative calculation. The obtained iterative reference vector is used as the input of the current iterative calculation, and then the visual latent vector matched with the visual feature, the description latent vector matched with the description feature, and the current iterative reference vector are calculated. It can be understood that, in this embodiment, since the process of each iterative calculation needs to be combined with the iterative reference latent vector obtained by the previous iterative calculation as the input, the result of each iterative calculation is obtained from the historical iterative calculation. In other words, each iterative calculation result indicates the characteristics of the historical iterative calculation results, and then the correlation between the output iterative calculation results can be determined by the sequence of the iterative outputs.

The above-mentioned count letters i, j will be described below. It can be understood that, for the i-th iterative calculation, the j-th candidate character sequence for generating the target character can be determined, and j can be less than or equal to i. That is, the number of iterative reference vectors obtained by iterative calculation can be greater than or equal to the number of candidate character sequences. In a specific implementation manner, the iterative reference vector obtained for the i-th time may be a vector for indicating a spacer. For example, the iterative reference vector calculated by the first iteration is used to determine the first candidate character sequence; the iteration reference vector calculated by the second iteration is used to determine the second candidate character sequence; the iterative reference vector calculated by the third iteration It is used to determine the first spacer; the iterative reference vector calculated in the fourth iteration is used to determine the third candidate character sequence; the iterative reference vector calculated in the fifth iteration is used to determine the fourth candidate character sequence; the sixth iteration The computed iteration reference vector indicates the end delimiter. Furthermore, corresponding to the above-mentioned six iterative reference calculations, four candidate character sequences can be determined, and further, four target characters can be determined and obtained according to the four candidate character sequences, and two target labels can be obtained based on the combination of the above-mentioned four target characters.

In another implementation manner, the above-mentioned count letters i and j may be equal, that is, the number of iterative reference vectors obtained through iterative calculation may be equal to the number of candidate character sequences, for example, five iterative reference vectors are determined corresponding to five A candidate character sequence is obtained, five target characters are determined, and one or more target labels are obtained by splicing the above five target characters.

Through the above method of the present application, in the process of performing the i-th iterative calculation on the video features and the description features, in the vector transformation network based on the visual features in the video features, the description features and the i-1th iterative calculation. Iterate the reference vector, and calculate the ith intermediate latent vector set; based on the iterative reference vector obtained by the ith iteration calculation, determine the respective matching confidence levels of each character in the target candidate font library, and determine the jth candidate according to the confidence level. character sequence, where j is a natural number greater than or equal to 0 and less than or equal to i; if the iteration reference vector obtained by the i-th iterative calculation does not reach the end condition, execute the i+1-th iteration calculation; When the iterative reference vector calculated by the next iteration reaches the end condition, the iterative reference vector calculated by the previous i iterations is determined as N iterative reference vectors, and then the corresponding candidate character sequence is determined based on the iterative reference vector. In the above-mentioned real-time mode of the present application, an iterative method is used to generate an intermediate latent vector, and a candidate character sequence is determined based on the iterative reference vector in the intermediate latent vector, thereby ensuring the sequential association relationship between the candidate character sequences. Determining the output target character and the spliced target label based on the candidate character sequence that has a sequential relationship question.

As an optional embodiment, the above-mentioned iterative reference vector obtained based on the i-th iterative calculation, determining the respective matching confidence levels of each character in the target candidate font library includes: in the label generation network connected to the vector conversion network, Based on the iterative reference vector calculated by the i-th iteration, determine the first confidence level of each reference character in the reference character set and the second confidence level of each description character in the description character set.

It can be understood that, in this embodiment, in the above-mentioned process of calculating the confidence level by iterating the reference vector, the process of calculating the confidence level for each reference character in the reference character set is different from the confidence level for each description character in the description character set. The process is different. For example, different calculation methods and different calculation parameters can be used to distinguish the methods of calculating the first confidence level of the reference character set and the method of calculating the second confidence level of the description character set.

In an optional manner, a higher weight parameter may be configured for the initial first confidence level corresponding to the calculated reference character set to obtain the first confidence level; for the initial second confidence level corresponding to the calculated description character set The confidence level can be configured with a lower weight parameter to obtain the second confidence level, thereby improving the confidence level of each character in the reference character set, thereby making the output character a higher probability of being a character in the reference character set, so as to achieve a higher probability of passing the reference character set The technical effect of constraining output character types and rules.

Through the above-mentioned embodiments of the present application, in the label generation network connected with the vector conversion network, based on the iterative reference vector obtained by the ith iteration calculation, the first confidence level of each reference character in the reference character set is determined, respectively, and the second confidence level of each description character set in the description character set, distinguish the confidence calculation methods corresponding to the reference character set and the description character set, and then adjust the output characters in the reference character set and description character set as required. Probability, to achieve the technical effect of constraining the output character type and rules by referring to the character set.

As an optional implementation manner, the above-mentioned iterative reference vector obtained by the i-th iteration calculation is used to determine the first confidence level of each reference character in the reference character set for each matching, and the first confidence level for each matching of each description character in the description character set. Two confidences include:

S1, determine a first weight coefficient and a first bias parameter that match each reference character in the label generation network, a second weight coefficient and a second bias parameter that match each description character, and the reference weight and reference Bias parameter;

S2, determine the first confidence level of each reference character based on the iterative reference vector, the first weight coefficient and the first bias parameter obtained based on the i-th iterative calculation;

S3, obtaining the first intermediate result calculated based on the i-th description implicit vector, the second weight coefficient and the second bias parameter, and the iterative reference vector, reference weight and reference bias parameter calculated based on the i-th iterative calculation the second intermediate result obtained;

S4, based on the first intermediate result and the second intermediate result, determine the second confidence level of each description character.

The above-mentioned embodiment will be specifically described below. It should be noted that, in this embodiment, the video description text is the video title.

指示与描述文本特征(在本实 施例中为标题)经过上述Transformer模型的一次迭代计算，输出的与第n 个字符对应的第n个隐向量；假设上述参考迭代隐向量为

需要说明 的是，在本实施例中，在迭代计算过程中，需要多次输入和多次输入，

表示在第t次迭代计算输出的最后一个位置的隐向量。First, assume that the above described hidden vector is

The indication and description text feature (title in this embodiment) is calculated by one iteration of the above-mentioned Transformer model, and the outputted nth latent vector corresponding to the nth character; it is assumed that the above-mentioned reference iterative latent vector is

It should be noted that, in this embodiment, in the iterative calculation process, multiple inputs and multiple inputs are required,

The latent vector representing the last position of the computed output at the t-th iteration.

Next, according to the training result of the model, determine the first weight coefficient and the first bias parameter in the label generation network that match each reference character, and the second weight coefficient and the second bias parameter that match each description character respectively, And the reference weight and reference bias parameters, that is, the following six parameters are determined:

表示参考字符集中第i个字符的权重参数；The first weight coefficient:

Indicates the weight parameter of the i-th character in the reference character set;

First bias parameter:

Second weight coefficient: W ^title , W ^dec ;

The second bias parameter: b ^title and b ^dec are bias parameters.

表示模型在标题位置的隐 向量输出(即描述隐向量)，在第t步解码阶段，

表示该步输出的最 后一个位置的隐向量输出(即参考迭代隐向量)，

表示在第t步参考字 符集中第i个字符的得分，那么

由以下公式计算出：Based on the parameters obtained from the above training, it is assumed that

Represents the latent vector output of the model at the title position (that is, describing the latent vector), in the t-th decoding stage,

The latent vector output representing the last position of the output of this step (that is, the reference iterative latent vector),

represents the score of the i-th character in the reference character set at step t, then

Calculated by the following formula:

表示参考字符集中第i个字符的权重参数，

是参考字 符集中第i个字符的偏置参数。In the formula,

represents the weight parameter of the i-th character in the reference character set,

is the bias parameter for the ith character in the reference character set.

表示在 第t步第n个字符的得分，

由以下公式计算出：In order to select characters from the title text, a pointer network module is added to predict the score for selecting words from the title, assuming that the latent vector corresponding to the nth character in the title is

represents the score of the nth character at step t,

Calculated by the following formula:

where W ^title and W ^dec are weight parameters, and b ^title and b ^dec are bias parameters.

上述第二中间结果即上式中的

每个参考字符的第一匹 配度即

每个描述字符的第二匹配度即

It should be noted that the above-mentioned first intermediate result is that in the above formula

The second intermediate result above is that in the above formula

The first matching degree of each reference character is

The second matching degree of each description character is

Through the above-mentioned embodiments of the present application, by determining the first weight coefficient and the first offset parameter matching each reference character in the label generation network, the second weight coefficient and the second offset matching each description character respectively parameters, and reference weight and reference bias parameters; based on the iterative reference vector, the first weight coefficient and the first bias parameter calculated by the i-th iteration, determine the first confidence level of each reference character; Describe the first intermediate result calculated by the implicit vector, the second weight coefficient and the second bias parameter, and the second intermediate result calculated based on the iterative reference vector, the reference weight and the reference bias parameter calculated based on the ith iteration ; Based on the first intermediate result and the second intermediate result, determine the second degree of confidence of each description character, determine the second degree of matching of each text word segmentation, thereby accurately determine the score of each candidate character in the reference character set and The score of each description character in the description character set can be used to accurately calculate the score of each candidate character in the current iterative calculation step, which improves the accuracy of label generation.

As an optional embodiment, the above-mentioned iterative reference vector obtained based on the i-th iterative calculation, determines the respective matching confidence levels of each character in the target candidate font library, and determines the j-th candidate character sequence according to the confidence level. The sequence includes: The fully connected layer in the label generation network performs a weighted sum calculation on the obtained multiple first confidence levels and multiple second confidence levels, and obtains the respective matching confidence levels of each character in the target candidate font to generate the jth candidate character sequences.

每个描述字符对应的的第二匹配度即

在一种实施方式 中，可以为上述第一匹配度和第二匹配度配置不同的权重，仅基于加权求 和的结果确定出每个标签对应的置信度。It can be understood that the first matching degree of each reference character obtained by the t-th iteration calculation is

The second matching degree corresponding to each description character is

In an embodiment, different weights may be configured for the first matching degree and the second matching degree, and the confidence corresponding to each tag is determined only based on the result of the weighted summation.

每个描述字符的第二匹 配度即

为上述第一匹配度和第二匹配度配置相同的权重，进而将得 到的

以及对应的字符进行组合，直接得到候选字符序列。In another implementation manner, in multiple label extraction steps, the first matching degree of each reference character calculated by the t-th iteration can be obtained, namely

The second matching degree of each description character is

The same weight is configured for the first matching degree and the second matching degree, and then the obtained

And the corresponding characters are combined to directly obtain the candidate character sequence.

Through the above-mentioned embodiments of the present application, the fully connected layer in the label generation network performs weighted sum calculation on the obtained multiple first confidence levels and multiple second confidence levels, and obtains each character in the target candidate font database. The confidence level of the matching is to generate the jth candidate character sequence, so as to determine multiple first matching degrees and second matching degrees through multiple iterative calculation steps, and determine based on the multiple first matching degrees and second matching degrees. Multiple candidate sequences and target characters corresponding to the candidate sequences improve the accuracy of generating labels.

As a kind of optional implementation, before obtaining the target video to be identified, the above also includes:

S1, obtain a sample video and a sample label matching the sample video, wherein the sample video is carried with the sample video description text;

S2, using the sample video description text to construct a sample candidate character library of the sample video, wherein the sample candidate character library includes a reference character set and a sample description character set, the reference character set includes reference characters obtained by segmenting multiple historical tags, and the sample description character set Including the description characters obtained by word segmentation of the sample video description text;

S3, use the sample video, sample label and sample candidate font to train the initialized label generation network until the training convergence condition is reached.

Embodiments of the training phase are described below. In the training phase, the labels corresponding to the known videos are known. For example, a video contains two labels of "game" and "city hero", and the two labels are connected with the symbol ";" as the real value, and the visual features of the video are used. , the category embedding vector, the word embedding vector of the title, and the word embedding vector of the real value are input together. The prediction stage needs to use the visual feature, the category embedding vector, the word embedding vector of the title and the currently predicted word embedding vector as input. The encoder operation is used to calculate the latent vector of the position of the visual feature and the original title word embedding vector, and the decoder operation is used to calculate the hidden vector of the ground truth position. The hidden vector calculates the probability distribution of the dictionary through a layer of full connection operation, where the loss function type is cross entropy, n represents the length of the predicted text, and V represents the dictionary.

表示参考字符集中i个字符的权重参数；When the loss function shown in the above formula reaches the convergence condition, it is determined that the training of the following parameters is completed: the first weight coefficient:

Indicates the weight parameter of i characters in the reference character set;

First bias parameter:

Second weight coefficient: W ^title , W ^dec ;

The second bias parameter: b ^title , b ^dec .

The following describes the training process of another UniLM (Unified Language Model, unified pre-trained language model) model.

The above method of the present application is used to obtain a sample video and a sample label matching the sample video, wherein the sample video carries the sample video description text; the sample video description text is used to construct a sample candidate font library of the sample video, wherein the sample candidate font library Including reference character set and sample description character set, the reference character set includes reference characters obtained by word segmentation of multiple historical tags, and the sample description character set includes description characters obtained by word segmentation of sample video description text; Using sample videos, sample labels and The sample candidate character library trains the initialized label generation network until the training convergence condition is reached, so as to obtain the above label generation network in a supervised way, improve the reliability of the target label obtained by the above label generation network, and solve the problem of current situation. The technical problem of low label accuracy in some label generation methods.

As an optional method, in the process of performing the i-th iterative calculation on the video features and the description features, the vector transformation network is based on the visual features in the video features, the description features, and the i-1th iterative calculation to obtain The iterative reference vector of , the i-th intermediate hidden vector set is calculated to include:

When i is equal to 1, convert the visual features and description features into the word embedding vector corresponding to the target video; extract the category embedding vector, paragraph embedding vector and position embedding vector corresponding to the target video, where the category embedding vector is used to indicate the target video The category information of the video. The paragraph embedding vector includes the first paragraph sub-vector obtained based on the visual feature and the second paragraph sub-vector obtained based on the description feature. The position embedding vector is used to indicate the frame sequence of each video frame in the target video, and the video description. The order of word segmentation of each text in the text; the word embedding vector, category embedding vector, paragraph embedding vector and position embedding vector are used as the input vector of the vector conversion network to calculate the i-th intermediate latent vector set;

When i is greater than 1, use the word embedding vector, category embedding vector, paragraph embedding vector, position embedding vector, and the iterative reference vector calculated by the i-1th iteration as the input vector of the vector conversion network to calculate the first The set of intermediate latent vectors i times.

As a kind of optional embodiment, above-mentioned in the vector conversion network based on the visual feature in the video feature, the description feature and the iterative reference vector that the ith-1th iterative calculation obtains, the calculation obtains the ith intermediate latent vector set including:

S1, determine the ith visual latent vector, the ith description latent vector based on the visual feature and the description feature, wherein, the ith visual latent vector and the ith description latent vector are used to indicate the contextual relationship of the character in the target video label;

S2: Determine the iterative reference vector obtained by the ith iterative calculation based on the visual feature, the description feature and the iterative reference vector obtained by the i-1th iterative calculation.

The category embedding vector of the above-mentioned target video is a feature used to indicate the general category of the target video. For example, if the target video is a video about a sports meeting, the category feature of the video can be used to indicate that the video is "sports category", which corresponds to the category embedding. The vector can be (0, 0, 0, 0, 1); for another example, if the target video is a video of a song mv, the category feature of the video can be used to characterize the video as a "song category", corresponding to the category embedding vector It can be (0, 0, 0, 0, 2); for another example, if the target video is a documentary "Chinese Food", the category feature of the video can be used to characterize the video as "food" and "documentary", The embedding vector corresponding to the category can be (0, 0, 0, 0, 3). The above is only an exemplary description, and does not limit the specific content corresponding form of the category embedding vector that specifically represents the target video in this embodiment.

The above method will be described below with reference to FIG. 4 .

As shown in Figure 4, before the iterative calculation, the vector combinations input into the above-mentioned vector transformation network include: position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector.

For the category features of videos, in this embodiment, each video category can be corresponding to a vector group, so that the video features are represented by the vector group. In the schematic diagram of the model shown in Figure 4, the converted category embedding vector is (5, 5, 5, 5, 5, 5, 5).

Next, the visual features and descriptive features are converted into word embedding vectors. First, extract the frame of the target video, the extracted frame rate is 1fps, and input the extracted video frame into the CLIP model to obtain the visual feature vector, which corresponds to the video vector (F, F, F) in Figure 4. And convert the video description text into a vector, corresponding to the title vector (I, Love, You) in Figure 4, and join the connection vector CLS, that is, the word embedding vector (F, F, F, CLS, I, Love, You) ).

At the same time, as shown in FIG. 4 , the paragraph embedding vector and the position embedding vector of the video are also obtained. Among them, the paragraph embedding vector is (0, 0, 0, 1, 1, 1, 1). It can be understood that the paragraph embedding vector is used to represent the video vector part and the title vector part in the word embedding vector, where, ( 0, 0, 0) corresponds to the video vector (F, F, F) and (1, 1, 1) corresponds to the title vector (I, Love, You). The position embedding vector (0, 1, 2, 0, 1, 2, 3) is used to characterize the order of vectors in the input model. For example, the (0, 1, 2) part is used to identify the frame sequence of the video frame of the video vector (F, F, F); the (1, 2, 3) part is used to identify the title vector (I, Love, You) three The order of text segmentation.

(图中未示出

)。As shown in Figure 4, after the above-mentioned position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector are input into the Transformer module for the first time, the intermediate hidden vector set is obtained.

(not shown in the figure

).

It can be understood that, by inputting the above-mentioned intermediate latent vector set into the pointer network module, the score of each character in the reference character set and the description character set can be obtained.

Through the above method of the present application, when i is equal to 1, the visual feature and the description feature are converted into the word embedding vector corresponding to the target video; the category embedding vector, paragraph embedding vector and position embedding vector corresponding to the target video are extracted, wherein, The category embedding vector is used to indicate the category information of the target video. The paragraph embedding vector includes the first paragraph sub-vector obtained based on the visual feature and the second paragraph sub-vector obtained based on the description feature. The position embedding vector is used to indicate each video frame in the target video. The frame order of , and the order of each text segmentation in the video description text; the word embedding vector, category embedding vector, paragraph embedding vector and position embedding vector are used as the input vector of the vector conversion network to calculate the ith intermediate hidden vector set, Therefore, the confidence value of each candidate character is obtained through the pointer network module and the Transformer module, thereby solving the technical problem of low label accuracy obtained by the existing label generation method.

As an optional implementation manner, determining the target character from each candidate character sequence based on the confidence includes: according to the contextual relationship of the characters in the target video tag, using the target character determined in the jth candidate character sequence as The jth target character in the target video tag.

和

拼 接起来，并选择得分最高的字符，即是第当前步骤得到的第t个候选字符 序列，并输出第t个目标字符。It can be understood that in the model prediction stage, the above method will be used to obtain

and

Splice them together, and select the character with the highest score, which is the t-th candidate character sequence obtained in the current step, and output the t-th target character.

Continue to describe the above method with reference to FIG. 4 .

As shown in Figure 4, the iterative calculation process of the first step is as follows:

以及

(图中未示出)。After the above-mentioned position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector shown in Figure 4 are input into the Transformer module for the first time, the output of the intermediate hidden vector set is obtained.

as well as

(not shown in the figure).

以及训练得到的

W^title、W^dec、b^title和 b^dec，通过以下式子分别算出第1步迭代计算过程中，参考字符集中第i 个字符的得分以及描述字符集中第n个字符的得分：based on

and trained

W ^title , W ^dec , b ^title and b ^dec , respectively calculate the score of the i-th character in the reference character set and the score of the n-th character in the description character set in the iterative calculation process in the first step by the following formulas:

表示在第1步参考字符集中第i个字符的得分，

表示 在第1步中描述字符集中第n个字符的得分。将

和

拼接起来，并 选择得分最高的单词，即是第1步的输出词。如图4所示，第一步的输出 词为“爱”；in,

represents the score of the i-th character in the reference character set in step 1,

represents the score describing the nth character in the character set in step 1. Will

and

Splice them together and select the word with the highest score, which is the output word of step 1. As shown in Figure 4, the output word of the first step is "love";

The second step of analysis is as follows:

输入Transformer模块，得到中间隐向量集

以及

Embed the above position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector as well as those obtained in the first step.

Enter the Transformer module to get the intermediate hidden vector set

as well as

以及训练得到的

W^title、W^dec、b^title和 b^dec，通过以下式子分别算出第2步迭代计算过程中，参考字符集中第i 个字符的得分以及描述字符集中第n个字符的得分：based on

and trained

W ^title , W ^dec , b ^title and b ^dec , respectively calculate the score of the i-th character in the reference character set and the score of the n-th character in the description character set in the iterative calculation process in the second step by the following formulas:

表示在第2步参考字符集中第i个字符的得分，

表示 在第2步中描述字符集中第n个字符的得分。将

和

拼接起来，并 选择得分最高的单词，即是第2步的输出词。如图4所示，第一步的输出 词为“情”；in,

represents the score of the i-th character in the reference character set in step 2,

represents the score describing the nth character in the character set in step 2. Will

and

Splice them together and select the word with the highest score, which is the output word of step 2. As shown in Figure 4, the output word of the first step is "love";

The end steps are as follows:

输入Transformer模块，得到中间隐向量集

以及

其中，

与预设的 结束标识向量相匹配，即确定迭代步骤结束，不再继续生成新的字符。Embed the above position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector as well as those obtained in the first step.

Enter the Transformer module to get the intermediate hidden vector set

as well as

in,

Matching with the preset end identification vector, that is, it is determined that the iteration step is over, and no new characters are continued to be generated.

Then, the output word "love" of the first step and the output word "love" of the second step are spliced together, and the target label of the target video titled "I love you" is obtained as "love".

Through the above-mentioned embodiments of the present application, through iterative calculation and word-by-word output, since the target candidate character library used for outputting characters in the embodiment of the present application combines the description character set, and the description character set includes fresh video-related characters Therefore, fresh characters can be output in combination with the description character set, and then fresh labels can be obtained by splicing the output fresh characters, which improves the accuracy of the output video labels, and solves the problem of the accuracy of the video labels obtained by the existing video label determination methods. Low technical issues.

As an optional implementation manner, after splicing the M target characters into target video tags matching the target video, the method further includes: adjusting the order of the M target characters in the target video tag based on the contextual semantic relationship, so as to obtain the updated target video tag.

In an optional way, it is assumed that multiple iterations are performed according to the above method, and it is assumed that the initial output M characters are "city", "city", "xiong", "ying", "you", " "Play", "You", "Play", and "Good", you can determine the order of the two characters "Xiong" and "English" according to the semantic analysis of the above characters, so as to obtain the entry for "Hero" At the same time, the order of the three characters "you", "play" and "good" can also be adjusted to obtain the entry of "good game"; and the final sequence of the updated M characters is determined as: "city" , "city", "ying", "hero", "you", "play", "good", "play", "play". And based on this character order, target labels are determined: "city hero", "game", "good game".

The manner in which the above-mentioned order of outputting characters is described below. As shown in Figure 5, the above Transformer module can use the Attention matrix to input and output characters. As shown in the figure, each row of the matrix represents the output, and each column represents the input, and the Attention matrix represents the association between the output and the input. Assuming that the white squares represent 0 and the black squares represent 1, then the first line indicates that the output character (x ₁ ) can only be related to the start tag <s>, and the second line indicates the output character (x _{2 )} ) can only be associated with start tags <s> and (x ₁ ), and so on. That is to say, a one-way language model can be implemented only by introducing a lower triangular Mask into the Attention matrix of the Transformer, and staggering the input and output by one bit for training, that is, outputting characters in a predetermined order.

Optionally, an out-of-order language model is described.

Like the language model, the out-of-order language model performs conditional probability decomposition, but the decomposition order of the out-of-order language model is random, as shown in the following formula:

p(x ₁ ,x ₂ ,x ₃ ,...x _n )=p(x ₁ )p(x ₂ |x ₂ )p(x ₃ |x ₁ ,x ₂ )...p(x _n |x ₁ ,x ₂ ,…x _n-1 )

=p(x ₃ )p(x ₁ |x ₃ )p(x ₂ |x ₃ ,x ₁ )…p(x _n |x ₃ ,x ₁ ,…x _n-1 )

=…

=p(x _n-1 )p(x ₁ |x _n-1 )p(x _n |x _n-1 ,x ₁ )…p(x ₂ |x _n-1 ,x ₁ ,…x ₃ )

According to the above formula, for the characters x ₁ , x ₂ . . . x _n , any "appearance order" is possible. In principle, each order corresponds to a model, so in principle there are n! a language model. With Transformer-based models, all of these sequences can be implemented in the same model.

The implementation is described below with reference to FIG. 6 . Taking the generation of "Beijing welcomes you" as an example, assuming that the sequence of characters required to be generated is: "<s>→ying→jing→you→huanbei→bei→<e>", then through the above embodiment of MSAK shown in Figure 6 In the Attention matrix, the character output in the above order can be realized. As shown in Figure 6, the fourth row has only one black box, which means that "Ying" can only be related to the start tag <s>, while the second row has two black boxes, which means that "Jing" can only be related to the start mark <s> is related to "Welcome", and so on. Intuitively, this is like "shuffling" the lower triangular Mask of the unidirectional language model.

That is to say, implementing a language model in a certain order is equivalent to disrupting the original lower-triangular form of Mask in some way. Because the above Attention provides such an n×n Attention matrix, the Transformer module in this embodiment can have enough degrees of freedom to mask the above character matrix in different ways, thereby realizing a variety of character output orders. Effect.

The complete process of the generation method of the video label provided by the application is described below in conjunction with the process shown in Figure 7:

S702, constructing a target candidate font library;

This embodiment will be described by taking the video description text as the video title as an example. Suppose the video title is "I love you", and the reference character library includes the characters "Kin" and "Qing"; that is, the target candidate character library is obtained by combining the above video title and the characters in the above reference character library: "I", "Love" ", "you", "pro", "love".

S704, extract the video vector;

Specifically, as shown in Figure 4, the vector combination input into the above-mentioned vector conversion network includes: position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector.

For the category features of videos, in this embodiment, each video category can be corresponding to a vector group, so that the video features are represented by the vector group. In the schematic diagram of the model shown in Figure 4, the converted category embedding vector is (5, 5, 5, 5, 5, 5, 5).

Next, the visual features and descriptive features are converted into word embedding vectors. First, extract the frame of the target video, the extracted frame rate is 1fps, and input the extracted video frame into the CLIP model to obtain the visual feature vector, which corresponds to the video vector (F, F, F) in Figure 4. And convert the video description text into a vector, corresponding to the title vector (I, Love, You) in Figure 4, and join the connection vector CLS, that is, the word embedding vector (F, F, F, CLS, I, Love, You) ).

At the same time, as shown in FIG. 4 , the paragraph embedding vector and the position embedding vector of the video are also obtained. Among them, the paragraph embedding vector is (0, 0, 0, 1, 1, 1, 1). It can be understood that the paragraph embedding vector is used to represent the video vector part and the title vector part in the word embedding vector, where, ( 0, 0, 0) corresponds to the video vector (F, F, F) and (1, 1, 1) corresponds to the title vector (I, Love, You). The position embedding vector (0, 1, 2, 0, 1, 2, 3) is used to characterize the order of vectors in the input model. For example, the (0, 1, 2) part is used to identify the frame sequence of the video frame of the video vector (F, F, F); the (1, 2, 3) part is used to identify the title vector (I, Love, You) three The order of text segmentation.

S706, the extracted video vector is input into the Transformer model, and the intermediate vector of the output is obtained;

以及

(图中 未示出)。Specifically, after the above-mentioned position embedding vector, paragraph embedding vector, word embedding vector, and category embedding vector shown in FIG. 4 are input into the Transformer module for the first time, the intermediate hidden vector output is obtained.

as well as

(not shown in the figure).

S708, determine whether the intermediate vector includes the end vector;

是否为结束向量，在

并非结束向量的情况下，执行后 续步骤。judgment

is the end vector, in

If it is not the end vector, perform the next steps.

S710, the intermediate vector is analyzed and calculated based on the trained parameter set, and the score of each candidate character in the t step is obtained;

S712, determine whether the current candidate character is the candidate character with the highest score;

S714, the candidate character with the highest score is determined as the output character of this step;

以及训练得到的

W^title、 W^dec、b^title和b^dec，通过以下式子分别算出第1步解析过程中，历史标签 词表里第i个单词的得分以及标题词表第n个词的得分：Specifically, based on

and trained

W ^title , W ^dec , b ^title and b ^dec , calculate the score of the i-th word in the historical tag vocabulary and the score of the n-th word in the title vocabulary in the parsing process of the first step by the following formulas:

表示在第1步历史标签词表里第i个单词的得分，

表 示在第1步中视频标题中第n个词的得分。将

和

拼接起来，并选 择得分最高的单词，即是第1步的输出词。假设得到的第一候选字符序列 为：“我”(70％)、“爱”(90％)、“你”(76％)、“亲”(40％)、“情”(80％)， 进而确定上述第一候选字符序列中置信度最高的字符为“爱”，进而如图4 所示，第一步的输出字符为“爱”。in,

Indicates the score of the i-th word in the history tag vocabulary in step 1,

Represents the score of the nth word in the video title in step 1. Will

and

Splice them together and select the word with the highest score, which is the output word of step 1. Suppose the obtained first candidate character sequence is: "I" (70%), "Love" (90%), "You" (76%), "Kin" (40%), "Qing" (80%), Further, it is determined that the character with the highest confidence in the above-mentioned first candidate character sequence is "love", and as shown in Fig. 4, the output character of the first step is "love".

并非为结束向量，接着执行第二步迭代计算步骤，如下：because

is not the end vector, and then performs the second iterative calculation step, as follows:

输入Transformer模块，得到中间隐向量输出

以及

Embed the above position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector as well as those obtained in the first step.

Enter the Transformer module to get the intermediate hidden vector output

as well as

同样不为结束向量的判断结果，基于

以及 训练得到的

W^title、W^dec、b^title和b^dec，通过以下式子分别 算出第2步解析过程中，历史标签词表里第i个单词的得分以及标题词表 第n个词的得分：based on judgment

It is also not the judgment result of the end vector, based on

and trained

W ^title , W ^dec , b ^title and b ^dec , calculate the score of the i-th word in the historical tag vocabulary and the score of the n-th word in the title vocabulary in the parsing process of the second step by the following formulas:

表示在第2步历史标签词表里第i个单词的得分，

表 示在第2步中视频标题中第n个词的得分。将

和

拼接起来，并选 择得分最高的单词，即是第2步的输出词。假设得到的第二候选字符序列 为：“我”(60％)、“爱”(20％)、“你”(60％)、“亲”(40％)、“情”(80％)， 进而确定上述第一候选字符序列中置信度最高的字符为“情”，如图4所 示，第二步的输出词为“情”；in,

Indicates the score of the i-th word in the historical tag vocabulary in step 2,

represents the score of the nth word in the video title in step 2. Will

and

Splice them together and select the word with the highest score, which is the output word of step 2. Suppose the obtained second candidate character sequence is: "I" (60%), "Love" (20%), "You" (60%), "Kin" (40%), "Qing" (80%), Then it is determined that the character with the highest confidence in the above-mentioned first candidate character sequence is "Love", as shown in Figure 4, the output word of the second step is "Love";

并非为结束向量，接着执行第三步解析步骤，如下：because

is not the end vector, and then performs the third parsing step, as follows:

输入Transformer模块，得到中间隐向量输出

以及

其中，

与预设的 结束标识向量相匹配，即确定解析步骤结束，不再继续生成标签。Embed the above position embedding vector, paragraph embedding vector, word embedding vector and category embedding vector as well as those obtained in the first step.

Enter the Transformer module to get the intermediate hidden vector output

as well as

in,

Matches with the preset end identification vector, that is, it is determined that the parsing step is over, and the label is no longer generated.

S716, outputting the output characters determined in each step, and splicing to generate a video tag.

Finally, the output word "love" of the first step and the output word "love" of the second step are spliced together, and the tag of the target video titled "I love you" is obtained as "love".

In the above-mentioned embodiment of the present invention, the target video to be recognized is acquired; the target candidate character library of the target video is constructed by using the video description text, wherein the target candidate character library includes a reference character set and a description character set, and the reference character set includes a plurality of The reference characters obtained by word segmentation of the historical label text, the description character set includes the description characters obtained by word segmentation of the video description text; in the case of extracting the video features of the target video and the description features of the video description text, the video features and description features Carry out N times of iterative calculation to obtain M candidate character sequences; Determine target characters from each candidate character sequence based on the confidence, and obtain M target characters; Then splicing the M target characters into a target video matching the target video Label. That is to say, in this embodiment of the present application, by iteratively calculating the video features and the description text features of the target video, the target characters are output word by word from the description character set composed of the reference character set and the video description text, and then the target characters are output through the output The target characters make up the target video tag. Since the target candidate character library for outputting characters in this embodiment of the present application incorporates a description character set, and the description character set includes fresh text content related to videos, fresh characters can be output in combination with the description character set, and then according to the output freshness Character splicing obtains fresh labels, which improves the accuracy of output video labels, and solves the technical problem of low accuracy of video labels obtained by existing video label determination methods.

It should be noted that, for the sake of simple description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence. As in accordance with the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

According to another aspect of the embodiments of the present invention, there is also provided an apparatus for generating a video tag for implementing the above-mentioned method for generating a video tag. As shown in Figure 8, the device includes:

Obtaining unit 802 is used to obtain the target video to be identified, wherein, in the target video, the video description text for describing the target video is carried;

The construction unit 804 is used to construct a target candidate character library of the target video by using the video description text, wherein, the target candidate character library includes a reference character set and a description character set, and the reference character set includes the reference characters obtained by segmenting multiple historical tag texts, and the description The character set includes description characters obtained by segmenting the video description text;

The calculation unit 806 is configured to perform N times of iterative calculation on the video features and the description features under the condition that the video features of the target video and the description features of the video description text are extracted to obtain M candidate character sequences, wherein each candidate The candidate character objects in the character sequence include each candidate character in the target candidate character library and the confidence level of matching the candidate character. The confidence level is used to indicate the matching degree between the candidate character and the target video. N and M are natural numbers greater than or equal to 1 ;

Determining unit 808, for determining the target character from each candidate character sequence based on the confidence, obtains M target characters;

The splicing unit 810 is used for splicing the M target characters into a target video tag matching the target video.

Optionally, in this embodiment, for the embodiments to be implemented by the foregoing unit modules, reference may be made to the foregoing respective method embodiments, and details are not described herein again.

According to another aspect of the embodiments of the present invention, an electronic device for implementing the above-mentioned method for generating a video tag is also provided, and the electronic device may be the terminal device or the server shown in FIG. 9 . This embodiment is described by taking the electronic device as a terminal device as an example. As shown in FIG. 9 , the electronic device includes a memory 902 and a processor 904, where a computer program is stored in the memory 902, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.

Optionally, in this embodiment, the above-mentioned electronic device may be located in at least one network device among multiple network devices of a computer network.

Optionally, in this embodiment, the above-mentioned processor may be configured to execute the following steps through a computer program:

S1, obtain the target video to be identified, wherein, in the target video, carry the video description text for describing the target video;

S2, using the video description text to construct a target candidate character library of the target video, wherein the target candidate character library includes a reference character set and a description character set, the reference character set includes reference characters obtained by segmenting multiple historical tag texts, and the description character set includes The description characters obtained by word segmentation of the video description text;

S3, in the case of extracting the video features of the target video and the description features of the video description text, perform N times of iterative calculation on the video features and the description features to obtain M candidate character sequences, wherein, in each candidate character sequence, The candidate character object includes each candidate character in the target candidate character library and the confidence level of matching with the candidate character, the confidence level is used to indicate the degree of matching between the candidate character and the target video, and N and M are natural numbers greater than or equal to 1;

S4, determine the target character from each candidate character sequence based on the confidence, obtain M target characters;

S5, splicing the M target characters into target video tags matching the target video.

Optionally, those of ordinary skill in the art can understand that the structure shown in FIG. 9 is for illustration only, and the electronic device may also be a vehicle terminal, a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, and a mobile Internet Equipment (Mobile Internet Devices, MID), PAD and other terminal equipment. FIG. 9 does not limit the structure of the above electronic device. For example, the electronic device may also include more or fewer components than those shown in FIG. 9 (such as network interfaces, etc.), or have a different configuration than that shown in FIG. 9 .

The memory 902 can be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for generating a video tag in the embodiment of the present invention, and the processor 904 runs the software programs and modules stored in the memory 902, thereby Execute various functional applications and data processing, that is, implement the above-mentioned method for generating video tags. Memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, memory 902 may further include memory located remotely from processor 904, which may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. Wherein, the memory 902 can be specifically, but not limited to, be used to store information such as each element in the viewing angle picture, the generation information of the video tag, and the like. As an example, as shown in FIG. 9 , the above-mentioned memory 902 may include, but is not limited to, the acquiring unit 802, the constructing unit 804, the calculating unit 806, the determining unit 808 and the splicing unit 810 in the above-mentioned video tag generating apparatus. In addition, it may also include, but is not limited to, other module units in the above-mentioned video tag generating apparatus, which will not be repeated in this example.

Optionally, the above-mentioned transmission device 906 is configured to receive or send data via a network. Specific examples of the above-mentioned networks may include wired networks and wireless networks. In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable to communicate with the Internet or a local area network. In one example, the transmission device 906 is a radio frequency (RF) module for wirelessly communicating with the Internet.

In addition, the above-mentioned electronic device further includes: a display 908, and a connection bus 910 for connecting various module components in the above-mentioned electronic device.

In other embodiments, the above-mentioned terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be communicated by the multiple nodes through a network A distributed system formed by connection in the form of. Among them, a peer-to-peer (P2P, Peer To Peer) network can be formed between nodes, and any form of computing equipment, such as servers, terminals and other electronic devices can become a node in the blockchain system by joining the peer-to-peer network.

According to one aspect of the present application, there is provided a computer program product comprising a computer program/instructions containing program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion, and/or installed from a removable medium. When the computer program is executed by the central processing unit, various functions provided by the embodiments of the present application are performed.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above-mentioned video tag. generator method.

Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be configured to store a computer program for performing the following steps:

S1, obtain the target video to be identified, wherein, in the target video, carry the video description text for describing the target video;

S2, using the video description text to construct a target candidate character library of the target video, wherein the target candidate character library includes a reference character set and a description character set, the reference character set includes reference characters obtained by segmenting multiple historical tag texts, and the description character set includes The description characters obtained by word segmentation of the video description text;

S3, in the case of extracting the video features of the target video and the description features of the video description text, perform N times of iterative calculation on the video features and the description features to obtain M candidate character sequences, wherein, in each candidate character sequence, The candidate character object includes each candidate character in the target candidate character library and the confidence level of matching with the candidate character, the confidence level is used to indicate the degree of matching between the candidate character and the target video, and N and M are natural numbers greater than or equal to 1;

S4, determine the target character from each candidate character sequence based on the confidence, obtain M target characters;

S5, splicing the M target characters into target video tags matching the target video.

Optionally, in this embodiment, those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a In the computer-readable storage medium, the storage medium may include: a flash disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

If the integrated units in the above-mentioned embodiments are implemented in the form of software functional units and sold or used as independent products, they may be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, Several instructions are included to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the above-mentioned methods of various embodiments of the present invention.

In the above-mentioned embodiments of the present invention, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed client may be implemented in other manners. The device embodiments described above are only illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. To another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, and can also be implemented in the form of software functional units.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (13)

1. A method for generating a video tag, comprising:

acquiring a target video to be identified, wherein the target video carries a video description text for describing the target video;

constructing a target candidate word library of the target video by using the video description text, wherein the target candidate word library comprises a reference character set and a description character set, the reference character set comprises reference characters obtained by segmenting a plurality of historical label texts, and the description character set comprises description characters obtained by segmenting the video description text;

under the condition that the video features of the target video and the description features of the video description text are extracted, performing iteration calculation on the video features and the description features for N times to obtain M candidate character sequences, wherein candidate character objects in each candidate character sequence comprise each candidate character in the target candidate word stock and a confidence coefficient matched with the candidate character, the confidence coefficient is used for indicating the matching degree between the candidate character and the target video, and N, M is a natural number greater than or equal to 1;

determining target characters from each candidate character sequence based on the confidence degrees to obtain M target characters;

and splicing the M target characters into target video labels matched with the target videos.

2. The method of claim 1, wherein the performing N iterative computations on the video feature and the description feature to obtain M candidate character sequences comprises:

in the process of performing ith iterative computation on the video features and the description features, an ith intermediate hidden vector set is computed in a vector conversion network based on the visual features in the video features, the description features and iterative reference vectors obtained by the (i-1) th iterative computation, wherein the ith intermediate hidden vector set comprises an ith visual hidden vector matched with the visual features, an ith description hidden vector matched with the description features and iterative reference vectors obtained by the ith iterative computation, and i is a natural number which is greater than or equal to 1 and less than or equal to N;

determining the confidence coefficient of each character in the target candidate word library which is matched with each other based on the iterative reference vector obtained by the ith iterative calculation, and determining a jth candidate character sequence according to the confidence coefficient, wherein j is a natural number which is greater than or equal to 0 and less than or equal to i;

executing the (i + 1) th iterative computation under the condition that the iterative reference vector obtained by the (i) th iterative computation does not reach the end condition;

and under the condition that the iteration reference vector obtained by the ith iteration calculation reaches an end condition, determining the iteration reference vector obtained by the previous i iteration calculations as N iteration reference vectors.

3. The method of claim 2, wherein the determining the confidence of each match of each character in the target candidate word library based on the iterative reference vector calculated by the ith iteration comprises:

and in a label generation network connected with the vector conversion network, determining a first confidence coefficient of each matching of each reference character in the reference character set and a second confidence coefficient of each matching of each description character in the description character set based on the iterative reference vector obtained by the ith iterative calculation.

4. The method of claim 3, wherein the determining a first confidence that each reference character in the reference character set matches and a second confidence that each description character in the description character set matches based on the iterative reference vector calculated in the ith iteration comprises:

determining a first weight coefficient and a first bias parameter which are respectively matched with each reference character in the label generation network, a second weight coefficient and a second bias parameter which are respectively matched with each description character, and a reference weight and a reference bias parameter;

determining the first confidence of each reference character based on the iterative reference vector, the first weight coefficient and the first bias parameter obtained by the ith iterative calculation;

obtaining a first intermediate result obtained by calculation based on the ith description hidden vector, the second weight coefficient and the second bias parameter, and a second intermediate result obtained by calculation based on the iteration reference vector obtained by the ith iteration calculation, the reference weight and the reference bias parameter;

determining the second confidence of each of the descriptive characters based on the first intermediate result and the second intermediate result.

5. The method according to claim 3, wherein the determining a confidence level of each matching of each character in the target candidate word library based on the iterative reference vector calculated by the ith iteration, and determining the jth candidate character sequence according to the confidence level comprises:

and performing weighted summation calculation on the acquired multiple first confidence degrees and the multiple second confidence degrees through a full connection layer in the label generation network to obtain the confidence degrees of respective matching of each character in the target candidate word library so as to generate the jth candidate character sequence.

6. The method of claim 3, further comprising, prior to obtaining the target video to be identified:

acquiring a sample video and a sample label matched with the sample video, wherein the sample video carries a sample video description text;

constructing a sample candidate word library of the sample video by using the sample video description text, wherein the sample candidate word library comprises a reference character set and a sample description character set, the reference character set comprises reference characters obtained by segmenting a plurality of historical labels, and the sample description character set comprises description characters obtained by segmenting the sample video description text;

and training the initialized label generation network by using the sample video, the sample label and the sample candidate word stock until a training convergence condition is reached.

7. The method of claim 2, wherein in the process of performing the ith iterative computation on the video feature and the description feature, computing an ith intermediate hidden vector set in a vector conversion network based on the visual feature in the video feature, the description feature and an iterative reference vector computed in the (i-1) th iterative computation comprises:

converting the visual features and the description features into word embedding vectors corresponding to the target video under the condition that i is equal to 1; extracting a category embedding vector, a paragraph embedding vector and a position embedding vector corresponding to the target video, wherein the category embedding vector is used for indicating category information of the target video, the paragraph embedding vector comprises a first paragraph falling sub-vector obtained based on the visual features and a second paragraph falling sub-vector obtained based on the description features, and the position embedding vector is used for indicating a frame sequence of each video frame in the target video and a sequence of each text word in the video description text; taking the word embedding vector, the category embedding vector, the paragraph embedding vector and the position embedding vector as input vectors of the vector conversion network to calculate and obtain the ith intermediate hidden vector set;

and under the condition that i is larger than 1, taking the word embedding vector, the category embedding vector, the paragraph embedding vector, the position embedding vector and an iteration reference vector obtained by the i-1 th iteration calculation as input vectors of the vector conversion network to calculate and obtain the i-th intermediate hidden vector set.

8. The method according to claim 2, wherein the calculating an i-th intermediate hidden vector set based on the visual features in the video features, the description features, and the i-1 st iterative reference vector in the vector conversion network comprises:

determining the ith visual hidden vector and the ith description hidden vector based on the visual features and the description features, wherein the ith visual hidden vector and the ith description hidden vector are used for indicating the context relationship of characters in the target video tag;

and determining the iteration reference vector obtained by the ith iteration calculation based on the visual feature, the description feature and the iteration reference vector obtained by the (i-1) th iteration calculation.

9. The method of claim 8, wherein the determining a target character from each of the candidate character sequences based on the confidence level comprises:

and according to the context relationship of the characters in the target video label, taking the target character determined in the jth candidate character sequence as the jth target character in the target video label.

10. The method of claim 9, further comprising, after said splicing the M target characters into a target video tag matching the target video:

and adjusting the sequence of the M target characters in the target video label based on the context semantic relationship to obtain the updated target video label.

11. An apparatus for generating a video tag, comprising:

the device comprises an acquisition unit, a recognition unit and a processing unit, wherein the acquisition unit is used for acquiring a target video to be recognized, and the target video carries a video description text for describing the target video;

the construction unit is used for constructing a target candidate word stock of the target video by using the video description text, wherein the target candidate word stock comprises a reference character set and a description character set, the reference character set comprises reference characters obtained by segmenting a plurality of historical label texts, and the description character set comprises description characters obtained by segmenting the video description text;

a calculating unit, configured to perform iterative calculation on the video features and the description features of the video description text N times to obtain M candidate character sequences under the condition that the video features of the target video and the description features of the video description text are extracted, where a candidate character object in each candidate character sequence includes each candidate character in the target candidate word stock and a confidence coefficient that matches the candidate character, the confidence coefficient is used to indicate a matching degree between the candidate character and the target video, and N, M is a natural number greater than or equal to 1;

the determining unit is used for determining target characters from each candidate character sequence based on the confidence coefficient to obtain M target characters;

and the splicing unit is used for splicing the M target characters into target video labels matched with the target videos.

12. A computer-readable storage medium, comprising a stored program, wherein the program when executed performs the method of any one of claims 1 to 10.

13. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 10 by means of the computer program.

Images