CN116229075A - Data processing method, device, computer equipment and storage medium - Google Patents

Abstract

The invention discloses a data processing method, device, computer equipment and storage medium. The method part includes: obtaining the user's expected image style and expected content description of the expected image; performing image generation based on the expected content description and expected image style, and obtaining the initial Generate an image, and perform semantic extraction on the desired content description to obtain key image information; perform image content recognition on the initial generated image to obtain content recognition data, and perform style recognition on the initial generated image to obtain the image recognition style; determine the image recognition style and expectations When the image style is consistent and the content recognition data matches the key image information, the initial generated image is output as the target retrieval image corresponding to the expected image. In the present invention, not only can the user retrieve more satisfactory image retrieval data, improve the image retrieval effect, but also improve the flexibility of image data acquisition.

Description

Data processing method, device, computer equipment and storage medium

technical field

The invention relates to the technical field of artificial intelligence, in particular to a data processing method, device, computer equipment and storage medium.

Background technique

In recent years, with the rapid development of the Internet, how to quickly find the information that users need has become a key issue in the utilization and management of big data, thus deriving artificial intelligence systems based on Internet big data. In artificial intelligence systems, people usually use retrieval methods to find useful information, such as text retrieval, image retrieval, etc.

Taking image retrieval as an example, the commonly used retrieval methods include retrieval based on text keywords and retrieval based on input images. The retrieval based on text keywords means that the user enters the query text, and the system matches the user query text with the text label of the image data. , so as to obtain the image retrieval result; the retrieval based on the input image means that the user inputs a standard image, and the system calculates the similarity between the standard image and the image data to obtain the image retrieval result. However, these two retrieval methods can only provide users with image data that already exists in the database, and require a large amount of manual annotation or calculation on the image data set, which limits the flexibility of image data acquisition and makes it difficult to obtain ideal images. Retrieval results, poor image retrieval.

Contents of the invention

The present invention provides a data processing method, device, computer equipment and storage medium to solve the problem that in the existing image retrieval process, only existing image data can be provided to users, the flexibility of image data acquisition is low, and the image retrieval effect is not good. good question.

Provide a data processing method, including:

Obtain the user's image description information of the desired image, the image description information includes the desired image style and desired content description;

Image generation is performed based on the desired content description and the desired image style to obtain an initial generated image, and semantic extraction is performed on the desired content description to obtain key image information corresponding to the desired content description;

Performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initially generated image to obtain the image recognition style;

Determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information;

If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as the target retrieval image corresponding to the expected image.

Optionally, the key image information includes multiple expected elements of the expected image, and determining whether the content identification data matches the key image information includes:

Recognize the constituent elements of the content recognition data to obtain multiple constituent elements of the initially generated image;

Match multiple expected elements with multiple constituent elements one by one to obtain a matching result;

If the element matching result is an exact match, then determining that the content identification data matches the key image information;

If the element matching result is an incomplete match, it is determined that the content identification data does not match the key image information.

Optionally, the key image information also includes the position and color of each expected element, and the multiple expected elements are matched with multiple constituent elements one by one to obtain a matching result, including:

Determine whether a plurality of constituent elements and a plurality of expected elements can be matched one-to-one;

If the plurality of constituent elements can be matched one-to-one with the plurality of expected elements, determining the position and color of each constituent element based on the content identification data;

Determine whether the position and color of each component element match the position and color of the corresponding expected element;

If the position and color of each component element match the position and color of the corresponding expected element, then determine that the matching result is a complete match;

If the plurality of constituent elements cannot be matched one-to-one with the plurality of expected elements, or the position or color of any constituent element does not match the position or color of the corresponding expected element, then it is determined that the matching result is an incomplete match.

Optionally, determining whether the position and color of each constituent element matches the position and color of the corresponding expected element includes:

Calculate the distance between the position of each component element and the position of the corresponding expected element, and obtain the position distance of each component element;

Calculate the similarity between the color of each component element and the color of the corresponding desired element to obtain the color difference of each component element;

Determine whether the position distance of each component element is less than a preset distance, and determine whether the color difference of each component element is less than a preset color difference value;

If the position distance of each component element is less than the preset distance, and the color difference of each component element is less than the preset color difference value, then determine the position and color of each component element, and match the position and color of the corresponding desired element.

Optionally, the key image information also includes an expected background description, matching multiple expected elements with multiple constituent elements one by one, and after obtaining the matching result, the method further includes:

If the matching result is a complete match, background recognition is performed on the content recognition data to obtain an image background description of the initially generated image;

determining the background similarity of the desired background description to the image background description;

If the background similarity is greater than or equal to the preset background similarity, it is determined that the content recognition data matches the key image information;

If the background similarity is smaller than the preset background similarity, it is determined that the content recognition data does not match the key image information.

Optionally, the desired background description includes a desired background color and a desired background pattern, the image background description includes identifying the background color and identifying the background pattern, and determining the background similarity between the expected background description and the image background description includes:

Calculate the background color similarity between the expected background color and the recognized background color, and calculate the pattern similarity between the expected background pattern and the recognized background pattern;

The background similarity is calculated based on the background color similarity and the pattern similarity.

Optionally, image generation is performed based on the desired content description and desired image style to obtain an initial generated image, including:

Obtain the target image generation model based on the deep learning of the generative confrontation network based on the model sample data set. Each historical image sample in the model sample data set corresponds to a sample content description vector and a sample style vector. Based on the sample content of multiple historical image samples The description vector and the sample style vector perform deep learning training on the generative confrontation network to obtain the target image generation model;

Perform semantic feature extraction on the desired content description to obtain an image description vector, and perform semantic feature extraction on the desired image style to obtain a desired image style vector;

Input the desired image style vector and image description vector into the target image generation model for image generation, and obtain the image output by the target image generation model as the initial generated image.

A data processing device is provided, comprising:

An acquisition module, configured to acquire the user's image description information for the desired image, where the image description information includes a desired image style and a desired content description;

A generating module, configured to generate an image based on the desired content description and the desired image style, obtain an initial generated image, and perform semantic extraction on the desired content description to obtain key image information corresponding to the desired content description;

The recognition module is used to perform image content recognition on the initially generated image to obtain content recognition data, and perform style recognition on the initially generated image to obtain the image recognition style;

A determining module, configured to determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information;

An output module, configured to output the initially generated image as the target retrieval image corresponding to the expected image if the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information.

Provided is a computer device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein the processor implements the steps of the above data processing method when executing the computer program.

Provided is a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above data processing method are realized.

In one solution provided by the above data processing method, device, computer equipment, and storage medium, by obtaining the image description information of the user for the desired image, the image description information includes the desired image style and the desired content description, based on the desired content description and the desired image style Perform image generation to obtain the initial generated image, and perform semantic extraction on the desired content description to obtain key image information corresponding to the desired content description, then perform image content recognition on the initial generated image to obtain content recognition data, and perform style recognition on the initially generated image , get the image recognition style, then determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information, if the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, Then output the initial generated image as the target retrieval image corresponding to the desired image; in the present invention, when the user performs desired image retrieval, the initial generated image is generated based on the desired image style and desired content description input by the user, and when it is determined that the initially generated image conforms to When the user describes the image information of the desired image, the generated initial generated image is output as the image retrieval result, which not only enables the user to retrieve more satisfactory image retrieval data, improves the image retrieval effect, but also improves the flexibility of image data acquisition.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention , for those skilled in the art, other drawings can also be obtained according to these drawings without paying creative labor.

Fig. 1 is a schematic diagram of an application environment of a data processing method in an embodiment of the present invention;

Fig. 2 is a schematic flow chart of a data processing method in an embodiment of the present invention;

FIG. 3 is a schematic diagram of an implementation process of step S20 in FIG. 2;

FIG. 4 is a schematic diagram of an implementation process of step S40 in FIG. 2;

FIG. 5 is a schematic flow diagram of another implementation of step S40 in FIG. 2;

Fig. 6 is a schematic structural diagram of data processing in an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a computer device in an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The data processing method provided by the embodiment of the present invention can be applied in the application scenario shown in FIG. 1 , where the server and the terminal device communicate through a network. Wherein, the server may be a background server of a preset editing platform, and the preset editing platform is a platform that provides design functions of visual digital products such as H5 pages or banners. When the user needs to perform image retrieval for visual digital product design, the user can input the image description information of the desired image through the terminal device. The image description information includes the desired image style and desired content description of the desired image, so that the server can perform image retrieval based on the image description information. Generate and retrieve tasks, so as to generate suitable images for users, and then return the generated images and matching images as image retrieval results to the user's terminal device, which is convenient for users to select images. After obtaining the user's image description information for the desired image, the server generates the image based on the desired content description and the desired image style in the image description information to obtain the initial generated image, and at the same time extracts the semantics of the desired content description to obtain the desired content description. Key image information; then, the server performs image content recognition on the initial generated image to obtain content recognition data, and performs style recognition on the initial generated image to obtain the image recognition style; determines whether the image recognition style is consistent with the expected image style, and determines the content recognition data Whether it matches the key image information, if the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, then output the initial generated image as the target retrieval image corresponding to the expected image. In this embodiment, when the user retrieves the desired image, the initial generated image is generated based on the desired image style and desired content description input by the user, and when it is determined that the initially generated image conforms to the user's image description information for the desired image, the generated initial The generated image is output as the image retrieval result, which not only enables the user to retrieve more satisfactory image retrieval data, improves the image retrieval effect, but also improves the flexibility of image data acquisition.

Wherein, the terminal device may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers and other devices. The server can be implemented by an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2 , a data processing method is provided, which is described by taking the method applied to the server in FIG. 1 as an example, including the following steps:

S10: Acquiring the user's image description information of the desired image, the image description information including the desired image style and desired content description.

When the user needs to search for an image, the user needs to input the keyword (text information or voice information) of the desired image through the terminal device to search for the image based on the image description information, and then select the appropriate image according to the image retrieval result. The traditional retrieval method In this method, the image retrieval result returned by the server is usually the image data that already exists in the database, which leads to the limitation of the flexibility of image data acquisition, unsatisfactory image retrieval results, and it is difficult to find satisfactory images for users.

In order to solve the aforementioned problems, in this embodiment, when the user has an image retrieval requirement, for example, when the user edits H5, posters and event pages on the preset editing platform, it needs to add or replace an image picture that the user is satisfied with, and this can be used The existing search engine on the preset editing platform is used for image search, so as to search for images satisfactory to users. During the search process, the user can input the user's image description information of the desired image into the search engine of the preset editing platform through the terminal device, and the server of the preset editing platform obtains the user's image description information of the desired image, and according to the image description information (The image description information includes the user's desired image style and desired content description of the desired image) to generate an image; at the same time, the server will also perform database image matching according to the traditional method, that is, extract the key fields in the image description information, and use the keyword Segments are matched with the labels of existing images (historical images) in the database, and then the matched images are obtained, so that the generated images and the matched images can be returned to the user as image retrieval results, thereby improving the efficiency of image data acquisition. Flexibility, improve image retrieval effect.

In this embodiment, the image description information is the user's description of the content of the image that needs to be generated (i.e. the desired image), including the user's desired image style and desired content description of the desired image, and the desired content description can be an image element (a component element of the image) ), image background (can include background graphics, background texture, background color, etc.), color and position of image elements, etc. Wherein, the image description information is natural language information, which can be either text information or voice information.

S20: Generate an image based on the desired content description and the desired image style to obtain an initial generated image, and perform semantic extraction on the desired content description to obtain key image information corresponding to the desired content description.

After obtaining the user's image description information for the desired image, that is, after obtaining the user's desired image style and desired content description for the desired image, the server needs to generate an image based on the desired content description and desired image style to obtain the initial generated image. The content also takes into account the marketing effect of the image style on the generation effect, so that the initial generated image can meet the user's requirements to the greatest extent. In this embodiment, based on the desired content description and desired image style, the pre-trained target image generation model can be used to perform the image generation task, so as to obtain the initial generated image. For example, the target image generation model can be a model based on a variational autoencoder, which encodes the desired content description to obtain the feature vector of the desired content description, and encodes the desired image style to obtain the feature vector of the desired image style, and then uses the target The image generation model performs high-dimensional pixel mapping on the feature vectors of the desired content description and desired image style to obtain the initial generated image. In other embodiments, the preset image generation model may also be other models for image generation through text, which will not be repeated here.

At the same time, the server also needs to perform semantic extraction on the desired content description to obtain key image information corresponding to the desired content description. Among them, the key image information includes information such as the components of the desired image, the image background, and the color and position of the components; through rich key image information, more details of the desired image can be obtained, so that the image generated by the subsequent model can be better. The mining image granularity can generate better images.

Wherein, performing semantic extraction on the desired content description to obtain the key image information corresponding to the desired content description includes: obtaining the editing data of the user's editing interface on the preset editing platform, and obtaining the user portrait and the user's image on the preset editing platform Historical behavior data; in the process of semantic extraction of the desired content description by the server, semantic association is performed based on data such as knowledge graphs, user editing data on the editing interface, user portraits, and user historical behavior data to obtain the key corresponding to the desired content description image information. That is, in the process of semantic extraction of the desired content description by the server, if it is detected that the user has editing data on the preset editing platform (such as a poster being edited or an H5 work), the editing data of the user on the editing interface can be obtained, and then The server can perform semantic association based on data such as knowledge graphs, user editing data on the editing interface, user portraits, and user historical behavior data, so as to obtain more complete key image information. In the process of semantic extraction of the desired content description by the server, semantic association based on the knowledge map can intelligently expand the desired content description from the perspective of the map, thereby enriching the user's intention, increasing the accuracy of the desired content description, and improving the key image information accuracy. In semantic extraction, user portraits and user historical behavior data are added on the basis of knowledge graphs for semantic association, which can further expand the description of desired content, making the identified user intentions closer to the user's behavior preferences, and then improve the key image. The accuracy of the information, thereby improving the image generation effect. In semantic extraction, on the basis of the knowledge map, adding the user’s editing data on the editing interface for semantic association can not only further expand the description of the desired content, but also make the identified user intent more in line with the user’s current editing needs, thereby enabling subsequent The generated images are more in line with user needs.

S30: Perform image content recognition on the initially generated image to obtain content recognition data, and perform style recognition on the initially generated image to obtain an image recognition style.

After the image is generated based on the desired content description and desired image style to obtain the initial generated image, the server needs to perform image content recognition on the initially generated image to obtain content recognition data; at the same time, the server also needs to perform style recognition on the initially generated image to obtain Image recognition style.

In this embodiment, a pre-trained style recognition model may be used to perform style recognition on the initially generated image, so as to quickly obtain the image recognition style of the initially generated image on the basis of ensuring recognition accuracy. Among them, the pre-trained content recognition model can be used to identify the image content such as composition elements, element colors, element positions, and backgrounds of the initial generated image, so as to quickly obtain accurate and diverse content recognition data.

S40: Determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information.

After identifying the content recognition data and image recognition style of the initial generated image, the server also needs to determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information, so as to confirm whether the generated initial generated image is Satisfy the image description information input by the user, that is, confirm whether the initially generated image meets the user description requirements.

S50: If the image recognition style is consistent with the desired image style, and the content recognition data matches the key image information, output the initially generated image as the target retrieval image corresponding to the desired image.

After determining whether the image recognition style is consistent with the expected image style, and determining whether the content recognition data matches the key image information, if it is determined that the image recognition style is consistent with the expected image style, and it is determined that the content recognition data matches the key image information, it means that the initial generation The image style of the image is consistent with the image style required in the image description information input by the user, and the image content of the initially generated image is the image content required in the image description information input by the user, that is, the generated initial generated image meets the user description requirements, then The initial generated image is output as the target retrieval image corresponding to the desired image, so that the target retrieval image fully meets the user's description requirements, and is closer to the user's desired image, which improves the image generation effect of the target retrieval image.

At the same time, the server will also perform database image matching according to the traditional method, that is, extract the key fields in the image description information, match the key fields with the labels of existing images (historical images) in the database, and then obtain the matched image. In order to subsequently return the generated image and the matched image to the user as an image retrieval result, thereby improving the flexibility of image data acquisition and improving the image retrieval effect. After outputting the initial generated image as the target retrieval image corresponding to the desired image, the server returns the initial generated image together with the image matched in the database as the image retrieval result to the user, thereby improving the flexibility of image data acquisition and improving the image quality. Retrieve the effect.

If it is determined that the image recognition style of the initially generated image is inconsistent with the expected image style, it means that the image style of the initially generated image is inconsistent with the image style required in the image description information input by the user, and the generated initial generated image does not meet the user's description requirements, then the initial The effect of the generated image is not good. The server needs to regenerate a new initial generated image according to the image description information input by the user. After the new initial generated image meets the user description requirements, the initial generated image is output as the target retrieval image corresponding to the desired image. Or, if it is determined that the content identification data of the initially generated image does not match the key image information, for example, the image elements of the initially generated image do not match (missing image elements described in the desired content, adding image elements not included in the desired content description), If the position or color of the same element does not match, the image background does not match, etc., it means that the image content of the initially generated image is not the image content required in the image description information input by the user, then the effect of the initially generated image is not good, and the server needs to rely on user input The image description information regenerates a new initial generated image until the new initial generated image satisfies the description requirements of the user, and outputs the initial generated image as the target retrieval image corresponding to the desired image. When it is determined that the image recognition style is inconsistent with the expected image style, or the content recognition data does not match the key image information, the server generates a new image again, which can ensure the image generation effect.

In this embodiment, when the user performs image retrieval, by obtaining the user's image description information of the desired image, the image description information includes the desired image style and the desired content description, and the image is generated based on the desired content description and the desired image style, and the initial generation image, and perform semantic extraction on the desired content description to obtain the key image information corresponding to the desired content description, then perform image content recognition on the initial generated image to obtain content recognition data, and perform style recognition on the initially generated image to obtain the image recognition style, and then Determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information. If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as The target retrieval image corresponding to the desired image; in the present invention, when the user performs desired image retrieval, an initial generated image is generated based on the desired image style and desired content description input by the user, and when it is determined that the initially generated image conforms to the user's image description of the desired image When information is generated, the generated initial image is output as the image retrieval result, which not only enables the user to retrieve more satisfactory image retrieval data, improves the image retrieval effect, but also improves the flexibility of image data acquisition.

In one embodiment, as shown in FIG. 3, in step S20, the image is generated based on the desired content description and the desired image style to obtain an initially generated image, which specifically includes the following steps:

S21: Obtain a target image generation model obtained by performing deep learning on the generative adversarial network based on the model sample data set.

After obtaining the user's image description information for the desired image, the server also needs to obtain a pre-trained target image generation model, so as to call the target image generation model to generate images based on the desired image style and desired content description. Among them, the target image generation model is a target image generation model obtained by performing deep learning on the generative confrontation network based on the model sample data set. The model sample data set includes a plurality of historical image samples, and each historical image sample in the model sample data set corresponds to a sample content description vector and a sample style vector.

In the process of model training, based on the sample content description vector and sample style vector of multiple historical image samples, deep learning training is carried out on the generative confrontation network, and the target image generation model is obtained. Among them, the generation confrontation network can be a general neural network structure including a generation module and a discrimination module; The module generates images based on the sample content description vector and the sample style vector; the discrimination module is used to judge the authenticity of the image generated by the generation module, and calculates the loss function based on the historical image samples until the loss function converges and outputs the generation module as the target image generation model .

S22: Perform semantic feature extraction on the desired content description to obtain an image description vector, and perform semantic feature extraction on the desired image style to obtain a desired image style vector.

After obtaining the pre-trained target image generation model, it is necessary to perform semantic feature extraction on the desired content description to obtain an image description vector, and perform semantic feature extraction on the desired image style to obtain the desired image style vector.

Among them, in the process of extracting the semantic features of the desired content description and/or desired image style, semantic association can be performed based on data such as the knowledge graph, user editing data on the editing interface, user portraits, and user historical behavior data to obtain The expected content description and/or expected image style after semantic association, and then perform feature encoding on the expected content description and/or expected image style after semantic association, and correspondingly obtain an image description vector and an expected image style vector. In the process of server extracting the semantic features of the desired content description, semantic association based on the knowledge map can intelligently expand the desired content description from the perspective of the map, thereby enriching the user's intention, increasing the diversity of the desired content description, and improving the image description. Vector, the accuracy of the expected image style vector, in order to improve the generation effect of subsequent images. When extracting semantic features, adding user portraits and user historical behavior data on the basis of knowledge graphs for semantic association can further expand the description of expected content, making the identified user intentions closer to the user's behavior preferences, and further improving the user experience. Corresponding to the accuracy of the vector, thereby improving the image generation effect. In semantic extraction, on the basis of the knowledge map, adding the user’s editing data on the editing interface for semantic association can not only further expand the description of the desired content, but also make the identified user intent more in line with the user’s current editing needs, thereby enabling subsequent The generated images are more in line with user needs.

S23: Input the desired image style vector and the image description vector into the target image generation model for image generation, and obtain an image output by the target image generation model as an initial generated image.

After obtaining the image description vector and the desired image style vector, input the desired image style vector and image description vector into the target image generation model for image generation, and obtain the image output by the target image generation model as the initial generated image.

In this embodiment, by obtaining the target image generation model obtained by performing deep learning on the generative adversarial network based on the model sample data set, and then performing semantic feature extraction on the desired content description to obtain an image description vector, and performing semantic feature extraction on the desired image style , get the desired image style vector, then input the desired image style vector and image description vector into the target image generation model for image generation, and obtain the image output by the target image generation model as the initial generated image; it is clear based on the expected content description and expected image style to generate an image, and obtain the specific steps of initial image generation. Based on the sample content description vector and sample style vector of multiple historical image samples, the high-precision target image generation model is obtained by using the generative confrontation network training, and the model is called directly in the subsequent call. Image generation, simple and fast with high image quality.

In a preferred embodiment, the sample content description vector of the historical image sample includes a semantic vector of sample constituent elements, a position vector and a color vector of sample constituent elements. The generation confrontation network includes a generation module, a style transfer network and a discrimination module. The style transfer network and the discrimination module are used to assist the generation module to generate images, so as to reduce the difference between the generated images and the historical images in terms of the desired image style, the position and color of image elements. sample gap. In the process of model training, the semantic vector of the sample constituent elements in the historical image sample, the position vector and the color vector of the sample constituent elements are input into the generation module to generate the image, and then the style transfer network is introduced to perform rendering based on the sample style vector, which is used in the generation module Generate image style information; finally, use the discriminant module to measure the difference between the generated image and the historical image samples, which is used to guide the image generated by the generation module to be closer to the historical image samples in terms of image style, image components, positions and colors. After the loss function converges, the converged generator and style transfer network are output as the target image generation model. Its structure enables the target image generation model to mine image information and generate images from fine-grained, visual quality and image granularity of generated images. All performed well. Correspondingly, in the process of extracting semantic features from the desired content description to obtain the image description vector, it is necessary to extract the text vector, the position vector and the color vector of the desired element of the desired image from the desired content description, that is, the image description vector It can include the text vector of the desired element, the position vector and the color vector of the desired element, and then after training the target image generation model, input the desired image style vector, the semantic vector of the desired element, the position vector and the color vector of the desired element into the target The image generation model performs image generation, and obtains the image output by the target image generation model as the initial generated image corresponding to the desired image, and can obtain an image with good visual quality and image granularity, which improves the initial generated image. generation effect.

In an embodiment, the key image information includes a plurality of desired elements of the desired image. As shown in Figure 4, in step S40, it is determined whether the content recognition data matches the key image information, specifically including the following steps:

S41: Recognize the constituent elements of the content recognition data to obtain multiple constituent elements of the initially generated image;

S42: Match multiple expected elements with multiple constituent elements one by one to obtain a matching result;

S43: If the element matching result is an exact match, determine that the content identification data matches the key image information;

S44: If the element matching result is an incomplete match, determine that the content identification data does not match the key image information.

In this embodiment, the key image information includes multiple expected elements of the expected image. After performing image content recognition on the initial generated image to obtain the content recognition data, it is necessary to perform element recognition on the content recognition data to obtain multiple constituent elements of the initial generated image; match to obtain the matching result; if the element matching result is a complete match, it means that the component element set of the initially generated image completely coincides with the expected element set of the expected image, and the component elements of the initially generated image are consistent with the user's expected image, then determine the content The recognition data matches the key image information; if the element matching result is not completely matched, it means that the element set of the initially generated image does not completely coincide with the expected element set of the expected image, and the information such as the elements of the initially generated image does not match the user's expected image , it is determined that the content-aware data does not match the key image information.

For example, the plurality of desired elements include element A, element B, and element C, and the plurality of constituent elements of the initially generated image are element A, element B, and element D, that is, the composition element set of the initially generated image and the desired element set of the desired image If they are not completely overlapped, it is determined that the content identification data does not match the key image information; if the multiple elements of the initially generated image are also element A, element B, and element C, then it is determined that the content identification data does not match the key image information.

In this embodiment, by identifying the constituent elements of the content recognition data, multiple constituent elements of the initially generated image are obtained, and then the multiple expected elements are matched one by one with the multiple constituent elements to obtain the matching result. If the element matching result is Complete match, it is determined that the content recognition data matches the key image information, if the element matching result is not a complete match, then it is determined that the content recognition data does not match the key image information, and the details of determining whether the content recognition data matches the key image information are refined. step. When performing image data matching, one-to-one matching is performed on the constituent elements of the image, which ensures the accuracy of the matching results and also provides a judgment basis for subsequent data processing.

In other embodiments, the key image information may also include one or more information such as the position and color of each desired element, and the background of the desired image. Correspondingly, the content identification data of the initial generated image also includes the initial generated image One or more information such as the position and color of each component element, and the background of the initial generated image. When matching the initial generated image with the user's desired image, not only the image element itself is matched, but other Information is matched to increase the granularity of information matching, thereby increasing the accuracy of matching results.

In an embodiment, it is taken that the key image information further includes the position and color of each desired element as an example. In step S42, the multiple expected elements are matched one by one with multiple constituent elements to obtain the matching result, which specifically includes the following steps:

S421: Determine whether a plurality of constituent elements and a plurality of expected elements can be matched one-to-one;

S422: If the multiple constituent elements can be matched one-to-one with the multiple expected elements, then determine the position and color of each constituent element based on the content identification data;

S423: Determine whether the position and color of each component element match the position and color of the corresponding desired element;

S424: If the position and color of each component element match the position and color of the corresponding expected element, determine that the matching result is a complete match;

S425: If the position or color of any component element does not match the position or color of the corresponding expected element, determine that the matching result is an incomplete match.

After identifying the constituent elements of the content recognition data and obtaining the multiple constituent elements of the initially generated image, determine whether the multiple constituent elements and the multiple expected elements can be matched one-to-one. If the multiple constituent elements and the multiple expected elements cannot One-to-one correspondence matching, that is, the element set of the initially generated image does not completely coincide with the expected element set of the expected image, indicating that the elements of the initially generated image do not match the elements expected by the user, and it is determined that the content recognition data does not match the key image information.

If multiple constituent elements and multiple expected elements can be matched one-to-one, that is, the set of constituent elements of the initially generated image does not completely coincide with the set of expected elements of the expected image, which means that the constituent elements of the initially generated image match the elements expected by the user. At this time Additional information is required for further judgment. Since the key image information in this embodiment includes multiple desired elements, and the position and color of each desired element, it is also necessary to determine the position and color of each component element based on the content recognition data. Then, determine whether the position and color of each constituent element match the position and color of the corresponding expected element; if the position and color of each constituent element match the position and color of the corresponding expected element, then determine that the matching result is a complete match ; If the position or color of any component element does not match the position or color of the corresponding expected element, then determine the matching result as an incomplete match.

For example, the multiple expected elements include element A and element B, and the multiple constituent elements of the initially generated image are element A and element B. If the color and position of element A in the expected image are the same as the color and position of element A in the initially generated image; the color and position of element B in the expected image are the same as the color and position of element B in the initially generated image, then the matching result is determined to be match exactly. If the color or position of element A (or B) in the expected image is different from the color or position of element A (or B) in the initially generated image; or, the color and position of element A in the expected image are different from the initially generated image The color and position of element A in the image are the same, but the color or position of element B in the expected image is different from the color or position of element B in the initially generated image, and the matching result is determined to be an incomplete match.

In this embodiment, the key image information also includes the position and color of each desired element. By determining whether a plurality of constituent elements and a plurality of expected elements can be matched in one-to-one correspondence; if a plurality of constituent elements can be matched in a one-to-one correspondence with a plurality of expected elements, then determine the position and color of each constituent element based on the content identification data; determine Whether the position and color of each component element matches the position and color of the corresponding expected element; if the position and color of each component element match the position and color of the corresponding expected element, then determine that the matching result is a complete match; if more A component element cannot be matched one-to-one with multiple expected elements, or the position or color of any component element does not match the position or color of the corresponding expected element, then the matching result is determined to be an incomplete match, and multiple expected One-to-one matching between an element and multiple constituent elements to obtain the specific steps of the matching result. When performing element matching, not only the element itself is matched, but also the color and position of the element are matched, which increases the granularity of information matching, thereby increasing the accuracy of the matching result sex.

In addition, only when the position and color of each component element match the position and color of the corresponding expected element, the matching result is determined to be a complete match, which improves the accuracy of the matching result, thereby improving the accuracy of the subsequent output target retrieval image. Accuracy, thereby improving the user retrieval effect. In other embodiments, when the positions and colors of multiple constituent elements in the initially generated image are matched with the positions and colors of the corresponding desired elements, when the ratio of successful matching of the constituent elements is greater than a preset ratio (such as 80%), It can be determined that the matching result is an exact match. This method lowers the requirements for image accuracy. The initial generated image basically satisfies the user's description, which can quickly obtain target retrieval images that meet the requirements, increase the number and diversity of target retrieval images, and provide users with more image choices. .

In one embodiment, in step S423, it is determined whether the position and color of each component element match the position and color of the corresponding desired element, specifically including the following steps:

S4231: Calculate the distance between the position of each component element and the position of the corresponding expected element to obtain the position distance of each component element.

In this embodiment, the key image information includes a plurality of desired elements, the position and color of each desired element; when identifying the constituent elements of the content recognition data, the multiple constituent elements of the initially generated image, and the position and color of each constituent element are obtained. position and color. After determining that the plurality of constituent elements and the plurality of expected elements can be matched one-to-one, it is necessary to calculate the distance between the position of each constituent element and the position of the corresponding expected element to obtain the position distance of each constituent element. The larger the position distance, the greater the deviation between the position of the component element and the position of the corresponding expected element; the smaller the position distance, the smaller the deviation between the position of the component element and the position of the corresponding expected element, and the closer the element positions.

Among them, the Euclidean distance algorithm can be used to calculate the direct shortest distance between two locations when calculating the location distance, which is simple and intuitive. In other embodiments, other distance calculation methods may also be used, such as cosine distance algorithm, standard Euclidean distance algorithm, and the like.

S4232: Calculate the similarity between the color of each constituent element and the color of the corresponding desired element, and obtain the color difference of each constituent element.

After it is determined that the plurality of constituent elements and the plurality of expected elements can be matched one-to-one, it is also necessary to calculate the similarity between the color of each constituent element and the color of the corresponding expected element to obtain the color difference of each constituent element. Among them, the color difference algorithm can be a color difference calculation method based on three-dimensional space. The implementation process is usually: convert the RGB color space into a normalized hsv color space, convert the three-dimensional coordinate points of the hsv color space, and then calculate the three-dimensional coordinates of the two colors The distance between coordinate points in space, and the distance between coordinate points in three-dimensional space is used as the color difference between two colors; when the two colors are closer, the coordinate distance between the two colors is closer to 0, on the contrary, when the two colors are farther apart, The closer the coordinate distance is to 1. Among them, HSV (Hue, Saturation, Value) is a color space created according to the intuitive characteristics of color, also known as the hexagonal pyramid model. The parameters of color in HSV are hue (H), saturation (S) and brightness (V).

S4233: Determine whether the position distance of each component element is less than a preset distance, and determine whether the color difference of each component element is less than a preset color difference value.

The smaller the position distance between the constituent elements, the closer the position of the constituent elements is to the user's expected position, and the higher the image accuracy; the smaller the color difference, the closer the colors of the constituent elements are, and the closer to the user's expected element color, The higher the image accuracy. After obtaining the position distance of each component element and the color difference of each component element, it is determined whether the position distance of each component element is less than a preset distance, and whether the color difference of each component element is less than a preset color similarity.

S4234: If the position distance of each component element is less than the preset distance, and the color difference of each component element is less than the preset color difference value, determine the position and color of each component element, and the position and color of the corresponding desired element match.

After determining whether the position distance of each component element is less than the preset distance, and determining whether the color difference of each component element is less than the preset color difference value, if the position distance of each component element is less than the preset distance, it means that the initial generated image The difference between the position of each component element in and the user's expected position is small and negligible, and the color difference of each component element is less than the preset color difference value, which means that the color difference between the color of each component element in the initially generated image and the user's expected color is small , it is difficult for human eyes to detect the difference in color, then determine the position and color of each component element, and match the position and color of the corresponding expected element, that is, it can be determined that the matching result is a complete match.

If the position distance of any component element is greater than or equal to the preset distance, it means that the position of the component element in the initially generated image is far from the user's expected position, and the position of the component element does not match the position of the corresponding expected element, then OK The matching result is an incomplete match; if the color difference of any component element is greater than or equal to the preset color difference value, it means that the color of the component element in the initial generated image is far from the user's expected color, and the color of the component element is different from the corresponding expected color. If the colors of the elements do not match, the matching result is determined to be an incomplete match.

Wherein, in order to ensure the image accuracy of the generated small-sized image, the preset distance may be less than 1 cm, such as 0.9 cm, 0.8 cm, 0.5 cm. In order to reduce the naked eye error caused by different colors, the preset color difference value needs to be less than 0.5; in order to reduce the color difference and make the color of the component elements closer to the user's desired color, thereby improving image accuracy, the preset color difference value can be 0.3.

In this embodiment, by calculating the distance between the position of each component element and the position of the corresponding desired element, the position distance of each component element is obtained, and the similarity between the color of each component element and the color of the corresponding desired element is calculated , get the color difference of each component element, then determine whether the position distance of each component element is less than the preset distance, and determine whether the color difference of each component element is less than the preset color difference value, if the position distance of each component element is less than The preset distance, and the color difference of each component element is less than the preset color difference value, then determine the position and color of each component element, and match the position and color of the corresponding expected element; clearly determine the position and color of each component element The specific implementation of whether the color matches the position and color of the corresponding expected element provides an accurate data basis for subsequent judgments.

In other embodiments, the key image information may also include one or more information such as the position and color of each desired element, and the background of the desired image. Correspondingly, the content identification data of the initial generated image also includes the initial generated image One or more information such as the position and color of each component element in the image, and the background of the initially generated image. When matching elements one by one, other information needs to be matched to increase the granularity of information matching, thereby increasing the matching the accuracy of the results.

In one embodiment, it is taken that the key image information further includes the position and color of each desired element as an example. In step S42, the multiple expected elements are matched one by one with the multiple constituent elements to obtain the matching result, which specifically includes the following steps:

In an embodiment, it is taken that the key image information also includes a desired background description (the background of the desired image) as an example. As shown in Figure 5, after step S42, the multiple expected elements are matched one by one with multiple constituent elements, after the matching result is obtained, before it is determined that the content identification data matches the key image information, the method further specifically includes the following steps:

S45: If the matching result is a complete match, perform background recognition on the content recognition data to obtain an image background description of the initially generated image.

After identifying the constituent elements of the content recognition data and obtaining the multiple constituent elements of the initially generated image, determine whether the multiple constituent elements and the multiple expected elements can be matched one-to-one. If the multiple constituent elements and the multiple expected elements cannot One-to-one correspondence matching, determining that the content recognition data does not match the key image information.

If the plurality of constituent elements can be matched one-to-one with the plurality of expected elements, it means that the constituent element information of the initially generated image is consistent with the user's expected element information, and other information needs to be further judged at this time. Since the key image information in this embodiment includes a plurality of expected elements and expected background descriptions, that is, to match each component element of the initially generated image, it is also necessary to determine the image background of the initially generated image based on the content recognition data (denoted as image background description ), so as to perform background matching and determine whether the content recognition data matches the key image information based on the background matching result, that is, determine whether the initially generated image meets the user description requirements.

If multiple expected elements are matched one by one with multiple constituent elements and the matching result is a complete match, it is also necessary to perform background recognition on the content recognition data to obtain the image background description of the initially generated image. The image background description is a semantic description of the background of the initially generated image.

S46: Determine the background similarity between the expected background description and the image background description.

Then, determine the background similarity between the expected background description and the image background description, that is, determine the similarity between the image background of the initially generated image and the image background expected by the user, that is, calculate the background sum of the initially generated image based on the expected background description and the image background description Similarity to the background of the desired image.

For example, semantic extraction can be performed on the desired background description and the image background description respectively to obtain multiple expected background keywords and multiple image background keywords; then, match the multiple expected background keywords with multiple image background keywords respectively , get the keyword matching rate, and use the keyword matching rate as the background similarity.

S47: If the background similarity is greater than or equal to the preset background similarity, determine that the content recognition data matches the key image information;

S48: If the background similarity is smaller than the preset background similarity, determine that the content recognition data does not match the key image information.

After determining the background similarity between the desired background description and the image background description, if the background similarity is greater than or equal to the preset background similarity, it means that the image background of the initially generated image is relatively close to the user's desired image background, and then the content recognition can be determined. The data matches the key image information; if the background similarity is less than the preset background similarity, it means that the image background of the initially generated image is far from the image background expected by the user. At this time, it can be determined that the content recognition data does not match the key image information.

In this embodiment, the key image information also includes the expected background description. After matching multiple expected elements with multiple constituent elements one by one to obtain the matching result, if the matching result is a complete match, perform background recognition on the content recognition data. , get the image background description of the initially generated image; determine the background similarity between the expected background description and the image background description; if the background similarity is greater than or equal to the preset background similarity, then determine that the content recognition data matches the key image information; if the background is similar If the degree is less than the preset background similarity, it is determined that the content recognition data does not match the key image information. The specific steps to determine whether the content recognition data matches the key image information are clarified. On the basis of judging the matching of the image composition elements, the matching of the image background is also considered, and the image effect of the initial generated image is multi-dimensionally inspected to improve the image quality. the accuracy of judgment.

In an embodiment, the desired background description includes a desired background color and a desired background pattern, and the image background description includes identifying a background color and identifying a background pattern. In step S46, determine the background similarity between the desired background description and the image background description, specifically including the following steps:

S461: Calculate the background color similarity between the expected background color and the recognized background color, and calculate the pattern similarity between the expected background pattern and the recognized background pattern.

In this embodiment, the desired background description includes a desired background color and a desired background pattern. Correspondingly, when background recognition is performed on the content recognition data, the image background description of the initially generated image also includes recognition of the background color and recognition of the background pattern. When determining the background similarity between the expected background description and the image background description, it is necessary to calculate the background color similarity between the expected background color and the recognized background color, and to calculate the pattern similarity between the expected background pattern and the recognized background pattern.

Among them, before calculating the background color similarity between the expected background color and the recognized background color, it is necessary to obtain a pre-built color relationship diagram, and then determine the background color similarity between the expected background color and the recognized background color based on the color relationship diagram. The color relationship graph is composed of multiple nodes and edges between nodes. The nodes in the color relationship graph represent colors, and the edges of two nodes represent the similarity between two colors. The closer the colors are, the greater the color similarity is. shorter. For example, the similarity between rose red and positive red is relatively large, and the two sides are short; the similarity between green and positive red is small, and the two sides are very small and close to 0. Therefore, the similarity between the expected background color and the recognized background color can be quickly and accurately determined through the color relationship diagram.

Among them, before calculating the pattern similarity between the expected background pattern and the recognized background pattern, it is necessary to obtain a pre-built pattern relationship graph, and then determine the pattern similarity between the expected background pattern and the recognized background pattern based on the pattern relationship graph. The pattern relationship graph is composed of a plurality of nodes and edges between the nodes. The nodes in the pattern relationship graph represent patterns (including textures, graphics and symbols), and the edges of two nodes represent the similarity of two patterns. The closer the patterns are, the closer the patterns are. The greater the pattern similarity, the shorter the side. For example, the similarity between the fine moire pattern and the coarse moire pattern is relatively large, and the sides of the two are relatively short; Therefore, the pattern similarity between the desired background pattern and the recognized background pattern can be quickly and accurately determined through the pattern relationship diagram.

S462: Calculate the background similarity based on the background color similarity and the pattern similarity.

After the background color similarity and the pattern similarity are calculated, it is necessary to calculate the background similarity based on the background color similarity and the pattern similarity. For example, the background color similarity and the pattern similarity can be averaged, and the obtained average can be used as the background similarity, which is simple and accurate. Alternatively, the sum of the background color similarity and the pattern similarity is used as the background similarity.

In other embodiments, it is also possible to set different weights for the background color similarity and the pattern similarity, and then perform weighted average calculation on the background color similarity and the pattern similarity according to the weight of the background color similarity and the weight of the pattern similarity , to get the background similarity. By calculating the background similarity by weighted average, the importance of color or pattern can be emphasized according to the requirement, so as to obtain a more accurate background similarity. Wherein, the weight can be dynamically determined according to the image description information of the user.

For example, if it is determined according to the user's image description information that the image that the user expects to generate (desired image) is a black and white image, it means that the user does not emphasize the color of the image, and at this time it can be determined that the weight of the pattern similarity is greater than the weight of the color similarity , emphasizing that the texture pattern of the image is more important. If it is determined according to the user's image description information that the image that the user expects to generate (desired image) is a color-based image, it means that the user emphasizes the color of the image. At this time, it can be determined that the weight of the pattern similarity is smaller than the weight of the color similarity, and the image is emphasized. The color is more important.

In this embodiment, the desired background description includes a desired background color and a desired background pattern, and the image background description includes a recognition background color and a recognition background pattern, by calculating the background color similarity between the desired background color and the recognition background color, and calculating the difference between the desired background pattern and the recognition background color Identify the pattern similarity of the background pattern, and then calculate the background similarity based on the background color similarity and pattern similarity, and clarify the specific steps to determine the background similarity between the desired background description and the image background description, based on the similarity between the background color and the pattern The degree can be calculated to obtain a more accurate background similarity, which provides a basis for subsequent judgments.

It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined by its functions and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present invention.

In an embodiment, a data processing device is provided, and the data processing device corresponds to the data processing method in the foregoing embodiments one by one. As shown in FIG. 6 , the data processing device includes an acquisition module 601 , a generation module 602 , an identification module 603 , a determination module 604 and an output module 605 . The detailed description of each functional module is as follows:

An acquisition module 601, configured to acquire the user's image description information for the desired image, the image description information including the desired image style and desired content description;

A generating module 602, configured to generate an image based on the desired content description and the desired image style, obtain an initial generated image, and perform semantic extraction on the desired content description, to obtain key image information corresponding to the desired content description;

The recognition module 603 is configured to perform image content recognition on the initially generated image to obtain content recognition data, and perform style recognition on the initially generated image to obtain an image recognition style;

A determining module 604, configured to determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information;

The output module 605 is configured to output the initially generated image as the target retrieval image corresponding to the expected image if the image recognition style is consistent with the desired image style, and the content recognition data matches the key image information.

Optionally, the key image information includes multiple expected elements of the expected image, and the determining module 604 is specifically configured to:

Recognize the constituent elements of the content recognition data to obtain multiple constituent elements of the initially generated image;

Match multiple expected elements with multiple constituent elements one by one to obtain a matching result;

If the element matching result is an exact match, then determining that the content identification data matches the key image information;

If the element matching result is an incomplete match, it is determined that the content identification data does not match the key image information.

Optionally, the key image information also includes the position and color of each desired element, and the determining module 604 is further configured to:

Determine whether a plurality of constituent elements and a plurality of expected elements can be matched one-to-one;

If the plurality of constituent elements can be matched one-to-one with the plurality of expected elements, determining the position and color of each constituent element based on the content recognition data;

Determine whether the position and color of each component element match the position and color of the corresponding expected element;

If the position and color of each component element match the position and color of the corresponding expected element, then determine that the matching result is a complete match;

If the plurality of constituent elements cannot be matched one-to-one with the plurality of expected elements, or the position or color of any constituent element does not match the position or color of the corresponding expected element, then it is determined that the matching result is an incomplete match.

Optionally, the determining module 604 is specifically further configured to:

Calculate the distance between the position of each component element and the position of the corresponding expected element, and obtain the position distance of each component element;

Calculate the similarity between the color of each component element and the color of the corresponding desired element to obtain the color difference of each component element;

Determine whether the position distance of each component element is less than a preset distance, and determine whether the color difference of each component element is greater than a preset color difference value;

If the position distance of each component element is less than the preset distance, and the color difference of each component element is greater than the preset color difference value, the position and color of each component element are determined to match the position and color of the corresponding desired element.

Optionally, the key image information also includes an expected background description, and one-to-one matching is performed between multiple expected elements and multiple constituent elements, and after the matching result is obtained, the determination module 604 is specifically further configured to:

If the matching result is a complete match, background recognition is performed on the content recognition data to obtain an image background description of the initially generated image;

determining the background similarity of the desired background description to the image background description;

If the background similarity is greater than or equal to the preset background similarity, it is determined that the content recognition data matches the key image information;

If the background similarity is smaller than the preset background similarity, it is determined that the content recognition data does not match the key image information.

Optionally, the desired background description includes a desired background color and a desired background pattern, the image background description includes identifying a background color and identifying a background pattern, and the determining module 604 is specifically further configured to:

Calculate the background color difference between the expected background color and the recognized background color, and calculate the pattern similarity between the expected background pattern and the recognized background pattern;

The background similarity is calculated based on the background color difference and the pattern similarity.

Optionally, the generating module 602 is specifically configured to:

Obtain the target image generation model based on the deep learning of the generative confrontation network based on the model sample data set. Each historical image sample in the model sample data set corresponds to a sample content description vector and a sample style vector. Based on the sample content of multiple historical image samples The description vector and the sample style vector perform deep learning training on the generative confrontation network to obtain the target image generation model;

Perform semantic feature extraction on the desired content description to obtain an image description vector, and perform semantic feature extraction on the desired image style to obtain a desired image style vector;

Input the desired image style vector and image description vector into the target image generation model for image generation, and obtain the image output by the target image generation model as the initial generated image.

For specific limitations on the data processing device, refer to the above-mentioned limitations on the data processing method, which will not be repeated here. Each module in the above-mentioned data processing device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

In one embodiment, as shown in FIG. 7 , a computer device is provided. The computer device may be a server, and the computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs and databases. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data used and generated by the above data processing method, such as image description information, key image information, initially generated images, and the like. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a data processing method is realized.

In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the following steps are implemented:

Obtain the user's image description information of the desired image, the image description information includes the desired image style and desired content description;

Image generation is performed based on the desired content description and the desired image style to obtain an initial generated image, and semantic extraction is performed on the desired content description to obtain key image information corresponding to the desired content description;

Performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initially generated image to obtain the image recognition style;

Determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information;

If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as the target retrieval image corresponding to the expected image.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Obtain the user's image description information of the desired image, the image description information includes the desired image style and desired content description;

Image generation is performed based on the desired content description and the desired image style to obtain an initial generated image, and semantic extraction is performed on the desired content description to obtain key image information corresponding to the desired content description;

Performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initially generated image to obtain the image recognition style;

Determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information;

If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as the target retrieval image corresponding to the expected image.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units, Completion of modules means that the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still carry out the foregoing embodiments Modifications to the technical solutions recorded in the examples, or equivalent replacement of some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should be included in within the protection scope of the present invention.

Claims (10)

1. A data processing method, characterized in that, comprising: Acquiring the user's image description information for the desired image, the image description information including the desired image style and desired content description; performing image generation based on the desired content description and the desired image style to obtain an initially generated image, and performing semantic extraction on the desired content description to obtain key image information corresponding to the desired content description; performing image content recognition on the initially generated image to obtain content recognition data, and performing style recognition on the initially generated image to obtain an image recognition style; determining whether the image recognition style is consistent with the desired image style, and determining whether the content recognition data matches the key image information; If the image recognition style is consistent with the desired image style, and the content recognition data matches the key image information, then output the initially generated image as the target retrieval image corresponding to the desired image. 2. The data processing method according to claim 1, wherein the key image information includes a plurality of expected elements of the desired image, and determining whether the content identification data matches the key image information, include: Identifying the constituent elements of the content recognition data to obtain a plurality of constituent elements of the initially generated image; matching the plurality of expected elements with the plurality of constituent elements one by one to obtain a matching result; If the element matching result is a complete match, then determining that the content identification data matches the key image information; If the element matching result is an incomplete match, it is determined that the content identification data does not match the key image information. 3. The data processing method according to claim 2, wherein the key image information also includes the position and color of each of the expected elements, and the combination of a plurality of the expected elements and a plurality of the components Elements are matched one by one to get matching results, including: Determining whether a plurality of the constituent elements and a plurality of the expected elements can be matched one-to-one; If a plurality of the constituent elements and a plurality of the desired elements can be matched one-to-one, then determining the position and color of each constituent element based on the content identification data; determining whether the position and color of each of the constituent elements matches the position and color of the corresponding desired element; If the position and color of each of the constituent elements matches the position and color of the corresponding desired element, then determining that the matching result is a complete match; If the position or color of any one of the constituent elements does not match the position or color of the corresponding desired element, it is determined that the matching result is an incomplete match. 4. The data processing method according to claim 3, wherein the determining whether the position and the color of each of the constituent elements matches the position and the color of the corresponding desired element comprises: calculating the distance between the position of each of the constituent elements and the position corresponding to the desired element, to obtain the position distance of each of the constituent elements; calculating the similarity between the color of each of the constituent elements and the color corresponding to the desired element to obtain the color difference of each of the constituent elements; determining whether the positional distance of each of the constituent elements is less than a preset distance, and determining whether the color difference of each of the constituent elements is less than a preset color difference value; If the position distance of each of the constituent elements is less than the preset distance, and the color difference of each of the constituent elements is less than the preset color difference value, then determine the position and color of each of the constituent elements, Match the position and color corresponding to the desired element. 5. The data processing method according to claim 2, wherein the key image information also includes a desired background description, and the multiple expected elements are matched one by one with a plurality of the constituent elements to obtain After matching the results, the method further includes: If the matching result is a complete match, background recognition is performed on the content recognition data to obtain an image background description of the initially generated image; determining a background similarity of the desired background description to the image background description; If the background similarity is greater than or equal to a preset background similarity, then determining that the content identification data matches the key image information; If the background similarity is smaller than the preset background similarity, it is determined that the content identification data does not match the key image information. 6. The data processing method according to claim 5, wherein the desired background description includes a desired background color and a desired background pattern, the image background description includes identifying a background color and identifying a background pattern, and the determining the The background similarity between the expected background description and the image background description includes: calculating the background color similarity between the expected background color and the recognized background color, and calculating the pattern similarity between the expected background pattern and the recognized background pattern; The background similarity is calculated based on the background color similarity and the pattern similarity. 7. The data processing method according to any one of claims 1-6, wherein said generating an image based on said desired content description and said desired image style to obtain an initially generated image comprises: Obtain the target image generation model obtained by performing deep learning on the generative adversarial network based on the model sample data set, each historical image sample in the model sample data set corresponds to a sample content description vector and a sample style vector, based on a plurality of the historical images The sample content description vector and sample style vector of the sample perform deep learning training on the generation confrontation network to obtain the target image generation model; performing semantic feature extraction on the desired content description to obtain an image description vector, and performing semantic feature extraction on the desired image style to obtain a desired image style vector; Inputting the desired image style vector and the image description vector into the target image generation model for image generation, and obtaining an image output by the target image generation model as the initial generated image. 8. A data processing device, characterized in that it comprises: An acquisition module, configured to acquire the user's image description information for the desired image, the image description information including the desired image style and desired content description; A generating module, configured to generate an image based on the desired content description and the desired image style, obtain an initially generated image, and perform semantic extraction on the desired content description, to obtain key image information corresponding to the desired content description; A recognition module, configured to perform image content recognition on the initially generated image to obtain content recognition data, and perform style recognition on the initially generated image to obtain an image recognition style; A determining module, configured to determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information; An output module, configured to output the initially generated image as the target retrieval corresponding to the expected image if the image recognition style is consistent with the desired image style, and the content recognition data matches the key image information image. 9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, characterized in that, when the processor executes the computer program, the computer program according to claim Steps of the data processing method described in any one of 1 to 7. 10. A computer-readable storage medium, the computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented A step of.

Images