CN114241244A

CN114241244A - Generative Adversarial Network Model Scheduling System and Method Based on Hand Drawing to Generate Images

Info

Publication number: CN114241244A
Application number: CN202111573065.1A
Authority: CN
Inventors: 董辰; 陈德程; 时梦然; 陈鲁蒙
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2021-12-21
Filing date: 2021-12-21
Publication date: 2022-03-25
Anticipated expiration: 2041-12-21
Also published as: CN114241244B

Abstract

The invention relates to a system and a method for generating confrontation network model scheduling based on hand-drawn image generation image, wherein the system comprises: the characteristic extraction module is used for extracting the characteristics of the hand-drawing graph to obtain image classification possibly corresponding to the hand-drawing graph; the model selection module is used for automatically selecting a corresponding model by combining the operation parameters of the terminal equipment and the image classification possibly corresponding to the hand-drawn picture; the model management module is used for managing a model space and a scheduling model; the generating and testing module is used for quickly testing the effect of the generated model and feeding back the effect in time; and the model judgment module is used for receiving a final model judgment instruction of a user and determining a final target model. The method and the device have the advantages that the image classification possibly corresponding to the hand drawing is obtained by dynamically extracting the hand drawing characteristics, the corresponding model is automatically selected for quick test and timely feedback, the target generation model is obtained by user judgment or automatic judgment and is scheduled and rendered, the generation of the corresponding image based on any kind of hand drawing is realized, and the purpose of drawing is realized.

Description

System and method for scheduling generation countermeasure network model based on hand-drawn image generation image

Technical Field

The invention belongs to the field of artificial intelligence, and particularly relates to a system and a method for generating confrontation network model scheduling based on a hand-drawn image generation image.

Background

Generation of a countermeasure network (GAN) is a deep learning model, and is one of the most promising methods for unsupervised learning in complex distribution in recent years. The model passes through (at least) two modules in the framework: the mutual game learning of the Generative Model (Generative Model) and the Discriminative Model (Discriminative Model) yields a reasonably good output. Image generation may be performed using a generative model therein. Generating an image based on a hand-drawn diagram is an application of generating an antagonistic network, and mainly relates to guiding the generation of an image of a generative model by using the hand-drawn diagram so that the generated image is matched with the hand-drawn diagram, namely the generated image is the image represented by the hand-drawn diagram.

At present, an algorithm model for generating an image based on a sketch can perform well when generating an image of a single classification, but when the classification is increased, the generation effect of the model is worse due to the increase of a loss function, and even the task of generating a corresponding image based on the sketch cannot be completed. Sketch-based drawings are diverse, and a single generation countermeasure network cannot simultaneously satisfy so many kinds of sketch-based image generation requirements.

Disclosure of Invention

In view of the above technical problems, the present invention provides a system and method for generating a model schedule of a countermeasure network based on a freehand drawing generated image. The method comprises the steps of obtaining image classification possibly corresponding to a sketch by dynamically extracting sketch features, automatically selecting a corresponding model for rapid test and timely feedback, obtaining a target generation model through user judgment or automatic judgment, scheduling and rendering, generating a corresponding image based on any kind of hand drawing, and achieving what is desired. And then, the system and the method of the invention dispatch the required generating model from the model space to realize that any kind of images can be generated based on the hand-drawn graph.

The technical scheme for solving the technical problems is as follows:

a system for generating a confrontational network model schedule based on a freehand sketch generated image, comprising:

the characteristic extraction module is used for extracting the characteristics of the hand-drawing graph to obtain image classification possibly corresponding to the hand-drawing graph;

the model selection module is used for automatically selecting a corresponding model by combining the operation parameters of the terminal equipment and the image classification possibly corresponding to the sketch;

the model management module is used for managing a model space and a scheduling model;

the generating and testing module is used for quickly testing the effect of the generated model and feeding back the effect in time;

and the model judgment module is used for receiving a final model judgment instruction of a user and determining a final target model.

Further, the system further comprises:

and the image generation module is used for rapidly rendering the finally obtained generation model, providing a service for generating an image based on the hand-drawn image to the user terminal in real time, and is responsible for caching the generation model.

The invention also provides a method for generating a confrontation network model scheduling based on the sketch generated image, which comprises the following steps:

carrying out feature extraction on the hand drawings to obtain possible target image classifications of the first N kinds of drawings;

adapting the operation condition of user terminal equipment, and comprehensively selecting a generation model with proper size and type and generating classification matching by combining the former N possible target image classifications;

testing the generated models corresponding to the previous N possible classifications at the server side and returning the results to the user side to be delivered to the user side for model judgment;

judging the classification with outstanding possibility as a target classification by a user or automatically;

and determining a target generation model by combining the target classification with the operation condition of the user terminal equipment.

Further, the method further comprises:

transmitting the target generation model to a user side for rendering, and providing a service for generating an image based on a hand-drawing in real time;

the generated model of the user is cached, and unnecessary resource expenses such as time waste and the like when the model is repeatedly used are avoided.

The invention has the beneficial effects that: when the hand-drawn picture is input into the system, the image classification which can be corresponding to the hand-drawn picture can be obtained by extracting the characteristics of the hand-drawn picture, the corresponding model is automatically selected to be quickly tested and fed back in time according to the running condition of the equipment, and the target generation model number is obtained by user judgment or automatic judgment, so that the generation model which is compatible with the user terminal running equipment, has excellent performance and is matched with the generation classification is obtained, the service quality of the generated image based on the sketch is ensured, the limitation of the generated image classification is broken, and the purpose of drawing can be realized on the premise that the model space is abundant enough.

Drawings

Fig. 1 is a block diagram of a system for generating a confrontation network model based on a freehand drawing generated image according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an operating principle of a generation countermeasure network model scheduling system for generating an image based on a freehand drawing according to an embodiment of the present invention;

fig. 3 is a flowchart of a method for scheduling a generation countermeasure network model based on a freehand drawing generated image according to an embodiment of the present invention.

Detailed Description

The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.

The embodiment of the invention provides a system for generating a confrontation network model scheduling based on a hand-drawn image generated image, which is suitable for a confrontation network model generating system based on the hand-drawn image generated image and provided with a plurality of generation models of various classifications, and as shown in figure 1, the system comprises:

the hand-drawing feature extraction module is used for extracting the hand-drawing features to obtain image classification possibly corresponding to the hand-drawing;

the model selection module is used for automatically selecting a corresponding model by combining the operation parameters of the terminal equipment and the image classification possibly corresponding to the hand-drawn picture;

Specifically, the feature extraction module is located at the user side, and extracts features of the hand drawing by using the convolutional neural network to obtain the possibility of classification that the hand drawing may correspond to, selects the top 10 classes according to the possibility, and delivers the class codes to the model selection module.

The model selection module is positioned at a user side, monitors the operation condition of user terminal equipment in real time, makes a generation model selection decision which is corresponding to classification, proper in size and excellent in performance by combining the operation parameters of the equipment and 10 classification codes from the model selection module or the unique classification codes from the model judgment module, and sends the number result of the selected model to the model management module.

And the model management module is positioned at the server end and is responsible for managing a model space and a scheduling model, wherein the model space is a set of all generation models related to the system, the model space comprises a plurality of classified models, each classification has models with different sizes and different network structures, and each model has a unique model number. The model management module receives the model number from the model selection module, directly calls the corresponding generated model from the model space, directly transmits the model to the image generation module at the user end if the received model number is unique, and loads the corresponding generated model to the generation test module if the received model number is not unique.

And the generation testing module is positioned at the server end and is responsible for rapidly testing the generation model to obtain a generated testing result image, and the generated testing result image is returned to the model judgment module positioned at the user end.

And the model judgment module is positioned at the user side, receives the generated test result image from the generation test module, presents the generated test result image to the user and judges the model. The model decision comprises two execution mechanisms, namely, a user selects and generates a test result image, so as to confirm the classification of a target and deliver a corresponding unique classification code to a model selection module; second, when the probability of a certain classification is far higher than that of other classifications, the model decision module will automatically decide the classification as the target classification and deliver the corresponding unique classification code to the model selection module.

The generation countermeasure network model scheduling system based on the hand-drawn image generated image provided by the embodiment of the invention can obtain the classification of the images possibly corresponding to the sketch by dynamically extracting the characteristics of the sketch, automatically select the corresponding model for quick test and timely feedback, obtain the target generation model by user judgment or automatic judgment, and realize the automatic scheduling of the corresponding generation model according to the hand-drawn image to perform the service based on the sketch generated image.

Optionally, in this embodiment, as shown in fig. 1, the system further includes:

and the image generation module is positioned at the user side and used for quickly rendering the finally obtained generation model, providing a service for generating the image based on the hand-drawing in real time for the user terminal and is responsible for caching the generation model, wherein the quantity of the caching model is determined according to the running condition of user terminal equipment.

The embodiment of the invention also provides a method for generating a confrontation network model scheduling based on the hand-drawn image generated image, as shown in fig. 3, the method comprises the following steps:

s1, acquiring the hand drawing drawn by the user in real time;

s2, extracting features of the hand drawing by using a convolutional neural network to obtain the top 10 possible classification codes;

s3, selecting 10 corresponding models by combining the operation parameters and the classification codes of the terminal equipment to obtain model numbers;

s4, calling 10 corresponding generation models to test at the server side according to the model numbers;

s5, returning the generated test result image to a user side to be submitted to a user for model judgment or automatically judging the target classification with outstanding possibility;

and S6, determining a unique object generation model by using the unique object classification code.

Claims

1. a system based on hand-drawn generation image generation adversarial network model scheduling, is characterized in that, comprises:

The feature extraction module is used to extract the hand drawing features to obtain the possible image classification corresponding to the hand drawing;

The model selection module is used to automatically select the corresponding model in combination with the operating parameters of the terminal equipment and the image classification that the sketch may correspond to;

Model management module for managing model space and scheduling models;

Generate a test module to quickly test the effect of the generated model and provide timely feedback;

The model judgment module is used to receive the user's final model judgment instruction and determine the final target model.

2. The system of claim 1, further comprising:

The image generation module is used to quickly render the final generated model, provide the user terminal with the service of generating images based on hand-drawn drawings in real time, and is responsible for caching the generated model.

3. A method for scheduling a confrontational generative network model based on a hand-drawn generation image, characterized in that, comprising:

Perform feature extraction on hand drawings to obtain the possible target image classification of the top N paintings;

Adapt to the operation of the user terminal equipment, combine the first N possible target image classifications, comprehensively select the appropriate size, appropriate type, and generate a classification matching generation model;

Test the generated models corresponding to the top N possible classifications on the server side, and return the results to the client side for model judgment on the client side;

User judgment or automatic judgment judges the classification with outstanding possibility as the target classification;

The target generation model is determined by the target classification combined with the operation of the user terminal equipment.

4. The method of claim 3, further comprising:

Numbering and unifying all generative models in the model space so that they can be quickly rendered by the specified generative test module and image generation module.

5. The method of claim 3, further comprising:

The target generation model is transmitted to the client for rendering, and the service of generating images based on hand-drawn drawings is provided in real time;

Cache the user's generated model to avoid wasting unnecessary time and other resource overhead when reusing the model.