CN114596412B - Method for generating virtual fitting 3D image - Google Patents
Method for generating virtual fitting 3D image Download PDFInfo
- Publication number
- CN114596412B CN114596412B CN202210460211.8A CN202210460211A CN114596412B CN 114596412 B CN114596412 B CN 114596412B CN 202210460211 A CN202210460211 A CN 202210460211A CN 114596412 B CN114596412 B CN 114596412B
- Authority
- CN
- China
- Prior art keywords
- user
- fitting
- model
- fitting state
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0641—Shopping interfaces
- G06Q30/0643—Graphical representation of items or shoppers
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
Embodiments disclosed herein provide a method of generating a virtual try-on 3D image. The user can acquire the virtual fitting 3D image representing the fitting effect of the user on the clothes without actually fitting the clothes, which is more convenient for the user, and the fusion effect of the human body model and the clothes model in the obtained virtual fitting 3D image is more vivid.
Description
Technical Field
Embodiments of the present disclosure relate to the field of information technology, and in particular, to a method for generating a virtual fitting 3D image.
Background
At present, in some application scenes, the demand exists for making fitting images of fitting clothing commodities for users.
Such an application scenario may be, for example, a user taking a virtual fitting in a shop online. If a user wants to know the upper body effect of the user on a certain piece of clothes, the user does not need to enter a fitting room to perform actual fitting, but performs virtual fitting by using the virtual image of the user, so that the user is more convenient.
Based on this, a relatively efficient solution for generating virtual fitting 3D images for the user is needed.
Disclosure of Invention
Various embodiments of the present description provide a method of generating a virtual try-on 3D image so as to be able to generate a more realistic virtual try-on 3D image.
The technical scheme provided by the embodiments of the specification is as follows:
according to a first aspect of various embodiments herein, there is provided a method of generating a virtual try-on 3D image, comprising:
acquiring a human body size parameter of a user; monitoring the simulation fitting operation of the user in the non-fitting state, and determining the posture parameters and expression parameters respectively specified by the user for a plurality of continuous different fitting states;
aiming at any fitting state, inputting the human body size parameter of the user, the posture parameter specified by the user for the fitting state and the expression parameter specified by the user for the fitting state into a parameterized human body model to obtain a 3D-mesh model, and taking the 3D-mesh model as a non-default human body model in the fitting state;
aiming at the 1 st fitting state, inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state, the posture parameter specified by the user for the 1 st fitting state and the default clothing model of the clothing to be fitted into a first mapping model trained in advance, and outputting a non-default clothing model in the 1 st fitting state; wherein the default clothing model and the non-default clothing model belong to a 3D-mesh model, and the first mapping model at least comprises a Deep Neural Network (DNN);
aiming at the ith fitting state, inputting the human body size parameter of the user, the posture parameter of the user in the ith-1 fitting state, the posture parameter specified by the user in the ith fitting state and the non-default clothing model in the ith-1 fitting state into the first mapping model and outputting the non-default clothing model in the ith fitting state; i =2, 3, …, N being the number of fitting states;
and aiming at any fitting state, fusing the non-default human body model in the fitting state with the non-default clothing model in the fitting state to obtain a corresponding frame of virtual fitting 3D image.
According to a second aspect of various embodiments herein, there is provided a method of generating a virtual try-on 3D image, comprising:
acquiring a human body size parameter of a user; monitoring the simulation fitting operation of the user in the non-fitting state, and determining the posture parameters and expression parameters which are specified by the user for the fitting state;
inputting the human body size parameter of the user, the posture parameter specified by the user for the fitting state and the expression parameter specified by the user for the fitting state into a parameterized human body model to obtain a 3D-mesh model, and taking the 3D-mesh model as a non-default human body model in the fitting state;
inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state, the posture parameter appointed by the user for the fitting state and the default clothing model of the clothing to be fitted into a first mapping model trained in advance, and outputting the non-default clothing model in the fitting state; wherein the default clothing model and the non-default clothing model belong to a 3D-mesh model, and the first mapping model at least comprises a Deep Neural Network (DNN);
and fusing the non-default human body model in the fitting state and the non-default clothing model in the fitting state to obtain a corresponding frame of virtual fitting 3D image.
According to a third aspect of various embodiments herein, there is provided a computing device comprising a memory, a processor; the memory is for storing computer instructions executable on the processor for implementing the method of the first aspect when the computer instructions are executed.
According to a fourth aspect of the various embodiments of the present description, a computer-readable storage medium is proposed, on which a computer program is stored, which when executed by a processor implements the method of the first aspect.
In the technical scheme, the user can perform simulated fitting operation (actual fitting of the clothes is not needed), and the posture parameters and the expression parameters respectively formulated by the user in a plurality of continuous different fitting states can be determined according to the simulated fitting operation of the user. In addition, the human body size parameters of the user are not changed in any posture and expression, and can be acquired.
The human body size parameter of the user, the posture parameter and the expression parameter of the user in a certain fitting state are input into the parameterized human body model, and the human body model in the fitting state can be output. For clarity of description, the body model in the non-fitting state is referred to as a default body model, and the body model in the fitting state (which may have a certain posture change or expression change relative to the default body model) is referred to as a non-default body model.
Considering that one frame of virtual fitting 3D image is formed by fusing a human body model and a clothing model, after obtaining non-default human body models in a plurality of fitting states in succession, it is further necessary to determine to fuse the clothing model with the corresponding non-default human body model in each fitting state. For clarity of description herein, a clothing model in an unlined state is referred to as a default clothing model, and a clothing model in a fitted state (which may be deformed somewhat relative to the default clothing model) is referred to as a non-default model.
Considering that the fusion effect of the non-default human body model and the non-default clothing model in the virtual fitting 3D image is as vivid as possible, the artificial intelligence AI capability is utilized to learn the relation rule between the non-fitting state and the first fitting state, and the relation rule between the previous fitting state and the next fitting state is trained by adopting the deep neural network DNN. And mapping the human body size parameter, the attitude parameter, the clothing model and the attitude parameter in the current fitting state to be the clothing model in the current fitting state by using the trained first mapping model.
The effect of fusing the non-default clothing model in each fitting state predicted by the first mapping model with the non-default human body model in the same fitting state is more vivid.
Certainly, the user may perform the fitting simulation operation only once in the non-fitting state to generate a fitting state, and obtain a frame of virtual fitting 3D image, and the related principles are similar and will not be described again.
Through the technical scheme, the user can acquire the virtual fitting 3D image representing the fitting effect of the user on the clothes without actually fitting the clothes, the user is more convenient, and the fusion effect of the human body model and the clothes model in the obtained virtual fitting 3D image is more vivid.
Drawings
Fig. 1 exemplarily provides a flow of a method of generating a virtual try-on 3D image.
Fig. 2 schematically provides a flow of another method of generating a virtual try-on 3D image.
Fig. 3 is a schematic diagram of a computer-readable storage medium provided by the present disclosure.
Fig. 4 is a schematic structural diagram of a computing device provided by the present disclosure.
In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described herein. In some other embodiments, the method may include more or fewer steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps for description in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.
Several concepts of the disclosure are presented herein.
Clothes decoration: the meaning of the apparel in the present disclosure is relatively broad, and may include not only the goods such as clothes, trousers, shoes, socks, etc., but also jewelry, hair accessories, hanging accessories, accessories (such as handbags), etc.
The human body model comprises: is a digitized 3D-mesh model and is generated by using a parameterized human model (parametric human model). The parameterized human body model is a human body capture model, and the input of the parameterized human body model is usually required to be the size parameter of the human body, the posture parameter of the human body and the expression parameter of the human body. The output of the parameterized human model is typically a 3D-mesh model. The parameterized mannequin has different model types for different genders (male, female, neutral).
The parameterized human body model can be an SMPL-X human body model, an MANO model, an SMPL + H model, an SMPL model and the like.
In some embodiments, a parameterized mannequin matching the gender of the user may be used. In other embodiments, a neutral parameterized mannequin may be used regardless of the user's gender.
Clothes model: is a digitized 3D-mesh model. The 3D modeling can be carried out on the clothes which are not tried on, and the default clothes model in the non-trying on state is obtained.
The mapping model is a model in the field of artificial intelligence and is used for mapping the characteristics of one meaning to the characteristics of the other meaning, and the mapping model can learn the relation rule between the characteristics of two different meanings through pre-training. In general, in the stage of training the mapping model, the input features and the output features to be mapped are used as samples, and model parameters of the mapping model are trained. In the application stage of the mapping model, the known input features are input into the mapping model for mapping to obtain the output features.
Those skilled in the art will readily appreciate that the so-called "models" of human body, apparel, etc., as used herein, are concepts that fall within the field of 3D modeling. The so-called "model" such as the mapping model and the parameterized human body model in this document is a concept belonging to the field of artificial intelligence. There is a substantial difference between the "model" concepts of the two domains.
Virtual fitting 3D image: the concept of one frame of 3D image is adopted, and it is easy to understand that a plurality of frames of virtual try-on 3D images can form a virtual try-on 3D video. The virtual try-on 2D image is a projection of the virtual try-on 3D image on a plane, and is a result of projecting the virtual try-on 3D image at a certain angle to the plane, different plane projection results (namely the virtual try-on 2D image) can be generated when a user rotates the angle of a combined model (namely the combination of a human body model and a clothing model of clothing goods) in the virtual try-on 3D image, and the projection results are calculated through rendering. The multi-frame virtual fitting 2D images can form a virtual fitting 2D video.
The technical scheme concept of the technical scheme is introduced as follows:
the user can simulate fitting operation (actual fitting of clothes is not needed), and the posture parameters and the expression parameters which are respectively formulated by the user under a plurality of continuous different fitting states can be determined according to the simulated fitting operation of the user. In addition, the human body size parameters of the user are not changed in any posture and expression, and can be acquired.
The human body size parameter of the user, the posture parameter and the expression parameter of the user in a certain fitting state are input into the parameterized human body model, and the human body model in the fitting state can be output. For clarity of description, the body model in the non-fitting state is referred to as a default body model, and the body model in the fitting state (which may have a certain posture change or expression change relative to the default body model) is referred to as a non-default body model.
Considering that one frame of virtual fitting 3D image is formed by fusing a human body model and a clothing model, after obtaining non-default human body models in a plurality of fitting states in succession, it is further necessary to determine to fuse the clothing model with the corresponding non-default human body model in each fitting state. For clarity of description herein, a clothing model in an unlined state is referred to as a default clothing model, and a clothing model in a fitted state (which may be deformed somewhat relative to the default clothing model) is referred to as a non-default model.
Considering that the fusion effect of the non-default human body model and the non-default clothing model in the virtual fitting 3D image is as vivid as possible, the artificial intelligence AI capability is utilized to learn the relation rule between the non-fitting state and the first fitting state, and the relation rule between the previous fitting state and the next fitting state is trained by adopting the deep neural network DNN. And mapping the human body size parameter, the attitude parameter, the clothing model and the attitude parameter in the current fitting state to be the clothing model in the current fitting state by using the trained first mapping model.
The effect of fusing the non-default clothing model in each fitting state predicted by the first mapping model with the non-default human body model in the same fitting state is more vivid.
Certainly, the user may perform the fitting simulation operation only once in the non-fitting state to generate a fitting state, and obtain a frame of virtual fitting 3D image, and the related principles are similar and will not be described again.
Through the technical scheme, the user can acquire the virtual fitting 3D image representing the fitting effect of the user on the clothes without actually fitting the clothes, the user is more convenient, and the fusion effect of the human body model and the clothes model in the obtained virtual fitting 3D image is more vivid.
In addition, when the technical scheme is implemented, the virtual fitting 3D image can be generated relatively quickly. A great deal of computing power and time are consumed in the process of training the first mapping model, so that after the trained first mapping model is obtained, output data can be obtained quickly according to input data acquired in real time.
The technical scheme for generating the virtual fitting 3D image can be particularly applied to a scene that a user performs virtual fitting in a shop on line. Under the scene, a user can get off a shop on line and is interested in a certain piece of clothes, and can use own portable equipment (such as a mobile phone and a tablet personal computer) or intelligent terminal equipment (such as large-screen interactive equipment) deployed on the off-line shop to realize the technical scheme without spending time and labor to enter a fitting room to perform actual fitting, so that one or more frames of virtual fitting 3D images are obtained, and the vivid effect of the person after trying on the clothes is known.
It should be noted that, the above technical solution for generating a virtual fitting 3D image can also be applied to other scenes where a user needs to fit a garment. For example, when a user uses a device (such as a mobile phone, a tablet computer, and a desktop computer) of the user to browse an interface of a clothing commodity on an e-commerce platform, the user may click a virtual fitting function button in the interface to trigger implementation of the above technical solution.
The technical solution is described in detail below with reference to the accompanying drawings.
Fig. 1 exemplarily provides a flow of a method of generating a virtual fitting 3D image, including:
s100: and acquiring the human body size parameters of the user.
S102: monitoring the simulation fitting operation of the user in the non-fitting state, and determining the posture parameters and the expression parameters respectively specified by the user for a plurality of continuous different fitting states.
S104: and aiming at any fitting state, inputting the human body size parameter of the user, the posture parameter specified by the user for the fitting state and the expression parameter specified by the user for the fitting state into a parameterized human body model to obtain a 3D-mesh model, and taking the 3D-mesh model as a non-default human body model in the fitting state.
S106: and aiming at the 1 st fitting state, inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state, the posture parameter specified by the user for the 1 st fitting state and the default clothing model of the clothing to be fitted into a first mapping model trained in advance, and outputting the non-default clothing model in the 1 st fitting state.
It should be noted that, in the method shown in fig. 1, the default clothing model and the non-default clothing model belong to a 3D-mesh model, and the first mapping model at least includes a deep neural network DNN, and the DNN may specifically be a multi-layer perceptron MLP.
S108: and aiming at the ith fitting state, inputting the human body size parameter of the user, the posture parameter of the user in the ith-1 fitting state, the posture parameter specified by the user for the ith fitting state and the non-default clothing model in the ith-1 fitting state into the first mapping model, and outputting the non-default clothing model in the ith fitting state.
Wherein i =2, 3, …, N is the number of the plurality of fitting states.
S110: and aiming at any fitting state, fusing the non-default human body model in the fitting state with the non-default clothing model in the fitting state to obtain a corresponding frame of virtual fitting 3D image.
In some embodiments, the anthropometric data input by the user may be acquired, the anthropometric data may be input into the trained second mapping model, and the human body size parameters of the user may be output. Wherein the second mapping model may comprise DNN. In practical application, a user can provide the self-thought accurate human body measurement data, and the human body measurement data is mapped to obtain the human body size parameters for generating the human body model.
In other embodiments, a whole-body photograph of a user may be acquired, the whole-body photograph is input to a third mapping model trained in advance, and a human body size parameter of the user is output; wherein the third mapping model comprises a convolutional neural network CNN. In practical application, a user can take a whole body picture to obtain the human body size parameters of the user without providing human body measurement data.
In some embodiments, a user may be subjected to a plurality of consecutive adjustment operations of the user's default mannequin in an un-fitted state; wherein each adjusting operation comprises posture adjustment and/or expression adjustment. The default human body model is a 3D-mesh model obtained by inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state and the expression parameter of the user in the non-fitting state into a parameterized human body module. The gesture parameter and the expression parameter corresponding to each adjustment operation can be determined for each adjustment operation, then the adjustment operation is defined as a fitting state, and the gesture parameter and the expression parameter corresponding to the adjustment operation are used as the gesture parameter and the expression parameter which are specified by the user for the fitting state.
In practical application, a user can interact with own equipment or an offline intelligent terminal (such as a touch screen mode, a remote controller mode, an entity knob mode and the like), and adjust a default human body model displayed by the equipment or the intelligent terminal, so that a signal for simulating fitting operation is given.
In the above embodiment, a whole body photo of a user may be acquired, the whole body photo is input to a fourth mapping model trained in advance, and a human body size parameter, a default posture parameter, and a default expression parameter of the user are output; wherein the fourth mapping model comprises CNN. The method also comprises the steps of obtaining human body measurement data input by a user, inputting the human body measurement data into a trained second mapping model, outputting human body size parameters of the user, wherein the second mapping model comprises DNN, and defining default posture parameters and default expression parameters according to preset definition rules.
Therefore, a relatively accurate default human body model can be obtained only according to one whole body picture of the user, and data measurement and estimation on the body type of the user are not needed, and manual 3D modeling is not needed.
In some embodiments, a plurality of continuous simulated fitting actions performed by the user in the non-fitting state may be photographed, each simulated fitting action may be defined as a fitting state, and a frame of simulated fitting image corresponding to each fitting state may be obtained. Each simulated fitting action comprises a simulated fitting posture and a simulated fitting expression. Then, inputting a frame of simulated fitting images corresponding to each fitting state into a fifth mapping model trained in advance, and outputting posture parameters and expression parameters corresponding to the frame of simulated fitting images as the posture parameters and expression parameters specified by the user for the fitting state; the fifth mapping model includes CNN. The posture parameters obtained in this way may include not only the information of the joint points of the human body but also the rotation information of the joint points.
In practical application, the user equipment or the offline intelligent terminal can utilize the camera to capture the real-time motion of the user, and the user can simulate the fitting motion, which is equivalent to providing a signal for simulating fitting operation.
In addition, the first mapping model described above may further include an encoder and a decoder. Wherein the encoder is used for carrying out normalized encoding on the data; the input of the encoder is the input of the first mapping model and the output of the encoder is the input of DNN. The decoder is used for carrying out inverse normalized decoding on the data; the input of the decoder is the output of the DNN, the output of the decoder is the output of the first mapping model.
In practical application, the difference between the numerical space of the relevant parameters (the human body size parameters and the posture parameters) of the human body and the numerical space of the clothing model is often large, so that after the numerical values of different numerical spaces are subjected to normalized coding, DNN is used for mapping to obtain a good mapping effect. After obtaining the normalized mapping result, the mapping result needs to be inversely normalized to the numerical space of the clothing model.
In addition, after one or more frames of virtual fitting 3D images are obtained, the virtual fitting 3D images corresponding to each fitting state may be arranged and combined from first to last according to the sequence of the fitting states to obtain a virtual fitting 3D video.
Or performing plane projection on a frame of virtual fitting 3D image corresponding to each fitting state to obtain a corresponding frame of virtual fitting 2D image. Furthermore, the virtual fitting 2D images of each frame can be arranged and combined from beginning to end according to the sequence of fitting states, so that the virtual fitting 2D video is obtained.
Fig. 2 exemplarily provides another method flow for generating a virtual fitting 3D image, which includes:
s200: and acquiring the human body size parameters of the user.
S202: monitoring the simulation fitting operation of the user in the non-fitting state, and determining the posture parameters and expression parameters which are specified by the user for the fitting state.
S204: and inputting the human body size parameter of the user, the posture parameter specified by the user for the fitting state and the expression parameter specified by the user for the fitting state into a parameterized human body model to obtain a 3D-mesh model, and taking the 3D-mesh model as a non-default human body model in the fitting state.
S206: inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state, the posture parameter appointed by the user for the fitting state and the default clothing model of the clothing to be fitted into a first mapping model trained in advance, and outputting the non-default clothing model in the fitting state.
Wherein the default clothing model and the non-default clothing model belong to a 3D-mesh model, and the first mapping model at least comprises a Deep Neural Network (DNN).
S208: and fusing the non-default human body model in the fitting state and the non-default clothing model in the fitting state to obtain a corresponding frame of virtual fitting 3D image.
For an understanding of the method shown in fig. 1, the method flow shown in fig. 2 can be referred to, and the related principles are not described in detail.
The present disclosure also provides a computer readable storage medium, as shown in fig. 3, on which medium 140 a computer program is stored, which when executed by a processor implements the method of an embodiment of the present disclosure.
The present disclosure also provides a computing device comprising a memory, a processor; the memory is used to store computer instructions executable on the processor for implementing the methods of the disclosed embodiments when the computer instructions are executed.
Fig. 4 is a schematic structural diagram of a computing device provided by the present disclosure, where the computing device 15 may include, but is not limited to: a processor 151, a memory 152, and a bus 153 that connects the various system components, including the memory 152 and the processor 151.
Wherein the memory 152 stores computer instructions executable by the processor 151 such that the processor 151 is capable of performing the methods of any of the embodiments of the present disclosure. The memory 152 may include a random access memory unit RAM1521, a cache memory unit 1522, and/or a read only memory unit ROM 1523. The memory 152 may further include: a program tool 1525 having a set of program modules 1524, the program modules 1524 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, one or more combinations of which may comprise an implementation of a network environment.
The bus 153 may include, for example, a data bus, an address bus, a control bus, and the like. The computing device 15 may also communicate with an external device 155 through the I/O interface 154, the external device 155 may be, for example, a keyboard, a bluetooth device, etc. The computing device 150 may also communicate with one or more networks, which may be, for example, local area networks, wide area networks, public networks, etc., through the network adapter 156. The network adapter 156 may also communicate with other modules of the computing device 15 via the bus 153, as shown.
Further, while the operations of the disclosed methods are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
While the spirit and principles of the present disclosure have been described with reference to several particular embodiments, it is to be understood that the present disclosure is not limited to the particular embodiments disclosed, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same one or more software and/or hardware implementations of the present description.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing describes several embodiments of the present specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The terminology used in the description of the various embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments herein. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in various embodiments of the present description to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the various embodiments herein. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the method embodiment, since it is substantially similar to the method embodiment, it is relatively simple to describe, and reference may be made to the partial description of the method embodiment for relevant points. The above-described method embodiments are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the functions of the modules may be implemented in one or more software and/or hardware when implementing the embodiments of the present specification. And part or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (11)
1. A method of generating a virtual try-on 3D image, comprising:
acquiring a human body size parameter of a user; monitoring the simulation fitting operation of the user in the non-fitting state, and determining the posture parameters and expression parameters respectively specified by the user for a plurality of continuous different fitting states;
aiming at any fitting state, inputting the human body size parameter of the user, the posture parameter specified by the user for the fitting state and the expression parameter specified by the user for the fitting state into a parameterized human body model to obtain a 3D-mesh model, and taking the 3D-mesh model as a non-default human body model in the fitting state;
aiming at the 1 st fitting state, inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state, the posture parameter specified by the user for the 1 st fitting state and the default clothing model of the clothing to be fitted into a first mapping model trained in advance, and outputting a non-default clothing model in the 1 st fitting state; wherein the default clothing model and the non-default clothing model belong to a 3D-mesh model, and the first mapping model at least comprises a Deep Neural Network (DNN);
aiming at the ith fitting state, inputting the human body size parameter of the user, the posture parameter of the user in the ith-1 fitting state, the posture parameter specified by the user in the ith fitting state and the non-default clothing model in the ith-1 fitting state into the first mapping model and outputting the non-default clothing model in the ith fitting state; i =2, 3, …, N being the number of said plurality of different fitting states;
and aiming at any fitting state, fusing the non-default human body model in the fitting state with the non-default clothing model in the fitting state to obtain a corresponding frame of virtual fitting 3D image.
2. The method of claim 1, wherein obtaining the body size parameter of the user comprises:
acquiring human body measurement data input by a user, inputting the human body measurement data into a trained second mapping model, and outputting human body size parameters of the user; the second mapping model comprises DNN;
or
Acquiring a whole body photo of a user, inputting the whole body photo into a pre-trained third mapping model, and outputting a human body size parameter of the user; wherein the third mapping model comprises a convolutional neural network CNN.
3. The method of claim 1, wherein monitoring the simulated fitting operation performed by the user in the non-fitting state to determine the user's successively specified pose parameters and expression parameters for a plurality of different respective orientations comprises:
continuously adjusting the default human body model of the user for multiple times when the user is in an undried state; wherein each adjustment operation comprises posture adjustment and/or expression adjustment, and the default human body model is a 3D-mesh model obtained by inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state and the expression parameter of the user in the non-fitting state into a parameterized human body module;
determining a posture parameter and an expression parameter corresponding to each adjustment operation;
defining the adjustment operation as a fitting state, and taking the posture parameter and the expression parameter corresponding to the adjustment operation as the posture parameter and the expression parameter specified by the user for the fitting state.
4. The method of claim 3, wherein the step of determining the body size parameter of the user, the posture parameter of the user in the non-fitting state, and the expression parameter of the user in the non-fitting state comprises:
acquiring a whole body photo of a user, inputting the whole body photo into a pre-trained fourth mapping model, and outputting a human body size parameter, a default posture parameter and a default expression parameter of the user; wherein the fourth mapping model comprises CNN;
or
Acquiring human body measurement data input by a user, inputting the human body measurement data into a trained second mapping model, outputting human body size parameters of the user, wherein the second mapping model comprises DNN, and defining default posture parameters and default expression parameters according to preset definition rules.
5. The method of claim 1, wherein the step of monitoring the simulated fitting operation performed by the user in the non-fitting state and determining the posture parameters and the expression parameters respectively specified by the user for a plurality of different fitting states comprises:
shooting a plurality of continuous simulated fitting actions of a user in an undried state, defining each simulated fitting action as a fitting state, and obtaining a frame of simulated fitting image corresponding to each fitting state; each simulated fitting action comprises a simulated fitting posture and a simulated fitting expression;
inputting a frame of simulated fitting images corresponding to each fitting state into a fifth pre-trained mapping model, and outputting posture parameters and expression parameters corresponding to the frame of simulated fitting images as the posture parameters and expression parameters specified by the user for the fitting state; the fifth mapping model includes CNN.
6. The method of claim 1, wherein the first mapping model further comprises an encoder and a decoder;
the encoder is used for carrying out normalized encoding on the data; the input of the encoder is the input of the first mapping model and the output of the encoder is the input of DNN;
the decoder is used for carrying out inverse normalized decoding on the data; the input of the decoder is the output of the DNN, the output of the decoder is the output of the first mapping model.
7. The method of claim 1, further comprising:
arranging and combining a frame of virtual fitting 3D image corresponding to each fitting state from first to second according to the sequence of the fitting states to obtain a virtual fitting 3D video;
and/or
Carrying out plane projection on a frame of virtual fitting 3D image corresponding to each fitting state to obtain a corresponding frame of virtual fitting 2D image; and arranging and combining the virtual fitting 2D images of each frame from first to second according to the sequence of fitting states to obtain the virtual fitting 2D video.
8. The method of any of claims 1-7, wherein the parameterized body model is a parameterized body model matched to a gender of the user, or a neutral parameterized body model.
9. A method of generating a virtual try-on 3D image, comprising:
acquiring a human body size parameter of a user; monitoring the simulation fitting operation of the user in the non-fitting state, and determining the posture parameters and expression parameters which are specified by the user for the fitting state;
inputting the human body size parameter of the user, the posture parameter specified by the user for the fitting state and the expression parameter specified by the user for the fitting state into a parameterized human body model to obtain a 3D-mesh model, and taking the 3D-mesh model as a non-default human body model in the fitting state;
inputting the human body size parameter of the user, the posture parameter of the user in the non-fitting state, the posture parameter appointed by the user for the fitting state and the default clothing model of the clothing to be fitted into a first mapping model trained in advance, and outputting the non-default clothing model in the fitting state; wherein the default clothing model and the non-default clothing model belong to a 3D-mesh model, and the first mapping model at least comprises a Deep Neural Network (DNN);
and fusing the non-default human body model in the fitting state and the non-default clothing model in the fitting state to obtain a corresponding frame of virtual fitting 3D image.
10. A computing device comprising a memory, a processor; the memory is for storing computer instructions executable on a processor for implementing the method of any one of claims 1 to 9 when the computer instructions are executed.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210460211.8A CN114596412B (en) | 2022-04-28 | 2022-04-28 | Method for generating virtual fitting 3D image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210460211.8A CN114596412B (en) | 2022-04-28 | 2022-04-28 | Method for generating virtual fitting 3D image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114596412A CN114596412A (en) | 2022-06-07 |
| CN114596412B true CN114596412B (en) | 2022-07-29 |
Family
ID=81812389
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210460211.8A Active CN114596412B (en) | 2022-04-28 | 2022-04-28 | Method for generating virtual fitting 3D image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114596412B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118229867A (en) * | 2022-12-20 | 2024-06-21 | 北京字跳网络技术有限公司 | Virtual reloading method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015181314A (en) * | 2015-07-16 | 2015-10-15 | ソニー株式会社 | Image processing system, image processing method and program |
| JP2018113060A (en) * | 2018-03-14 | 2018-07-19 | 株式会社東芝 | Virtual try-on apparatus, virtual try-on system, virtual try-on method and program |
| CN113610612A (en) * | 2020-09-27 | 2021-11-05 | 玉辉纺织服装科技发展(南通)有限公司 | 3D virtual fitting method, system and storage medium |
| WO2021227425A1 (en) * | 2020-05-12 | 2021-11-18 | 浙江大学 | Virtual clothing try-on method based on sample |
| CN114202630A (en) * | 2020-08-27 | 2022-03-18 | 北京陌陌信息技术有限公司 | Illumination matching virtual fitting method, device and storage medium |
| CN114202629A (en) * | 2020-08-27 | 2022-03-18 | 北京陌陌信息技术有限公司 | A method, system, device and storage medium for establishing a human body model |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108898460A (en) * | 2018-06-28 | 2018-11-27 | 深圳市有钱科技有限公司 | Virtual fit method and device |
| RU2019125602A (en) * | 2019-08-13 | 2021-02-15 | Общество С Ограниченной Ответственностью "Тексел" | COMPLEX SYSTEM AND METHOD FOR REMOTE SELECTION OF CLOTHES |
-
2022
- 2022-04-28 CN CN202210460211.8A patent/CN114596412B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015181314A (en) * | 2015-07-16 | 2015-10-15 | ソニー株式会社 | Image processing system, image processing method and program |
| JP2018113060A (en) * | 2018-03-14 | 2018-07-19 | 株式会社東芝 | Virtual try-on apparatus, virtual try-on system, virtual try-on method and program |
| WO2021227425A1 (en) * | 2020-05-12 | 2021-11-18 | 浙江大学 | Virtual clothing try-on method based on sample |
| CN114202630A (en) * | 2020-08-27 | 2022-03-18 | 北京陌陌信息技术有限公司 | Illumination matching virtual fitting method, device and storage medium |
| CN114202629A (en) * | 2020-08-27 | 2022-03-18 | 北京陌陌信息技术有限公司 | A method, system, device and storage medium for establishing a human body model |
| CN113610612A (en) * | 2020-09-27 | 2021-11-05 | 玉辉纺织服装科技发展(南通)有限公司 | 3D virtual fitting method, system and storage medium |
Non-Patent Citations (5)
| Title |
|---|
| Automatic 3D virtual fitting systembased on skeleton driving;Guangyuan Shi,et al.;《The Visual Computer》;20200525;第1075-1088页 * |
| CLO3D-Based 3D Virtual Fitting Technology of Down Jacket and Simulation Research on Dynamic Effect of Cloth;Shuxian Huang and Li Huang;《Wireless Communications and Mobile Computing》;20220218;第1-11页 * |
| 基于服装参数的虚拟服装试穿模拟;陈青青;《莆田学院学报》;20160430;第23卷(第2期);第54-58页 * |
| 基于深度学习的三维人体感知及着装图像生成;李佳蔓;《中国优秀博硕士学位论文全文数据库（硕士） 工程科技I辑》;20210615;B024-402 * |
| 生成对抗网络在虚拟试衣中的应用研究进展;张颖 等;《丝绸》;20211231;第58卷(第12期);第63-72页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114596412A (en) | 2022-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111787242B (en) | Method and apparatus for virtual fitting | |
| US11164381B2 (en) | Clothing model generation and display system | |
| US20180144237A1 (en) | System and method for body scanning and avatar creation | |
| CN109961507A (en) | A kind of Face image synthesis method, apparatus, equipment and storage medium | |
| CN118298127B (en) | Three-dimensional model reconstruction and image generation method, device, storage medium and program product | |
| CN111754303A (en) | Method and apparatus for virtual changing of clothing, device and medium | |
| JP2021174553A (en) | Deep learning platform virtual image generation method and system | |
| CN114596412B (en) | Method for generating virtual fitting 3D image | |
| CN114841783A (en) | Commodity information processing method and device, terminal device and storage medium | |
| CN114782661B (en) | Training method and device for lower body posture prediction model | |
| CN114445271B (en) | Method for generating virtual fitting 3D image | |
| CN115984455A (en) | Image processing method, device and equipment | |
| CN113283953A (en) | Virtual fitting method, device, equipment and storage medium | |
| CN116524088A (en) | Jewelry virtual try-on method, jewelry virtual try-on device, computer equipment and storage medium | |
| CN117332105A (en) | Method for providing commodity collocation information and electronic equipment | |
| CN115619478A (en) | Method for wearing model and clothes based on neural network | |
| CN116266408A (en) | Body type estimating method, body type estimating device, storage medium and electronic equipment | |
| CN114119861A (en) | Human reconstruction network training, human reconstruction, fitting method and related device | |
| CN114862997A (en) | Image rendering method and apparatus, medium, and computer device | |
| CN114779948A (en) | Method, device and equipment for controlling instant interaction of animation characters based on facial recognition | |
| CN114863005B (en) | A method, device, storage medium and equipment for rendering body special effects | |
| WO2024169854A1 (en) | Image rendering method and apparatus, and electronic device and storage medium | |
| WO2024169884A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| Ajit et al. | Furnitureal-Home Furniture Dècor Application using AR | |
| Pralhad et al. | Voice Controlled Augmented Reality For Real Estate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |