CN108765273B - Virtual cosmetic surgery method and device for photographing faces - Google Patents

Abstract

The application provides a virtual face-lifting method and a virtual face-lifting device for face photographing, wherein the method comprises the following steps: acquiring a current original two-dimensional face image of a user and depth information corresponding to the original two-dimensional face image; performing three-dimensional reconstruction according to the depth information and the original two-dimensional face image to obtain an original face three-dimensional model; inquiring face information registered in advance, and judging whether a user is registered; if the fact that the user is registered is known, face three-dimensional model shaping parameters corresponding to the user are obtained, key points on the original face three-dimensional model are adjusted according to the face three-dimensional model shaping parameters, and a target face three-dimensional model after virtual face-lifting is obtained; and mapping the virtual face-lifting target face three-dimensional model to a two-dimensional plane to obtain a target two-dimensional face image. Therefore, the registered user is beautified based on the human face three-dimensional model, the beautifying effect is optimized, and the satisfaction degree of the target user on the beautifying effect and the viscosity of the product are improved.

Description

Virtual cosmetic surgery method and device for photographing faces

technical field

The present application relates to the technical field of portrait processing, and in particular, to a virtual cosmetic surgery method and device for photographing a human face.

Background technique

With the popularization of terminal devices, more and more users are accustomed to using the terminal devices to take pictures. Therefore, the camera functions of the terminal devices are also becoming more and more diversified. For example, relevant camera applications provide users with beauty functions.

In the related art, beauty processing is performed based on a two-dimensional face image, but the processing effect is not good, and the processed image is not very realistic.

SUMMARY OF THE INVENTION

The present application aims to solve one of the technical problems in the related art at least to a certain extent.

In order to achieve the above purpose, an embodiment of the first aspect of the present application proposes a virtual face-lifting method for photographing a face, including: acquiring a current original two-dimensional face image of a user, and a depth corresponding to the original two-dimensional face image information; perform three-dimensional reconstruction according to the depth information and the original two-dimensional face image, and obtain the original three-dimensional face model; query the pre-registered face information to determine whether the user is registered; if it is known that the user has been registered , then obtain the face 3D model shaping parameters corresponding to the user, and adjust the key points on the original face 3D model according to the face 3D model shaping parameters to obtain the target face 3D model after virtual plastic surgery ; Map the three-dimensional model of the target face after the virtual plastic surgery to a two-dimensional plane to obtain the target two-dimensional face image.

In order to achieve the above purpose, a second aspect of the present application provides a virtual cosmetic surgery device for photographing a human face, including: an acquisition module for acquiring a current original two-dimensional face image of a user, and a depth information corresponding to the face image; a reconstruction module for performing 3D reconstruction according to the depth information and the original 2D face image to obtain a 3D model of the original face; a query module for querying pre-registered faces information, to determine whether the user is registered; the adjustment module is used to obtain the three-dimensional face model shaping parameters corresponding to the user when it is known that the user has been registered, and adjust the original three-dimensional face model shaping parameters according to the Adjusting key points on the three-dimensional face model to obtain a three-dimensional model of the target face after virtual plastic surgery; a mapping module for mapping the three-dimensional model of the target face after virtual plastic surgery to a two-dimensional plane to obtain a target two-dimensional face face image.

In order to achieve the above purpose, an embodiment of the third aspect of the present application proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, when the processor executes the computer program , to implement the virtual face-lifting method for photographing a human face as described in the embodiment of the first aspect.

In order to achieve the above purpose, a fourth aspect embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. A virtual cosmetic method for face photography.

To achieve the above purpose, an image processing circuit is provided in the fifth aspect of the present application. The image processing circuit includes: an image unit, a depth information unit and a processing unit;

The image unit is used to output the current original two-dimensional face image of the user;

the depth information unit for outputting depth information corresponding to the original two-dimensional face image;

The processing unit is electrically connected to the image unit and the depth information unit respectively, and is configured to perform three-dimensional reconstruction according to the depth information and the original two-dimensional face image, obtain the three-dimensional model of the original face, and query Pre-registered face information, determine whether the user is registered, if it is known that the user has been registered, then obtain the face 3D model shaping parameters corresponding to the user, according to the face 3D model shaping parameters for the original face. The key points on the three-dimensional face model are adjusted to obtain a three-dimensional model of the target face after virtual plastic surgery, and the three-dimensional model of the target face after virtual plastic surgery is mapped to a two-dimensional plane to obtain a two-dimensional face image of the target. The technical solution provided by this application at least includes the following beneficial effects:

Based on the three-dimensional face model, the registered users are beautified, the beautification effect is optimized, and the target user's satisfaction with the beautification effect and the stickiness with the product are improved.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic flowchart of a virtual face-lifting method for photographing a human face provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of a virtual plastic surgery method for photographing a human face provided by another embodiment of the present application;

3 is a schematic structural diagram of a depth image acquisition component provided by an embodiment of the present application;

FIG. 4(a) is a schematic technical flow diagram of a virtual cosmetic surgery method for photographing a human face provided by an embodiment of the present application;

FIG. 4(b) is a schematic technical flow diagram of a virtual cosmetic surgery method for photographing a human face provided by another embodiment of the present application;

5 is a schematic structural diagram of a virtual cosmetic device for photographing a human face according to an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a virtual cosmetic device for photographing a human face according to another embodiment of the present application;

FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application; and

8 is a schematic diagram of an image processing circuit in one embodiment;

FIG. 9 is a schematic diagram of an image processing circuit in one embodiment.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

Aiming at the technical problems in the prior art that beauty processing is performed based on two-dimensional face images, the processing effect is not good, and the processed images are not very realistic, in the embodiments of the present application, by acquiring two-dimensional face images, and The depth information corresponding to the face image, according to the depth information and the face image, perform 3D reconstruction to obtain a 3D face model, and perform beauty based on the 3D model of the face. Compared with the 2D beauty, the depth of the face is considered. It realizes the differentiated beautification of different parts of the face and improves the realism of beautification. For example, when beautifying the face based on the 3D model of the face, when the nose is dermabrasion, the depth information helps to distinguish clearly. Out of the nose and other parts, therefore, avoid mistakenly dermabrasion of other parts to cause blurred faces, etc.

The following describes the virtual face-lifting method and device for photographing a human face according to the embodiments of the present application with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a virtual cosmetic surgery method for photographing a human face provided by an embodiment of the present application.

The virtual face plastic surgery method of the embodiment of the present application can be applied to a computer device having a device for acquiring depth information and color information, wherein the device having the function of acquiring device for depth information and color information (two-dimensional information) can be a dual-camera system, etc., The computer device may be a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and/or display screens.

Step 101: Acquire a current original two-dimensional face image of the user and depth information corresponding to the original two-dimensional face image.

It should be noted that, according to different application scenarios, in the embodiments of the present application, the hardware devices for acquiring depth information and original two-dimensional face image information are different:

As a possible implementation manner, the hardware device for obtaining the original two-dimensional face information is a visible light RGB image sensor, and the original two-dimensional face can be obtained based on an RGB visible light image sensor in a computer device. Specifically, the visible light RGB image sensor may include a visible light camera, and the visible light camera may capture the visible light reflected by the imaging object for imaging, and obtain the original two-dimensional face corresponding to the imaging object.

As a possible implementation, the way to obtain depth information is to obtain through a structured light sensor. Specifically, as shown in Figure 2, the way to obtain the depth information corresponding to each face image includes the following steps:

Step 201, projecting structured light on the face of the current user.

Step 202, shooting a structured light image modulated by the current user's face.

Step 203: Demodulate phase information corresponding to each pixel of the structured light image to obtain depth information corresponding to the face image.

In this example, referring to FIG. 3 , when the computer device is a smart phone 1000 , the depth image acquisition component 12 includes a structured light projector 121 and a structured light camera 122 . Step 201 may be implemented by the structured light projector 121 , and steps 202 and 203 may be implemented by the structured light camera 122 .

That is to say, the structured light projector 121 can be used to project structured light to the face of the current user; the structured light camera 122 can be used to capture the structured light image modulated by the face of the current user, and the corresponding pixels of the demodulated structured light image. phase information to obtain depth information.

Specifically, after the structured light projector 121 projects a certain pattern of structured light onto the face of the current user, a structured light image modulated by the face of the current user is formed on the surface of the face of the current user. The structured light camera 122 captures the modulated structured light image, and then demodulates the structured light image to obtain depth information. Among them, the mode of structured light can be laser stripes, Gray codes, sinusoidal stripes, non-uniform speckle, and the like.

The structured light camera 122 can be further used to demodulate the phase information corresponding to each pixel in the structured light image, convert the phase information into depth information, and generate a depth image according to the depth information.

Specifically, compared with the unmodulated structured light, the phase information of the modulated structured light has changed, and the structured light presented in the structured light image is the structured light after distortion is generated, wherein the changed phase information It can represent the depth information of the object. Therefore, the structured light camera 122 first demodulates the phase information corresponding to each pixel in the structured light image, and then calculates the depth information according to the phase information.

Step 102: Perform three-dimensional reconstruction according to the depth information and the original two-dimensional face image to obtain a three-dimensional model of the original face.

Specifically, three-dimensional reconstruction is performed according to the depth information and the original two-dimensional face image, depth information and two-dimensional information of relevant points are given, and the original three-dimensional face model is obtained by reconstruction. The original three-dimensional face model is a three-dimensional model and can be fully restored. A face, relative to the two-dimensional face model, also includes information such as the three-dimensional angle of the facial features of the face.

According to different application scenarios, the ways to obtain the original face 3D model by performing 3D reconstruction according to the depth information and the face image include but are not limited to the following ways:

As a possible implementation, perform key point recognition on each two-dimensional sample face image to obtain positioning key points. For each face image, according to the depth information of the positioning key points and the positioning key points The distance on the face image, including the x-axis distance and the y-axis distance in the two-dimensional space, determines the relative position of the positioning key point in the three-dimensional space, and connects the adjacent positioning keys according to the relative position of the positioning key point in the three-dimensional space. point to generate a 3D model of the original sample face. Among them, the key points are the feature points on the face, which may include points on the corners of the eyes, the tip of the nose, and the corners of the mouth.

As another possible implementation, obtain original two-dimensional face images from multiple angles, screen out face images with higher definition as original data, locate feature points, and use the feature positioning results to roughly estimate the face angle , build a rough three-dimensional deformation model of the face according to the angle and contour of the face, adjust the face feature points to the same scale as the face three-dimensional deformation model through translation and zoom operations, and extract the corresponding face feature points. The coordinate information of the points forms a sparse face 3D deformation model.

Then, according to the rough estimated value of the face angle and the sparse face 3D deformation model, the particle swarm algorithm iterative face 3D reconstruction is performed to obtain the face 3D geometric model. After the face 3D geometric model is obtained, the texture posting method is used to The face texture information in the input two-dimensional image is mapped to the three-dimensional geometric model of the face, and the complete original three-dimensional model of the face is obtained.

In an embodiment of the present application, in order to improve the effect of beautifying the face, the original three-dimensional face model can also be constructed based on the beautified original two-dimensional face image. Therefore, the constructed original three-dimensional face model is more beautiful, ensuring that beauty of beauty.

Specifically, the user attribute characteristics of the user are extracted, wherein the user attribute characteristics may include gender, age, race, and skin color, wherein the user attribute characteristics may be obtained according to the personal information input by the user during registration, or may be The original two-dimensional face image is obtained by collecting the two-dimensional face image information during user registration, and the original two-dimensional face image is beautified according to the user's attribute characteristics to obtain the original two-dimensional face image after beautification. The way of beautifying the two-dimensional face image can be pre-established the corresponding relationship between user attribute characteristics and beauty parameters. For example, the beauty parameters of women are acne removal, skin resurfacing, and whitening, and the beauty parameters of men are acne removal, etc. , thus, after acquiring the user attribute features, query the corresponding relationship to obtain the corresponding beauty parameters, and perform beautification processing on the original two-dimensional face image according to the query beauty parameters.

Of course, the way to beautify the original two-dimensional face image can include brightness optimization, sharpness improvement, denoising, obstacle processing, etc., in addition to the above-mentioned beauty, to ensure that the original three-dimensional face model is more accurate.

Step 103: Query pre-registered face information to determine whether the user is registered.

It can be understood that, in this embodiment, the optimized beauty service is provided based on the registered users. On the one hand, the registered users can obtain the most beautiful beauty effect when taking pictures, especially when multiple people take pictures, and improve the registered users' beauty. Satisfaction, on the other hand, helps promote relevant products. In practical applications, in order to further improve the photographing experience of registered users, when identifying registered users, special symbolic symbols can be used to mark registered users, for example, using different colored face focus frames to highlight registered users Users, use different shaped focus boxes to highlight registered users.

In different application scenarios, querying pre-registered face information to determine whether a user is registered includes but is not limited to the following methods:

As a possible implementation, pre-acquire the facial features of registered users, such as special marking features such as birthmarks, the shape and position features of facial features such as nose and eyes, etc., and analyze the original two-dimensional face image, such as using image recognition. The technology extracts the user's facial features, queries the pre-registered facial database, and determines whether there are facial features. If there is, it is determined that the user has been registered; if not, it is determined that the user is not registered.

Step 104, if it is known that the user has been registered, obtain the face 3D model shaping parameters corresponding to the user, and adjust the key points on the original face 3D model according to the face 3D model shaping parameters, so as to obtain the target face after virtual plastic surgery. 3D model.

The three-dimensional face model shaping parameters include, but are not limited to, the adjustment positions and distances of the target key points adjusted in the three-dimensional face model.

Specifically, if it is known that the user is a registered user, in order to provide an optimized beauty service for the registered user, the three-dimensional face model shaping parameters corresponding to the user are obtained, and the original three-dimensional face model is reconstructed according to the three-dimensional face model shaping parameters. The key points on the model are adjusted to obtain the 3D model of the target face after virtual plastic surgery. It is understandable that the original three-dimensional face model is actually constructed by key points and a triangular network formed by the connection of key points. Therefore, when adjusting the key points of the original face three-dimensional The three-dimensional face model changes, thereby obtaining the target face model after virtual plastic surgery.

The method of shaping parameters of the three-dimensional face model corresponding to the user may be actively registered by the user, or may be automatically generated according to the analysis of the original three-dimensional face model of the user.

As a possible implementation, obtain 2D sample face images from multiple angles of the user, and depth information corresponding to each 2D sample face image, and perform 3D reconstruction according to the depth information and the 2D sample face image , obtain the 3D model of the original sample face, adjust the key points on the 3D face model of the original sample face to be plastic surgery, obtain the 3D face model of the target sample face after virtual plastic surgery, and compare the 3D face model of the original sample face with the target sample face 3D model, extracting the face 3D model shaping parameters corresponding to the user, for example, generating corresponding coordinate difference information according to the coordinate difference of key points corresponding to the same part.

In this embodiment, in order to make the adjustment of the three-dimensional face model more convenient, the key points of each plastic surgery part are displayed on the original sample face three-dimensional model, for example, the key points of each plastic surgery part are displayed in a highlighted manner , detect the shift operation performed by the user on the key points of the part to be plastic surgery, such as detecting the user's drag operation on the selected key point, etc., adjust the key points according to the shift operation, and adjust the key points according to the adjusted key points and other adjacent key points. Connect the points to obtain the 3D face model of the target sample after virtual plastic surgery.

In the actual execution process, the key points of the part to be plastic surgery on the original sample face 3D model can be received and adjusted based on different implementation methods. The examples are as follows:

First example:

In this example, in order to facilitate the user's operation, an adjustment control may be provided for the user to adjust the three-dimensional face model in real time through the user's operation on the control.

Specifically, in this embodiment, an adjustment control corresponding to the key point of each cosmetic part is generated, the touch operation performed by the user on the adjustment control corresponding to the key point of the part to be cosmetic surgery is detected, corresponding adjustment parameters are obtained, and according to the adjustment The parameters adjust the key points of the part to be plastic surgery on the original sample face 3D model, and obtain the target sample face 3D model after virtual plastic surgery. Gap gets cosmetic parameters. Among them, the adjustment parameters include the moving direction and moving distance of the key points.

In this embodiment, plastic surgery suggestion information can also be provided to the user, for example, plastic surgery suggestions such as "enriching lips, filling an Apple computer", etc., where the plastic surgery suggestion information can be in the form of text, voice, etc. If the user confirms the plastic surgery suggestion information , determine the key points and adjustment parameters of the part to be plastic surgery according to the plastic surgery suggestion information. For example, if the user confirms the above plastic surgery suggestion, the determined cosmetic parameters are to adjust the depth value of the mouth and the depth value of the cheek, etc., among which, the size of the change of the depth value can be It is determined according to the depth value of the corresponding part of the user's original sample face 3D model. In order to ensure a natural adjustment effect, the difference between the adjusted depth value and the initial depth value is within a certain range. The key points of the part to be plastic surgery on the model are adjusted to obtain a three-dimensional face model of the target sample after virtual plastic surgery.

Among them, in order to further improve the beauty of the cosmetic effect, before adjusting the key points of the original face 3D model to be cosmetic, the skin texture map covering the surface of the original face 3D model can also be beautified. 3D model of the original human face.

It can be understood that when there are acne in the face image, the color of the part corresponding to the acne in the skin texture image can be red, or, when there are freckles in the face image, the color of the part corresponding to the freckles in the skin texture image can be red. The color may be brown or black, or, when there are black moles in the face image, the color of the part corresponding to the black mole in the skin texture map may be black.

Therefore, it can be determined whether there is an abnormal range according to the color of the skin texture map of the original face three-dimensional model. The relative positional relationship of the points in the three-dimensional space and the color information of the abnormal range are used to beautify the abnormal range by using the corresponding beautification strategy.

In general, acne protrudes from the skin surface, black moles can also protrude from the skin surface, and freckles do not protrude from the skin surface. Therefore, in the embodiment of the present application, the center point and the edge point of the abnormal range can be determined The height difference between them is used to determine the abnormality type to which the abnormality range belongs. For example, the abnormality type can be raised or not raised. After the abnormality type is determined, the corresponding beautification strategy can be determined according to the abnormality type and color information, and then according to the matching skin color corresponding to the abnormality range, the abnormality range is subjected to microdermabrasion using the filtering range and filter strength indicated by the beautification strategy.

For example, when the abnormal type is raised and the color information is red, at this time, the abnormal range can be acne, and the degree of microdermabrasion corresponding to acne is relatively strong. When the abnormal type is not raised, the color is cyan. At this time, the abnormal range can be a tattoo, and the degree of microdermabrasion corresponding to the tattoo is relatively weak.

Alternatively, the skin color within the abnormal range can also be filled according to the matching skin color corresponding to the abnormal range.

For example, when the abnormal type is bulge and the color information is red, at this time, the abnormal range can be acne, and the beautification strategy for acne removal can be: microdermabrasion on the acne, and according to the surrounding area of the acne The normal skin color is marked as matching skin color in the examples of this application, and fills the skin color within the abnormal range corresponding to acne, or, when the abnormal type is not raised and the color is brown, at this time, the abnormal range can be freckles , then the beautification strategy for freckle removal may be: according to the normal skin color near the freckles, which is recorded as matching skin color in the embodiment of the present application, and filling the skin color within the abnormal range corresponding to the freckles.

In this application, since the depth information of the enclosed area obtained by taking each key point as a vertex in the model of the original three-dimensional face model is consistent, when the skin texture map covering the surface of the three-dimensional face model is beautified, it can be divided into By beautifying each enclosed area, the reliability of the pixel value in the beautified enclosed area can be increased, and the beautification effect can be improved.

As another possible implementation manner of the embodiment of the present application, a beautification strategy corresponding to a partial human face may be preset, wherein the partial human face may include facial parts such as nose, lips, eyes, and cheeks. For example, for the nose, the corresponding beautification strategy can be to brighten the nose tip and shadow the nose to increase the three-dimensional effect of the nose, or, for the cheek, the corresponding beautification strategy can be adding blush and / or dermabrasion treatment.

Therefore, in the embodiment of the present application, the partial face can be identified from the skin texture map according to the color information and the relative position in the original three-dimensional model of the face, and then the partial face can be processed according to the beautification strategy corresponding to the partial face. beautify.

Optionally, when the partial human face is the eyebrows, the partial human face may be subjected to microdermabrasion according to the filtering strength indicated by the beautification strategy corresponding to the eyebrows.

When the partial face is a cheek, the partial face can be subjected to microdermabrasion according to the filtering strength indicated by the beautification strategy corresponding to the cheek. It should be noted that, in order to make the beautification effect more natural and the beautification effect more prominent, the filtering strength indicated by the beautification strategy corresponding to the cheeks may be greater than the filtering strength indicated by the beautification strategy corresponding to the eyebrows.

When the partial face belongs to the nose, the shadow of the partial face can be increased according to the shadow intensity indicated by the beautification strategy corresponding to the nose.

In the present application, beautification processing is performed on the partial face based on its relative position in the original face three-dimensional model, which can make the beautified skin texture map more natural and the beautification effect more prominent. In addition, targeted beautification processing can be performed on a local face, thereby improving the imaging effect and enhancing the user's photographing experience.

Of course, in practical applications, if it is known that the user is not registered, the user can also be provided with a better beauty service.

In this embodiment, if it is known that the user is not registered, the user attribute characteristics of the user are extracted, wherein the user attribute characteristics may include gender, age, race, and skin color, for example, the user's hairstyle, jewelry, Whether there is makeup, etc. to determine the user attribute characteristics, and then obtain the preset standard face 3D model shaping parameters corresponding to the user attribute characteristics, and perform key points on the original face 3D model according to the standard face 3D model shaping parameters. After adjustment, the 3D model of the target face after virtual plastic surgery is obtained.

Step 105: Map the three-dimensional model of the target face after virtual plastic surgery to a two-dimensional plane to obtain a target two-dimensional face image.

Specifically, after adjusting the key points of the part to be plastic surgery on the original three-dimensional face model to obtain the three-dimensional model of the target face after virtual plastic surgery, the three-dimensional model of the target face after the virtual plastic surgery can be mapped to a two-dimensional plane to obtain The target face image is processed, and the target two-dimensional face image is beautified.

In the present application, since the skin texture map is three-dimensional, beautifying the skin texture map can make the beautified skin texture map more natural, so that the target face three-dimensional model generated after virtual plastic surgery according to the beautified three-dimensional face model Map to a two-dimensional plane to obtain a beautified target two-dimensional face image, and perform beautification processing on the target two-dimensional face image, which can make the beautified target two-dimensional face image more realistic and the beautification effect more prominent. The user provides a display of the beauty effect after cosmetic surgery to further enhance the user's cosmetic surgery experience.

In order to make those skilled in the art clearer about the process of the virtual face-lifting method for taking pictures of human faces, an example is given below in conjunction with its application in a specific scene, and the description is as follows:

In this example, the calibration refers to calibrating the camera to determine the key points corresponding to the key points in the face image in the three-dimensional space.

In the registration stage, as shown in Figure 4(a), you can preview and scan the face through the camera module to obtain 2D sample face images from multiple angles of the user. For example, collect nearly 20 2D sample faces from different angles. The image and depth map are used for subsequent 3D face reconstruction, in which the missing angle and scanning progress can be prompted during scanning, as well as the depth information corresponding to each 2D sample face image. According to the depth information and the 2D sample face image Perform 3D reconstruction to obtain the 3D model of the original sample face.

Perform facial features analysis on the 3D face model, such as face shape, nose width, nose height, eye size, lip thickness, etc., and give plastic surgery advice. If the user confirms the cosmetic advice information, determine the key points and adjustments of the part to be cosmetic surgery based on the cosmetic advice information. parameters, and according to the adjustment parameters, adjust the key points of the part to be plastic surgery on the original sample face three-dimensional model, and obtain the target sample face three-dimensional model after virtual plastic surgery.

Further, as shown in Fig. 4(b), in the identification stage, the current original two-dimensional face image of the user and the depth information corresponding to the original two-dimensional face image are obtained, according to the depth information and the original two-dimensional face image Perform three-dimensional reconstruction of the image, obtain the original three-dimensional model of the face, query the pre-registered face information, and determine whether the user is registered. The plastic parameters adjust the key points on the original face 3D model to obtain the target face 3D model after virtual plastic surgery, and map the target face 3D model after virtual plastic surgery to the 2D plane to obtain the target 2D face image.

To sum up, the virtual face-lifting method for photographing a human face according to the embodiment of the present application obtains the current original two-dimensional face image of the user and the depth information corresponding to the original two-dimensional face image. Perform 3D reconstruction of the face image, obtain the original 3D model of the face, query the pre-registered face information, and determine whether the user is registered. The model shaping parameters adjust the key points on the original face 3D model to obtain the target face 3D model after virtual facelift, map the target face 3D model after virtual facelift to the 2D plane, and obtain the target 2D face image . As a result, the registered users are beautified based on the three-dimensional face model, the beautification effect is optimized, and the target user's satisfaction with the beautification effect and the stickiness with the product are improved.

In order to realize the above-mentioned embodiment, the present application also proposes a virtual cosmetic surgery device for taking a photo of a human face. FIG. 5 is a schematic structural diagram of a virtual cosmetic surgery device for taking a photo of a human face according to an embodiment of the present application. As shown in FIG. 5 , the virtual cosmetic surgery device for photographing a human face includes an acquisition module 10 , a reconstruction module 20 , a query module 30 , an adjustment module 40 and a mapping module 50 .

Wherein, the obtaining module 10 is configured to obtain the current original two-dimensional face image of the user and depth information corresponding to the original two-dimensional face image.

The reconstruction module 20 is configured to perform three-dimensional reconstruction according to the depth information and the original two-dimensional face image to obtain a three-dimensional model of the original face.

The query module 30 is configured to query the pre-registered face information to determine whether the user is registered.

In an embodiment of the present application, as shown in FIG. 6 , the query module 30 includes an extraction unit 31 and a determination unit 32 .

The extraction unit 31 is configured to analyze the original two-dimensional face image and extract the facial features of the user.

The determining unit 32 is configured to query a pre-registered face database to determine whether the facial feature exists, if so, determine that the user has been registered, and if not, determine that the user is not registered.

The adjustment module 40 is configured to obtain the three-dimensional face model shaping parameters corresponding to the user when it is known that the user has been registered, and adjust the key points on the original three-dimensional face model according to the three-dimensional face model shaping parameters. Make adjustments to obtain a 3D model of the target face after virtual plastic surgery.

The mapping module 50 is configured to map the three-dimensional model of the target face after virtual plastic surgery to a two-dimensional plane to obtain a two-dimensional face image of the target.

It should be noted that, the foregoing explanations of the embodiment of the virtual cosmetic surgery method for photographing a human face are also applicable to the virtual cosmetic surgery device for photographing a human face in this embodiment, which will not be repeated here.

To sum up, the virtual cosmetic surgery device for photographing a human face according to the embodiment of the present application acquires the current original two-dimensional human face image of the user and the depth information corresponding to the original two-dimensional human face image. Perform 3D reconstruction of the face image, obtain the original 3D model of the face, query the pre-registered face information, and determine whether the user is registered. The model shaping parameters adjust the key points on the original face 3D model to obtain the target face 3D model after virtual facelift, map the target face 3D model after virtual facelift to the 2D plane, and obtain the target 2D face image . As a result, the registered users are beautified based on the three-dimensional face model, the beautification effect is optimized, and the target user's satisfaction with the beautification effect and the stickiness with the product are improved.

In order to realize the above-mentioned embodiments, the present application also proposes a computer-readable storage medium on which a computer program is stored, and when the program is executed by the processor of the mobile terminal, the virtual face-lifting for taking photos of the human face as described in the foregoing embodiments is realized. method.

In order to realize the above embodiments, the present application also proposes an electronic device.

FIG. 7 is a schematic diagram of the internal structure of the electronic device 200 in one embodiment. The electronic device 200 includes a processor 220 , a memory 230 , a display 240 and an input device 250 connected through a system bus 210 . The memory 230 of the electronic device 200 stores an operating system and computer-readable instructions. The computer-readable instructions can be executed by the processor 220 to implement the face beautification method of the embodiment of the present application. The processor 220 is used to provide computing and control capabilities to support the operation of the entire electronic device 200 . The display 240 of the electronic device 200 may be a liquid crystal display screen or an electronic ink display screen, etc., and the input device 250 may be a touch layer covered on the display 240, or a button, a trackball or a touchpad provided on the housing of the electronic device 200, It can also be an external keyboard, trackpad or mouse, etc. The electronic device 200 may be a mobile phone, a tablet computer, a notebook computer, a personal digital assistant, or a wearable device (eg, a smart bracelet, a smart watch, a smart helmet, and smart glasses).

Those skilled in the art can understand that the structure shown in FIG. 7 is only a schematic diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device 200 to which the solution of the present application is applied. The specific electronic device 200 may include more or fewer components than shown, or combine certain components, or have a different arrangement of components.

In order to realize the above embodiments, the present invention also provides an image processing circuit, the image processing circuit includes an image unit 310 , a depth information unit 320 and a processing unit 330 . in,

The image unit 310 is configured to output the current original two-dimensional face image of the user.

The depth information unit 320 is configured to output depth information corresponding to the original two-dimensional face image.

The processing unit 330 is electrically connected to the image unit and the depth information unit respectively, and is used for performing three-dimensional reconstruction according to the depth information and the original two-dimensional face image, obtaining the three-dimensional model of the original face, querying the pre-registered face information, and judging the user Whether to register or not, if it is known that the user has registered, obtain the face 3D model shaping parameters corresponding to the user, adjust the key points on the original face 3D model according to the face 3D model shaping parameters, and obtain the target face after virtual plastic surgery The three-dimensional model is to map the three-dimensional model of the target face after virtual plastic surgery to a two-dimensional plane to obtain a two-dimensional face image of the target.

In this embodiment of the present application, the image unit 310 may specifically include: an electrically connected image sensor 311 and an image signal processing (Image Signal Processing, ISP for short) processor 312 . in,

The image sensor 311 is used to output raw image data.

The ISP processor 312 is configured to output the original two-dimensional face image according to the original image data.

In this embodiment of the present application, the raw image data captured by the image sensor 311 is first processed by the ISP processor 312 , and the ISP processor 312 analyzes the raw image data to capture image statistics that can be used to determine one or more control parameters of the image sensor 311 Information, including face images in YUV format or RGB format. The image sensor 311 may include a color filter array (such as a Bayer filter) and a corresponding photosensitive unit. The image sensor 311 may acquire the light intensity and wavelength information captured by each photosensitive unit, and provide information that can be processed by the ISP processor 312. A set of raw image data. After the ISP processor 312 processes the original image data, a face image in YUV format or RGB format is obtained, and sent to the processing unit 330 .

When processing the original image data, the ISP processor 312 can process the original image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 312 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

As a possible implementation manner, the depth information unit 320 includes an electrically connected structured light sensor 321 and a depth map generation chip 322 . in,

The structured light sensor 321 is used to generate an infrared speckle pattern.

The depth map generation chip 322 is used for outputting depth information corresponding to the original two-dimensional face image according to the infrared speckle map.

In the embodiment of the present application, the structured light sensor 321 projects speckle structured light to the subject, acquires the structured light reflected by the subject, and images the infrared speckle image according to the reflected structured light. The structured light sensor 321 sends the infrared speckle image to the depth map generation chip 322, so that the depth map generation chip 322 determines the morphological change of the structured light according to the infrared speckle image, and then determines the depth of the subject based on this to obtain the depth map (Depth Map), the depth map indicates the depth of each pixel in the infrared speckle map. The depth map generation chip 322 sends the depth map to the processing unit 330 .

As a possible implementation manner, the processing unit 330 includes: a CPU 331 and a GPU (Graphics Processing Unit, graphics processor) 332 that are electrically connected. in,

The CPU 331 is configured to align the face image and the depth map according to the calibration data, and output a three-dimensional face model according to the aligned face image and the depth map.

The GPU 332 is configured to acquire the three-dimensional face model shaping parameters corresponding to the user if it is known that the user has been registered, and adjust the key points on the original three-dimensional face model according to the three-dimensional face model shaping parameters to obtain a three-dimensional model of the target face after virtual plastic surgery, and map the three-dimensional model of the target face after virtual plastic surgery to a two-dimensional plane to obtain a two-dimensional face image of the target.

In the embodiment of the present application, the CPU 331 obtains the face image from the ISP processor 312, and obtains the depth map from the depth map generation chip 322. Combined with the pre-obtained calibration data, the face image and the depth map can be aligned to determine the person The depth information corresponding to each pixel in the face image. Further, the CPU 331 performs three-dimensional reconstruction according to the depth information and the face image to obtain a three-dimensional face model.

The CPU 331 sends the three-dimensional face model to the GPU 332, so that the GPU 332 executes the virtual facelifting method for photographing a face as described in the foregoing embodiments according to the three-dimensional face model, and obtains a target two-dimensional face image.

Further, the image processing circuit may further include: a first display unit 341 .

The first display unit 341, electrically connected to the processing unit 330, is used for displaying adjustment controls corresponding to key points of the part to be plastic surgery.

Further, the image processing circuit may further include: a second display unit 342 .

The second display unit 342, electrically connected to the processing unit 340, is used for displaying the three-dimensional face model of the target sample after virtual plastic surgery.

Optionally, the image processing circuit may further include: an encoder 350 and a memory 360 .

In this embodiment of the present application, the beautified face image processed by the GPU 332 may also be encoded by the encoder 350 and stored in the memory 360, where the encoder 350 may be implemented by a coprocessor.

In one embodiment, the memory 360 may be multiple or divided into multiple storage spaces, and the image data processed by the GPU 312 may be stored in a dedicated memory or dedicated storage space, and may include DMA (Direct Memory Access). access) feature. Memory 360 may be configured to implement one or more frame buffers.

The above process will be described in detail below with reference to FIG. 9 .

It should be noted that FIG. 9 is a schematic diagram of an image processing circuit as a possible implementation manner. For the convenience of description, only various aspects related to the embodiments of the present application are shown.

As shown in FIG. 9, raw image data captured by image sensor 311 is first processed by ISP processor 312, which analyzes the raw image data to capture image statistics that can be used to determine one or more control parameters of image sensor 311 Information, including face images in YUV format or RGB format. The image sensor 311 may include a color filter array (such as a Bayer filter) and a corresponding photosensitive unit. The image sensor 311 may acquire the light intensity and wavelength information captured by each photosensitive unit, and provide information that can be processed by the ISP processor 312. A set of raw image data. The ISP processor 312 processes the original image data to obtain a face image in YUV format or RGB format, and sends it to the CPU 331 .

When processing the original image data, the ISP processor 312 can process the original image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 312 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

As shown in FIG. 9 , the structured light sensor 321 projects speckle structured light to the subject, acquires the structured light reflected by the subject, and forms an infrared speckle image according to the reflected structured light. The structured light sensor 321 sends the infrared speckle image to the depth map generation chip 322, so that the depth map generation chip 322 determines the morphological change of the structured light according to the infrared speckle image, and then determines the depth of the subject based on this to obtain the depth map (Depth Map), the depth map indicates the depth of each pixel in the infrared speckle map. The depth map generation chip 322 sends the depth map to the CPU 331 .

The CPU 331 obtains the face image from the ISP processor 312, obtains the depth map from the depth map generation chip 322, and combines the pre-obtained calibration data to align the face image with the depth map, thereby determining each pixel in the face image. corresponding depth information. Further, the CPU 331 performs three-dimensional reconstruction according to the depth information and the face image to obtain a three-dimensional face model.

The CPU 331 sends the three-dimensional face model to the GPU 332, so that the GPU 332 executes the method described in the foregoing embodiments according to the three-dimensional face model, realizes virtual face-lifting, and obtains a face image after virtual face-lifting. The face image after virtual plastic surgery that is processed by the GPU 332 can be displayed by the display 340 (including the above-mentioned first display unit 341 and the second display unit 351), and/or stored in the memory 360 after being encoded by the encoder 350, wherein, The encoder 350 may be implemented by a coprocessor.

In one embodiment, the memory 360 may be multiple or divided into multiple storage spaces, and the image data processed by the GPU 332 may be stored in a dedicated memory or dedicated storage space, and may include DMA (Direct Memory Access). access) feature. Memory 360 may be configured to implement one or more frame buffers.

For example, the following are the steps of implementing the control method using the processor 220 in FIG. 9 or using the image processing circuit (specifically CPU331 and GPU332) in FIG. 9 :

The CPU331 obtains a two-dimensional face image and the depth information corresponding to the face image; the CPU 331 performs three-dimensional reconstruction according to the depth information and the face image to obtain a three-dimensional face model; Corresponding face 3D model shaping parameters, adjust the key points on the original face 3D model according to the face 3D model shaping parameters to obtain the target face 3D model after virtual plastic surgery; GPU332 The 3D model of the target face is mapped to a 2D plane to obtain a 2D face image of the target.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program is stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (19)

1. A virtual face-lifting method for face photographing is characterized by comprising the following steps:

acquiring original two-dimensional face images of a user at multiple current angles, projecting non-uniform speckle structured light to the face of the user, shooting a structured light image modulated by the face of the user, and demodulating phase information corresponding to each pixel of the structured light image to obtain depth information corresponding to the original two-dimensional face image;

performing three-dimensional reconstruction according to the depth information and the original two-dimensional face image to obtain an original face three-dimensional model;

inquiring face information registered in advance, and judging whether the user is registered;

if the fact that the user is registered is known, inquiring registration information to obtain a face three-dimensional model shaping parameter corresponding to the user, adjusting key points on the original face three-dimensional model according to the face three-dimensional model shaping parameter to obtain a target face three-dimensional model after virtual face shaping, wherein before the inquiring registration information obtains the face three-dimensional model shaping parameter corresponding to the user, the method comprises the following steps:

acquiring two-dimensional sample face images of the user at multiple angles and depth information corresponding to each two-dimensional sample face image, performing three-dimensional reconstruction according to the depth information and the two-dimensional sample face images to acquire an original sample face three-dimensional model, adjusting key points of a part to be trimmed on the original sample face three-dimensional model to obtain a target sample face three-dimensional model after virtual trimming, comparing the original sample face three-dimensional model with the target sample face three-dimensional model, extracting face three-dimensional model trimming parameters corresponding to the user, and storing the face three-dimensional model trimming parameters in the registration information;

if the user is not registered, acquiring attribute features of the user, searching standard face three-dimensional model shaping parameters corresponding to the attribute features according to the attribute features, and adjusting key points on the original face three-dimensional model according to the standard face three-dimensional model shaping parameters to obtain a target face three-dimensional model after virtual face-lifting;

and mapping the virtual face-lifting target face three-dimensional model to a two-dimensional plane to obtain a target two-dimensional face image.

2. The method of claim 1, wherein the querying pre-registered face information and determining whether the user is registered comprises:

analyzing the original two-dimensional face image and extracting facial features of the user;

inquiring a face database registered in advance, judging whether the face features exist, and if so, determining that the user is registered; and if not, determining that the user is not registered.

3. The method of claim 1, wherein the attribute characteristics of the user comprise:

gender, age, race, and skin color.

4. The method according to claim 1, before the obtaining an original face three-dimensional model according to the depth information and the original two-dimensional face image by three-dimensional reconstruction, further comprising:

extracting attribute features of the user;

and beautifying the original two-dimensional face image according to the attribute characteristics to obtain a beautified original two-dimensional face image.

5. The method of claim 1, after said obtaining the original human face three-dimensional model, further comprising:

and beautifying the skin texture map covered on the surface of the original human face three-dimensional model to obtain the beautified original human face three-dimensional model.

6. The method of claim 1, wherein the performing three-dimensional reconstruction based on the depth information and the two-dimensional sample face image to obtain an original sample face three-dimensional model comprises:

performing key point identification on each two-dimensional sample face image to obtain positioning key points;

determining the relative position of the positioning key point in a three-dimensional space according to the depth information of the positioning key point and the distance of the positioning key point on each two-dimensional sample face image;

and connecting adjacent positioning key points according to the relative positions of the positioning key points in the three-dimensional space to generate an original sample human face three-dimensional model.

7. The method according to claim 1, wherein the adjusting key points of the portion to be face-trimmed on the original sample human face three-dimensional model to obtain a virtual face-trimmed target sample human face three-dimensional model comprises:

generating an adjusting control corresponding to the key point of each face-lifting part;

detecting touch operation of an adjusting control corresponding to a key point of a part to be beautified by the user, and acquiring corresponding adjusting parameters;

and adjusting key points of the part to be subjected to face-lifting on the original sample human face three-dimensional model according to the adjustment parameters to obtain a target sample human face three-dimensional model after virtual face-lifting.

8. The method according to claim 1, wherein the adjusting key points of the portion to be face-trimmed on the original sample human face three-dimensional model to obtain a virtual face-trimmed target sample human face three-dimensional model comprises:

displaying key points of each face-lifting part on the original sample human face three-dimensional model;

and detecting the displacement operation of the key points of the part to be subjected to face-lifting by the user, and adjusting the key points according to the displacement operation to obtain the target sample human face three-dimensional model after virtual face-lifting.

9. The method according to claim 1, wherein the adjusting key points of the portion to be face-trimmed on the original sample human face three-dimensional model to obtain a virtual face-trimmed target sample human face three-dimensional model comprises:

providing a cosmetic suggestion information to the user;

if the user confirms the face-lifting suggestion information, determining key points and adjustment parameters of the part to be face-lifted according to the face-lifting suggestion information;

and adjusting key points of the part to be subjected to face-lifting on the original sample human face three-dimensional model according to the adjustment parameters to obtain a target sample human face three-dimensional model after virtual face-lifting.

10. A virtual face-lifting device for face photographing, comprising:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring original two-dimensional face images of a plurality of current angles of a user, projecting non-uniform speckle structured light to the face of the current user, shooting a structured light image modulated by the face of the current user, and demodulating phase information corresponding to each pixel of the structured light image to obtain depth information corresponding to the original two-dimensional face image;

the reconstruction module is used for carrying out three-dimensional reconstruction according to the depth information and the original two-dimensional face image to obtain an original face three-dimensional model;

the query module is used for querying face information registered in advance and judging whether the user is registered;

the adjusting module is used for inquiring registration information to obtain a face three-dimensional model shaping parameter corresponding to the user when the user is registered, and adjusting key points on the original face three-dimensional model according to the face three-dimensional model shaping parameter to obtain a target face three-dimensional model after virtual face-lifting;

a registration module, configured to, before the registration information is queried to obtain face three-dimensional model shaping parameters corresponding to the user, obtain two-dimensional sample face images of the user at multiple angles and depth information corresponding to each two-dimensional sample face image, perform three-dimensional reconstruction according to the depth information and the two-dimensional sample face images, obtain an original sample face three-dimensional model, adjust a key point of a portion to be reshaped on the original sample face three-dimensional model, obtain a target sample face three-dimensional model after virtual reshaping, compare the original sample face three-dimensional model with the target sample face three-dimensional model, extract face three-dimensional model shaping parameters corresponding to the user, and store the face three-dimensional model shaping parameters in the registration information;

the adjusting module is further configured to, when the user is not registered, acquire attribute features of the user, search for standard face three-dimensional model shaping parameters corresponding to the attribute features according to the attribute features, and adjust key points on the original face three-dimensional model according to the standard face three-dimensional model shaping parameters to obtain a target face three-dimensional model after virtual face-lifting;

and the mapping module is used for mapping the virtual face-lifting target face three-dimensional model to a two-dimensional plane to obtain a target two-dimensional face image.

11. The apparatus of claim 10, wherein the query module comprises:

the extracting unit is used for analyzing the original two-dimensional face image and extracting the facial features of the user;

and the determining unit is used for inquiring a face database registered in advance, judging whether the face features exist or not, if so, determining that the user is registered, and if not, determining that the user is not registered.

12. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the virtual face-lift method for photographing a human face as claimed in any one of claims 1 to 9 when executing the computer program.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a virtual face-lift method for photographing a human face according to any one of claims 1 to 9.

14. An image processing circuit, characterized in that the image processing circuit comprises: an image unit, a depth information unit and a processing unit;

the image unit is used for outputting original two-dimensional face images of a plurality of current angles of a user;

the depth information unit comprises a structured light sensor and a depth map generation chip which are electrically connected, wherein the depth information unit is used for projecting non-uniform speckle structured light to the face of a current user, the structured light sensor is used for generating a structured light image modulated by the face of the current user, and the depth map generation chip is used for demodulating phase information corresponding to each pixel of the structured light image to obtain depth information corresponding to the original two-dimensional face image;

the processing unit is respectively electrically connected with the image unit and the depth information unit and is used for carrying out three-dimensional reconstruction according to the depth information and the original two-dimensional face image to obtain an original face three-dimensional model, inquiring pre-registered face information and judging whether the user is registered or not, inquiring registered information to obtain face three-dimensional model shaping parameters corresponding to the user if the user is registered, adjusting key points on the original face three-dimensional model according to the face three-dimensional model shaping parameters to obtain a target face three-dimensional model after virtual face-lifting,

before the query registration information obtains the face three-dimensional model shaping parameters corresponding to the user, the method comprises the following steps:

acquiring two-dimensional sample face images of the user at a plurality of angles and depth information corresponding to each two-dimensional sample face image,

performing three-dimensional reconstruction according to the depth information and the two-dimensional sample face image to obtain an original sample face three-dimensional model,

adjusting key points of a part to be reshaped on the original sample human face three-dimensional model to obtain a target sample human face three-dimensional model after virtual reshaping,

comparing the original sample human face three-dimensional model with the target sample human face three-dimensional model, extracting human face three-dimensional model shaping parameters corresponding to the user, and storing the human face three-dimensional model shaping parameters in the registration information,

if the user is not registered, acquiring attribute features of the user, searching standard face three-dimensional model shaping parameters corresponding to the attribute features according to the attribute features, and adjusting key points on the original face three-dimensional model according to the standard face three-dimensional model shaping parameters to obtain a target face three-dimensional model after virtual face-lifting;

and mapping the virtual face-lifting target face three-dimensional model to a two-dimensional plane to obtain a target two-dimensional face image.

15. The image processing circuit of claim 14, wherein the image unit comprises an image sensor and an image signal processing ISP processor electrically connected;

the image sensor is used for outputting original image data;

and the image signal processing ISP processor is used for outputting the original two-dimensional face image according to the original image data.

16. The image processing circuit of claim 15, wherein the processing unit comprises a CPU and a GPU electrically connected;

the CPU is used for carrying out three-dimensional reconstruction according to the depth information and the original two-dimensional face image, acquiring an original face three-dimensional model, inquiring face information registered in advance and judging whether the user is registered or not;

and the GPU is used for acquiring a face three-dimensional model shaping parameter corresponding to the user if the user is registered, adjusting key points on the original face three-dimensional model according to the face three-dimensional model shaping parameter to obtain a target face three-dimensional model after virtual face shaping, and mapping the target face three-dimensional model after virtual face shaping to a two-dimensional plane to obtain a target two-dimensional face image.

17. The image processing circuit of claim 16, wherein the GPU is further configured to:

extracting attribute features of the user;

and beautifying the original two-dimensional face image according to the attribute characteristics to obtain a beautified original two-dimensional face image.

18. The image processing circuit according to any of claims 14-17, further comprising a first display unit;

the first display unit is electrically connected with the processing unit and used for displaying the adjusting control corresponding to the key point of the part to be beautified.

19. The image processing circuit according to any of claims 14-17, further comprising a second display unit;

and the second display unit is electrically connected with the processing unit and is used for displaying the target sample human face three-dimensional model after virtual face-lifting.

Images