CN107705248A - Image processing method, device, electronic device, and computer-readable storage medium - Google Patents

Abstract

The application relates to an image processing method, an image processing device, an electronic device and a computer readable storage medium. The method comprises the following steps: carrying out face recognition on an image to be processed, and recognizing a face image in the image to be processed; acquiring a first front face image corresponding to the face image; selecting a plurality of characteristic points from the first front face image according to a preset rule, and respectively obtaining horizontal distance values of the plurality of characteristic points and a face contour; determining a face shape corresponding to the face image according to the ratio of the horizontal distance values; and adjusting the five sense organs in the face image and the contour of the face image according to the face shape. According to the method, after the face image is identified in the image to be processed, the face shape of the face image can be analyzed and obtained, and then the size of five sense organs in the face image and the outline of the face image are correspondingly adjusted according to the face shape of the face image, so that the face shape adjustment of the face image is more intelligent and personalized.

Description

Image processing method, device, electronic device, and computer-readable storage medium

technical field

The present application relates to the field of computer technology, in particular to an image processing method, device, electronic equipment and computer-readable storage medium.

Background technique

With the rapid development of electronic equipment, more and more users use electronic equipment to capture images. The electronic device can beautify the captured image. Furthermore, the electronic device can also perform beauty treatment on the captured portrait image, for example, whitening, skin smoothing, freckle removal, acne removal, etc. on the portrait.

Contents of the invention

Embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium, which can adjust facial features and contours of the face image for the face shape of the face image.

An image processing method, comprising:

Perform face recognition on the image to be processed, and identify the face image in the image to be processed;

Acquiring a first front face image corresponding to the face image;

selecting a plurality of feature points from the first front face image according to preset rules, and obtaining horizontal distance values between the plurality of feature points and the contour of the face;

determining the face shape corresponding to the face image according to the ratio between the horizontal distance values;

Adjusting the facial features in the face image and the outline of the face image according to the face shape.

An image processing device, comprising:

The identification module is used to perform face recognition on the image to be processed, and identify the face image in the image to be processed;

An acquisition module, configured to acquire a first front face image corresponding to the face image;

A face shape determination module, configured to select a plurality of feature points from the first frontal face image according to preset rules, and respectively obtain horizontal distance values between the plurality of feature points and the contour of the face; The ratio determines the face shape corresponding to the face image;

A processing module, configured to adjust the facial features in the face image and the contour of the face image according to the face shape.

An electronic device includes a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the instructions are executed by the processor, the processor is made to execute the method as described above.

A computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the method as described above.

In the embodiment of the present application, after the face image is identified in the image to be processed, the face shape of the face image can be analyzed and obtained, and then the size of the facial features in the face image and the contour of the face image can be adjusted according to the face shape of the face image. The face shape adjustment of the face image is more intelligent and personalized.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic diagram of the internal structure of an electronic device in an embodiment;

Fig. 2 is a flowchart of an image processing method in an embodiment;

Fig. 3 is a schematic diagram of acquiring feature points and horizontal distance values of feature points on the first front face image in one embodiment;

FIG. 4 is a schematic diagram of obtaining feature points and horizontal distance values of feature points on the first front face image in another embodiment;

Fig. 5 is a schematic diagram of rotating the face image to obtain the corresponding first front face image in one embodiment;

Fig. 6 is the flowchart of image processing method in another embodiment;

Fig. 7 is the flowchart of image processing method in another embodiment;

Fig. 8 is a flowchart of an image processing method in another embodiment;

Fig. 9 is a structural block diagram of an image processing device in an embodiment;

Fig. 10 is a structural block diagram of an image processing device in another embodiment;

Fig. 11 is a structural block diagram of an image processing device in another embodiment;

Fig. 12 is a schematic diagram of an image processing circuit in one embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

Fig. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 1, the electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor is used to provide calculation and control capabilities, and support the operation of the entire computer equipment. The memory is used to store data, programs, etc., and at least one computer program is stored on the memory, and the computer program can be executed by the processor to implement the image processing method suitable for electronic equipment provided in the embodiments of the present application. The memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random-access-memory (Random-Access-Memory, RAM). For example, in one embodiment, the memory includes non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs. The computer program can be executed by a processor, so as to implement a wireless network communication method provided in each of the following embodiments. The internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium. The network interface can be an Ethernet card or a wireless network card, etc., and is used for communicating with external computer equipment. The computer device may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device.

Fig. 2 is a flowchart of an image processing method in one embodiment. As shown in Figure 2, an image processing method includes:

Step 202, face recognition is performed on the image to be processed, and the face image in the image to be processed is recognized.

After the electronic device acquires the image to be processed, it may use a face recognition algorithm to perform face recognition on the image to be processed, and identify the face image included in the image to be processed. The image to be processed may be an image taken by the electronic device, an image stored by the electronic device, an image downloaded by the electronic device through a data network or a wireless local area network, and the like.

Step 204, acquiring a first front face image corresponding to the face image.

After the electronic device recognizes the face image in the image to be processed, it can acquire the first front face image corresponding to the face image. If the face image in the image to be processed is a front face image, then the above-mentioned first front face image is a face image; The front face image obtained by restoring the above-mentioned face image according to the shift angle is the first front face image. The above-mentioned front face image refers to a left-right symmetric image of the face area, and the above-mentioned front face image may be a two-dimensional image captured by the subject's face facing the camera.

Step 206, select a plurality of feature points from the first frontal face image according to preset rules, and obtain horizontal distance values between the plurality of feature points and the contour of the human face respectively.

After the electronic device acquires the first front face image corresponding to the human face image, a plurality of feature points may be selected on the first front face image. The electronic device selects a plurality of feature points from the first frontal face image according to preset rules, and the plurality of feature points are located in a closed area formed by the contour of the face in the face image. Wherein, the electronic device selects a plurality of feature points from the first front face image according to preset rules, including: the electronic device can obtain the central axis of the first front face image, and then determine according to the facial features feature points and the central axis in the first face area. The above-mentioned multiple feature points; the electronic device may also directly determine the above-mentioned multiple feature points according to the feature points of facial features, for example, the position of the brow center in the face image is taken as the first feature point.

After acquiring the plurality of feature points on the first frontal face image, the electronic device may respectively acquire horizontal distance values between the plurality of feature points and the contour of the human face. The horizontal distance between the feature point and the face contour refers to the length of the line segment formed by the feature point and the intersection point when a straight line extending in one direction in the horizontal direction starting from the feature point intersects the face contour. In one embodiment, the horizontal distance value between the feature point and the contour of the face may be the length of the line segment formed by the two intersection points when the horizontal straight line including the feature point intersects with the contour of the face.

Step 208, determine the face shape corresponding to the face image according to the ratio between the horizontal distance values.

After the electronic device obtains the horizontal distance values between the plurality of feature points and the contour of the face, the electronic device may obtain a ratio between the horizontal distance values between the plurality of feature points and the contour of the face. The electronic device can determine the face shape corresponding to the face image according to the above ratio.

Face shape refers to the outline of the face. The upper part of the face is composed of the maxilla, zygomatic bone, temporal bone, frontal bone and parietal bone, usually in a circular arc shape; the lower part of the face is mainly composed of the lower bones, which are formed differently. The halves are shaped differently. Due to the different bone sizes and bone shapes of different people, the length and width of the faces of different people are different, that is, different people have different face shapes. in. The face shape of human face can be roughly divided into round face, rectangular face, square face, triangular face and oval face.

Electronic devices can analyze face images of different face shapes in massive data through big data analysis algorithms. According to the analyzed massive data, electronic devices can pre-store the ratio range of horizontal distance values of feature points in different face shapes. After the mobile terminal obtains the ratio of the horizontal distance values between the multiple feature points in the first front face image, it detects the ratio range in which the above ratio falls, and the face shape corresponding to the above ratio range is the face shape of the first front face image, That is, the face shape corresponding to the face image in the image to be processed.

Taking setting four feature points in the first frontal face image as an example, the horizontal distance values corresponding to the four feature points in the round face prestored in the electronic device are respectively a1, b1, c1 and d1. To simplify the ratio of a1:b1:c1:d1, the above simplification method can be to adjust the values of b1, c1 and d1 corresponding to the value of a1 being 1. Then the value range of b1 in the round face shape is 0.95-1, the value range of c1 is 0.68-0.75, and the value range of d1 is 0.45-0.52. If the electronic device detects that the ratio of the horizontal distances of the four feature points in the first human face image is within the range of the ratio of the round face shape, the first frontal face image is a round face shape. The ratio ranges of the above-mentioned 4 feature points and the horizontal distance values of the 4 feature points are only for illustration. In other embodiments, the electronic device can obtain other numbers of feature points, such as 5 or 6; The ratio of the horizontal distance values of the feature points can also be other values.

Step 210, adjusting the facial features in the face image and the outline of the face image according to the face shape.

After acquiring the face shape corresponding to the face image, the electronic device may adjust the facial features in the face image and the outline of the face image according to the face shape corresponding to the face image. Wherein, adjusting the facial features in the face image by electronic equipment refers to adjusting the size of the facial features; adjusting the outline of the facial image by electronic equipment generally refers to reducing the size of the facial image, that is, thinning the face image.

Adjustment strategies corresponding to different face shapes are pre-stored in the electronic device. The adjustment strategy above includes: adjusting the size of the facial features in the face image, and adjusting the contour of the face in the face image.

Adjusting the size of the facial features in the face image includes: the ratio of the standard facial features corresponding to each face shape to the area of the face can be pre-stored in the electronic device. The electronic device can obtain the ratio of the facial features in the face image to the face area, and the electronic device compares the ratio of the facial features in the face image to the ratio of the standard facial features in the corresponding face shape. If the ratio of a certain organ in the facial features to the face area in the face image is greater than the ratio of the corresponding standard organ to the face area in the corresponding face shape, the organ in the face image will be enlarged according to the preset ratio; if the facial features in the face image If the ratio of a certain organ to the area of the face is smaller than the ratio of the corresponding standard organ to the area of the face in the corresponding face shape, the organ in the face image is reduced according to a preset ratio. For example, the ratio of a single standard eye to the face area of a melon-seed face pre-stored in the electronic device is 1/45, if the face image is a melon-seed face and the ratio of a single eye to the face area in the face image is 1/50, then the Eyes in face images are scaled up by 5%.

Adjusting the contour of the face image includes: an adjustment strategy for each face shape can be pre-stored in the electronic device, and the adjustment strategy includes: the direction and angle of the displacement of each point on the face contour to the inside or outside of the face contour. After the electronic device obtains the face shape corresponding to the face image, it can obtain the ratio of the area of the face image to the area of the corresponding face shape, and then adaptively adjust the direction and Angle, adjust the face contour according to the direction and angle of the displacement of each point to the inside or outside of the face contour after adjustment.

The liquefaction tool can be used for adjusting the size of the facial features in the face image and adjusting the outline of the face image.

In the method of the embodiment of the present application, after the face image is recognized in the image to be processed, the face shape of the face image can be analyzed and obtained, and then the size of the facial features in the face image and the contour of the face image can be adjusted according to the face shape of the face image. The face shape adjustment of the face image is more intelligent and personalized.

In one embodiment, respectively obtaining the horizontal distance values between a plurality of feature points and the contour of the human face comprises:

(1) Determine the central axis of the first frontal face image, and determine the positions of a plurality of feature points according to the central axis and the facial features feature points in the first frontal face image.

(2) Obtain the horizontal distance values between multiple feature points and the contour of the face.

After the electronic device acquires the first front face image corresponding to the face image, it can acquire the central axis of the first front face image. The central axis is a straight line in the first front face image, and the first front face image is symmetrical about the central axis. After obtaining the central axis, the electronic device may obtain the feature points of the facial features in the first frontal face image, and the intersection of a horizontal line passing through the feature points of the facial features and the central axis is the feature point. After acquiring the feature points, the electronic device can respectively acquire the horizontal distance values between the feature points and the outline of the human face. The above-mentioned horizontal distance value between the feature point and the contour of the face may be the length of the line segment between the feature point and the intersection point when a horizontal straight line passing through the feature point intersects the contour of the face. It may also be the length of a line segment between two intersection points of a horizontal straight line passing through a feature point and the contour of the face when it intersects with the contour of the face.

As shown in FIG. 3 , the first front face image 30 is left-right symmetrical about the central axis L. As shown in FIG. The electronic device determines the feature points according to the left brow point A, the left alar point B, the left mouth corner point C and the intersection point D of the central axis and the contour of the face in the first front face image 30 . Among them, the intersection point of the horizontal line passing through point A and the central axis L is E1, the intersection point of the horizontal line passing point B and the central axis L is E2, the intersection point of the horizontal line passing point C and the central axis L is E3, and the intersection point of point E3 and point D The midpoint is E4, then E1, E2, E3 and E4 are the feature points in the first front face image. The length of the line segment L1 formed by the intersection of the horizontal line passing through point E1 and the face contour is the horizontal distance value of E1. Similarly, the length of L2 is the horizontal distance value of E2, the length of L3 is the horizontal distance value of E3, and the length of L4 is the horizontal distance value of E1. A horizontal distance value of length E4. The value of L1:L2:L3:L4 is the ratio of the horizontal distance values of E1, E2, E3, and E4 feature points. The above embodiments are only for illustration. In other embodiments, the electronic device may also obtain the feature points according to the feature points of the facial features in the right face area in the first frontal face image.

In one embodiment, acquiring the feature points further includes: acquiring the first facial features feature point in the left face area of the first frontal face image, and acquiring the second facial features feature point in the right face area that is symmetrical to the first facial features feature , take the midpoint of the first feature point and the second feature point as the feature point.

As shown in FIG. 4 , the first front face image 40 is left-right symmetrical. The first facial features feature points in the left face area include: left brow point A, left nose point C, and left mouth corner point E. The second feature points symmetrical to the first feature point in the right face area include: right brow point B, right nose point D, and right mouth corner point F. The electronic device takes the midpoint between point A and point B as G1 , the midpoint between point C and point D as G2 , and the midpoint between point E and point F as G3 . The aforementioned G1, G2, and G3 are the feature points of the first front face image 40 . The length of the line segment L1 formed by the two intersection points where the horizontal straight line passing through the point G1 intersects the contour of the human face is the horizontal distance value of G1. Similarly, the length of L2 is the horizontal distance value of G2, and the length of L3 is the horizontal distance value of G3.

In the method of the embodiment of the present application, the electronic device can acquire the feature points in the face image, and determine the horizontal distance value of the feature point according to the feature point in the face image, which is beneficial to the electronic device according to the feature point and the horizontal distance value of the feature point Determining the face shape of a human face makes it simple and quick for the electronic device to determine the face shape.

In one embodiment, obtaining the first front face image corresponding to the face image includes:

(1) If the face image is a side face image, obtain the offset angle of the face image relative to the standard image.

(2) Obtain the first front face image corresponding to the face image according to the offset angle.

The above-mentioned side face image is a non-frontal face image. Since the human face has a three-dimensional structure, the human face can be rotated in three-dimensional space. When the face rotates at different angles in the three-dimensional space, the two-dimensional face images presented by the face in the image are also different. The rotation angle of the human face in the image in the three-dimensional space can be detected by the software method, that is, the offset angle of the human face image relative to the standard image, and then the above-mentioned side face image can be restored to the corresponding first front face image through an algorithm. The above-mentioned standard image is a frontal face image, and the above-mentioned standard image can be a frontal face image synthesized by an electronic device using software, and the above-mentioned standard head portrait is pre-stored in the electronic device.

The electronic device obtains the offset angle of the face image relative to the standard image, including: the electronic device obtains the distance and angle between the face feature recognition points in the face image, and then determines the offset according to the distance and angle between the above-mentioned face feature points. shift angle. Wherein, the electronic device can obtain the above-mentioned offset angle through artificial intelligence. The offset angle of the face area can be represented by three angles. In three-dimensional space, three mutually perpendicular straight lines intersect at one point to obtain a three-dimensional coordinate system. Every two straight lines of the above three straight lines can form a plane, and a total of three face. Then the rotation angles of the face area relative to the standard face on these three planes are the offset angles of the face area.

After acquiring the offset angle of the face image, the electronic device restores the first frontal face image corresponding to the above-mentioned face image through an algorithm. It specifically includes: sequentially restoring the face image according to the three angles in the offset angles to obtain the first front face image corresponding to the face image.

In the method of the embodiment of the present application, when the face image in the image is a side face image, the side face image can be restored to a front face image, which is beneficial for the electronic device to determine the face shape of the side face image according to the restored front face image.

In one embodiment, the offset angle includes a first angle, a second angle and a third angle; the first angle represents the rotation angle of the face image on the first plane relative to the standard image; the second angle represents the rotation angle of the face image relative to the standard image; The rotation angle of the standard image on the second plane; the third angle represents the rotation angle of the face image relative to the standard image on the third plane; every two planes between the first plane, the second plane and the third plane are perpendicular to each other.

Because the human face can rotate in three-dimensional space, the two-dimensional images presented on the image when the human face rotates are different. When the electronic device analyzes the offset angle of the human face relative to the standard human face, the offset angle can be represented by three angles in a three-dimensional space. The above-mentioned standard face is a two-dimensional face image captured by the camera facing the face. As shown in Figure 5, in three-dimensional space, three mutually perpendicular straight lines intersect at one point to form a space Cartesian coordinate system xyz coordinate system, where the intersection of the x-axis and the y-axis forms the first plane, and the intersection of the x-axis and the z-axis forms In the second plane, the intersection of the y-axis and the z-axis forms a third plane. The first plane, the second plane and the third plane are perpendicular to each other. The first angle represents the rotation angle of the face area relative to the standard face around the z axis, the second angle represents the rotation angle of the face area relative to the standard face around the y axis, and the third angle represents the face area relative to the standard face The rotation angle around the x-axis. In Fig. 5, the side face image 502 is rotated clockwise with respect to the y-axis by an angle α to obtain the first front face image 504, that is, the offset angle of the side face image 502 is the first angle 0, the second angle α, and the third angle angle 0.

When the electronic device records the first angle, the second angle and the third angle corresponding to the offset angle, it can set the positive direction of rotation on the first plane, the second plane and the third plane respectively, and use the face area relative to the standard face The angle of rotation in the positive direction of rotation represents the above-mentioned first angle, second angle, and third angle. For example, taking the second angle as an example, the clockwise direction is the positive direction of rotation. When the face area rotates β clockwise relative to the standard face, the second angle is β; the face area rotates counterclockwise relative to the standard face When γ, the second angle is -γ.

The method in the embodiment of the present application decomposes the offset angle of the human face area relative to the standard human face into rotation angles in three mutually perpendicular planes, and decomposes complex three-dimensional changes into each of the three planes, It is beneficial for the electronic device to analyze the offset angle of the human face.

In one embodiment, after step 210, the above method further includes:

Step 212: Rotate the pre-stored standard beautification template according to the offset angle, where the standard beautification template is a beautification template corresponding to the standard image.

Step 214, fusing the rotated beautification template with the face image.

A standard beautifying template is pre-stored in the electronic device, and the above-mentioned standard beautifying template is a beautifying template corresponding to a standard human face. The above beauty template refers to the layer generated according to the beauty parameters. Different positions in this layer can correspond to different facial feature points in the face. Electronic devices can add different colors to the above layer, for example, the pupil area corresponds to The position corresponding to the black area is used for color contact lenses, the position corresponding to the lip area is displayed red to display lip color, and the position corresponding to the cheekbone area is displayed red to display blush. The electronic device covers the face image with the beautifying template to perform beautification processing on the face image. In layman's terms, the beauty template is similar to a transparent mask, on which different colors can be painted, and the mask can be covered and displayed on the human face.

After the electronic device acquires the offset angle of the face image, it can rotate the above-mentioned standard beautifying template according to the above-mentioned offset angle. The offset angle of the face image relative to the standard face can be represented by the first angle, the second angle and the third angle in the space coordinate system. When the electronic device rotates the standard beautification template according to the offset angle, the standard beautification template can be rotated according to the first angle, the second angle, and the third angle respectively in the space coordinate system, and the offset angle can be obtained The corresponding beauty template.

The beautification parameters in the above beautification template may be the beautification parameters preset by the electronic device, the beautification parameters input by the user, or the beautification parameters given by the electronic device according to the birth date of the image information in the face image.

After the electronic device obtains the rotated beauty template, it can fuse the rotated beauty template with the face image in the image to be processed. Wherein, during fusion processing, the electronic device can determine the position of the beautification template according to the feature points of the facial features in the face image by identifying the feature points of the facial features.

In the method of the embodiment of the present application, for the side face image in the image, the electronic device can perform beautification processing on the side face image by rotating the standard beautification template, and then fusing the rotated beautification template with the face image. The beauty treatment method is more intelligent.

In one embodiment, after step 210, the above method further includes:

Step 216, acquiring the skin color, skin quality and gender corresponding to the face image in the face image.

Step 218, determine the beautification parameters corresponding to the face image according to the skin color, skin quality and gender.

Step 220, perform beautification processing on the face image according to the beautification parameters.

The electronic device can recognize the skin color and skin texture of the face image and the gender corresponding to the face in the face image. Wherein, the electronic device represents the skin color of the face image by the color value of the skin color area, and the electronic device can determine the grade of the skin quality by the number of wrinkles, spots and acne in the face image. Electronic devices can identify the gender corresponding to a face through a machine learning model.

For different skin colors, skin types and genders, electronic devices can match different beauty parameters. For example, when beautifying a female face image in an image, the electronic device will adjust the skin color, lip color, pupil color, blush, etc. of the face image; when beautifying a male face image in an image, the electronic device will only Adjust the skin color and pupil color in the face image. The corresponding relationship between skin color, skin quality, gender and beauty parameters can be pre-stored in the electronic device, and after obtaining the skin color, skin quality and gender of the face image, the electronic device can search for the corresponding beauty parameters. The electronic device may also use a machine learning model to find beauty parameters corresponding to the skin color, skin quality and gender of the face image.

After acquiring the beautification parameters corresponding to the face image, the electronic device may perform beautification processing on the face image according to the above beautification parameters. The above beauty treatments may include: whitening, skin smoothing, freckle removal, acne removal, dark circle removal, etc.

The method in the embodiment of this application can determine the corresponding beautification parameters according to the skin color, skin quality and gender in the face image, that is, different beautification parameters can be obtained for different face images, and different beautification parameters can be realized for different face images. Beauty processing, the beauty processing of face images is more intelligent and personalized.

In one embodiment, after step 210, the above method further includes:

Step 222, identifying the contact person corresponding to the face image.

Step 224, if the contact information corresponding to the contact is stored in the electronic device, send the adjusted image to the electronic device corresponding to the contact.

The electronic device can search for a contact corresponding to a face area. Wherein, the electronic device finds the contacts corresponding to the face area, including any one of the following methods:

(1) The electronic device obtains the marking information of the face image input by the user. The above marking information can be the name of the person corresponding to the face area. The electronic device searches whether there is a name corresponding to the face area in the stored contacts. If there is a person name corresponding to the face area in , the electronic device obtains the contact person corresponding to the face area.

(2) The electronic device can also obtain the avatar corresponding to the stored contact, and the electronic device performs similarity matching between the face area and the avatar corresponding to the stored contact. If the matching is successful, the above-mentioned contact is the contact corresponding to the face area .

After the electronic device obtains the contact corresponding to the face area, it may check whether the contact corresponding to the face area has stored contact information. The above-mentioned contact information may be a mobile phone number, a landline number, a social account number, and the like. When the electronic device has stored the contact information of the above contact, the electronic device sends the fused image to the electronic device corresponding to the above contact.

Usually, when multiple people take a group photo, the user needs to manually share the group photo with other people in the image. In the method of the embodiment of the present application, the electronic device can automatically share the beautified image to the user corresponding to the face in the image, without manual operation by the user, which simplifies the user operation steps and makes the user operation more concise.

Fig. 9 is a structural block diagram of an image processing device in an embodiment. As shown in Figure 9, an image processing device includes:

The recognition module 902 is configured to perform face recognition on the image to be processed, and identify the face image in the image to be processed.

An acquisition module 904, configured to acquire a first frontal face image corresponding to a human face image.

The face shape determining module 906 is configured to select a plurality of feature points from the first frontal face image according to preset rules, and respectively obtain horizontal distance values between the plurality of feature points and the contour of the human face. The face shape corresponding to the face image is determined according to the ratio between the horizontal distance values.

The processing module 908 is configured to adjust the facial features in the face image and the outline of the face image according to the face shape.

In one embodiment, the face shape determining module 906 obtains the horizontal distance values between the plurality of feature points and the contour of the face, respectively, including: determining the central axis of the first frontal face image, and determining The location of multiple feature points. Obtain the horizontal distance values between multiple feature points and the face contour.

In one embodiment, the acquiring module 904 acquiring the first front face image corresponding to the face image includes: if the face image is a side face image, acquiring an offset angle of the face image relative to the standard image. The first front face image corresponding to the face image is acquired according to the offset angle.

In one embodiment, the offset angle includes a first angle, a second angle and a third angle. The first angle represents a rotation angle of the face image relative to the standard image on the first plane. The second angle represents a rotation angle of the face image relative to the standard image on the second plane. The third angle represents the rotation angle of the face image relative to the standard image on the third plane. Every two planes among the first plane, the second plane and the third plane are perpendicular to each other.

Fig. 10 is a structural block diagram of an image processing device in another embodiment. As shown in FIG. 10 , an image processing device includes: an identification module 1002 , an acquisition module 1004 , a face shape determination module 1006 , a processing module 1008 , a rotation module 1010 , and a beautification module 1012 . Among them, the recognition module 1002, the acquisition module 1004, the face shape determination module 1006, and the processing module 1008 have the same functions as the corresponding modules in FIG. 9 .

The rotation module 1010 is configured to rotate a pre-stored standard beautification template according to an offset angle, where the standard beautification template is a beautification template corresponding to a standard image.

The beautifying module 1012 is configured to fuse the rotated beautifying template with the face image.

In one embodiment, the obtaining module 1004 is also used to obtain the skin color, skin quality and gender corresponding to the face image in the face image. The beautification module 1012 is also used to determine the beautification parameters corresponding to the face image according to the skin color, skin quality and gender. Perform beautification processing on the face image according to the beautification parameters.

Fig. 11 is a structural block diagram of an image processing device in another embodiment. As shown in FIG. 11 , an image processing device includes: an identification module 1102 , an acquisition module 1104 , a face shape determination module 1106 , a processing module 1108 , a contact identification module 1110 , and a sending module 1112 . Among them, the recognition module 1102 , the acquisition module 1104 , the face shape determination module 1106 , and the processing module 1108 have the same functions as the corresponding modules in FIG. 9 .

The contact identification module 1110 is configured to identify the contact corresponding to the face image.

The sending module 1112 is configured to send the adjusted image to the electronic device corresponding to the contact if the contact information corresponding to the contact is stored in the electronic device.

The division of each module in the above image processing device is for illustration only. In other embodiments, the image processing device may be divided into different modules as required to complete all or part of the functions of the above image processing device.

The embodiment of the present application also provides a computer-readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to:

(1) Perform face recognition on the image to be processed, and identify the face image in the image to be processed.

(2) Obtain the first front face image corresponding to the face image.

(3) Select a plurality of feature points from the first frontal face image according to preset rules, and obtain horizontal distance values between the plurality of feature points and the contour of the face respectively.

(4) Determine the face shape corresponding to the face image according to the ratio between the horizontal distance values.

(5) Adjust the facial features in the face image and the outline of the face image according to the face shape.

In one embodiment, obtaining the horizontal distance values between the plurality of feature points and the contour of the face includes: determining the central axis of the first frontal face image, and determining a plurality of feature points according to the central axis and the facial features feature points in the first frontal face image s position. Obtain the horizontal distance values between multiple feature points and the face contour.

In one embodiment, acquiring the first front face image corresponding to the face image includes: if the face image is a side face image, acquiring an offset angle of the face image relative to the standard image. The first front face image corresponding to the face image is acquired according to the offset angle.

In one embodiment, the offset angle includes a first angle, a second angle and a third angle. The first angle represents a rotation angle of the face image relative to the standard image on the first plane. The second angle represents a rotation angle of the face image relative to the standard image on the second plane. The third angle represents the rotation angle of the face image relative to the standard image on the third plane. Every two planes among the first plane, the second plane and the third plane are perpendicular to each other.

In an embodiment, it is further performed: rotating a pre-stored standard beautification template according to the offset angle, where the standard beautification template is a beautification template corresponding to the standard image. Fuse the rotated beauty template with the face image.

In one embodiment, it is also performed: acquiring the skin color, skin quality and gender corresponding to the face image in the face image. Determine the beautification parameters corresponding to the face image according to the skin color, skin quality and gender. Perform beautification processing on the face image according to the beautification parameters.

In an embodiment, it is further performed: identifying a contact person corresponding to the face image. If the contact information corresponding to the contact is stored in the electronic device, the adjusted image is sent to the electronic device corresponding to the contact.

The present application also provides a computer program product containing instructions. When the above computer program product is run on a computer, the computer is made to perform the following steps:

(1) Perform face recognition on the image to be processed, and identify the face image in the image to be processed.

(2) Obtain the first front face image corresponding to the face image.

(3) Select a plurality of feature points from the first frontal face image according to preset rules, and obtain horizontal distance values between the plurality of feature points and the contour of the face respectively.

(4) Determine the face shape corresponding to the face image according to the ratio between the horizontal distance values.

(5) Adjust the facial features in the face image and the outline of the face image according to the face shape.

In one embodiment, obtaining the horizontal distance values between the plurality of feature points and the contour of the face includes: determining the central axis of the first frontal face image, and determining a plurality of feature points according to the central axis and the facial features feature points in the first frontal face image s position. Obtain the horizontal distance values between multiple feature points and the face contour.

In one embodiment, acquiring the first front face image corresponding to the face image includes: if the face image is a side face image, acquiring an offset angle of the face image relative to the standard image. The first front face image corresponding to the face image is acquired according to the offset angle.

In one embodiment, the offset angle includes a first angle, a second angle and a third angle. The first angle represents a rotation angle of the face image relative to the standard image on the first plane. The second angle represents a rotation angle of the face image relative to the standard image on the second plane. The third angle represents the rotation angle of the face image relative to the standard image on the third plane. Every two planes among the first plane, the second plane and the third plane are perpendicular to each other.

In an embodiment, it is further performed: rotating a pre-stored standard beautification template according to the offset angle, where the standard beautification template is a beautification template corresponding to the standard image. Fuse the rotated beauty template with the face image.

In one embodiment, it is also performed: acquiring the skin color, skin quality and gender corresponding to the face image in the face image. Determine the beautification parameters corresponding to the face image according to the skin color, skin quality and gender. Perform beautification processing on the face image according to the beautification parameters.

In an embodiment, it is further performed: identifying a contact person corresponding to the face image. If the contact information corresponding to the contact is stored in the electronic device, the adjusted image is sent to the electronic device corresponding to the contact.

Take the electronic device as a mobile terminal as an example. The embodiment of the present application also provides a mobile terminal. The above-mentioned mobile terminal includes an image processing circuit, which may be implemented by hardware and/or software components, and may include various processing units defining an ISP (Image Signal Processing, image signal processing) pipeline. Fig. 12 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 12 , for ease of description, only various aspects of the image processing technology related to the embodiment of the present application are shown.

As shown in FIG. 12 , the image processing circuit includes an ISP processor 1240 and a control logic 1250 . Image data captured by imaging device 1210 is first processed by ISP processor 1240 , which analyzes the image data to capture image statistics that may be used to determine and/or control one or more parameters of imaging device 1210 . Imaging device 1210 may include a camera having one or more lenses 1212 and an image sensor 1214 . Image sensor 1214 may include a color filter array (such as a Bayer filter), and image sensor 1214 may obtain light intensity and wavelength information captured with each imaging pixel of image sensor 1214 and provide a set of raw images that may be processed by ISP processor 1240. image data. The sensor 1220 (such as a gyroscope) may provide the collected image processing parameters (such as anti-shake parameters) to the ISP processor 1240 based on the interface type of the sensor 1220 . The interface of the sensor 1220 may utilize a SMIA (Standard Mobile Imaging Architecture, Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

In addition, image sensor 1214 may also send raw image data to sensor 1220, which may provide raw image data to ISP processor 1240 based on sensor 1220 interface type, or sensor 1220 may store raw image data in image memory 1230.

The ISP processor 1240 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 1240 may perform one or more image processing operations on raw image data, collect statistical information about the image data. Among other things, image processing operations can be performed with the same or different bit depth precision.

The ISP processor 1240 may also receive image data from the image memory 1230 . For example, the sensor 1220 interface sends raw image data to the image memory 1230, and the raw image data in the image memory 1230 is then provided to the ISP processor 1240 for processing. The image memory 1230 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.

Upon receiving raw image data from the image sensor 1214 interface or from the sensor 1220 interface or from the image memory 1230, the ISP processor 1240 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 1230 for additional processing before being displayed. The ISP processor 1240 may also receive processed data from the image memory 1230 and perform image data processing in the raw domain and in the RGB and YCbCr color spaces on the processed data. The processed image data may be output to the display 1280 for viewing by the user and/or for further processing by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the ISP processor 1240 may also be sent to the image memory 1230 , and the display 1280 may read image data from the image memory 1230 . In one embodiment, image memory 1230 may be configured to implement one or more frame buffers. Also, the output of the ISP processor 1240 may be sent to an encoder/decoder 1270 in order to encode/decode image data. The encoded image data may be saved and decompressed prior to display on the display 1280 device.

The steps for the ISP processor 1240 to process the image data include: performing VFE (Video Front End, video front end) processing and CPP (Camera Post Processing, camera post processing) processing on the image data. VFE processing of image data may include correction of contrast or brightness of image data, modification of digitally recorded light state data, compensation processing of image data (such as white balance, automatic gain control, gamma correction, etc.), filter processing, etc. CPP processing of image data may include scaling the image, providing preview frames and recording frames for each path. Among them, CPP can use different codecs to process preview frames and record frames. The image data processed by the ISP processor 1240 may be sent to the beautifying module 1260 so as to perform beautifying processing on the image before being displayed. The beauty treatment of the image data by the beauty module 1260 may include: whitening, freckle removal, skin smoothing, face thinning, acne removal, eye enlargement, etc. Wherein, the beautifying module 1260 may be a CPU (Central Processing Unit, central processing unit), a GPU or a co-processor in an electronic device. The data processed by the beauty module 1260 may be sent to the encoder/decoder 1270 for encoding/decoding image data. The encoded image data may be saved and decompressed prior to display on the display 1280 device. Wherein, the beautification module 1260 may also be located between the encoder/decoder 1270 and the display 1280, that is, the beautification module performs beautification processing on the imaged image. The aforementioned encoder/decoder 1270 may be a CPU, GPU, or coprocessor in an electronic device.

The statistics determined by the ISP processor 1240 may be sent to the control logic 1250 unit. For example, statistics may include image sensor 1214 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 1212 shading correction, etc. Control logic 1250 may include a processor and/or a microcontroller that executes one or more routines (e.g., firmware) that determine control parameters of imaging device 1210 and ISP processing based on received statistical data. The control parameters of the device 1240. For example, control parameters of imaging device 1210 may include sensor 1220 control parameters (e.g., gain, integration time for exposure control), camera flash control parameters, lens 1212 control parameters (e.g., focal length for focus or zoom), or combinations of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), as well as lens 1212 shading correction parameters.

Using the image processing technology in Figure 12 can achieve the following steps:

(1) Perform face recognition on the image to be processed, and identify the face image in the image to be processed.

(2) Obtain the first front face image corresponding to the face image.

(3) Select a plurality of feature points from the first frontal face image according to preset rules, and obtain horizontal distance values between the plurality of feature points and the contour of the face respectively.

(4) Determine the face shape corresponding to the face image according to the ratio between the horizontal distance values.

(5) Adjust the facial features in the face image and the outline of the face image according to the face shape.

In one embodiment, obtaining the horizontal distance values between the plurality of feature points and the contour of the face includes: determining the central axis of the first frontal face image, and determining a plurality of feature points according to the central axis and the facial features feature points in the first frontal face image s position. Obtain the horizontal distance values between multiple feature points and the face contour.

In one embodiment, acquiring the first front face image corresponding to the face image includes: if the face image is a side face image, acquiring an offset angle of the face image relative to the standard image. The first front face image corresponding to the face image is acquired according to the offset angle.

In one embodiment, the offset angle includes a first angle, a second angle and a third angle. The first angle represents a rotation angle of the face image relative to the standard image on the first plane. The second angle represents a rotation angle of the face image relative to the standard image on the second plane. The third angle represents the rotation angle of the face image relative to the standard image on the third plane. Every two planes among the first plane, the second plane and the third plane are perpendicular to each other.

In an embodiment, it is further performed: rotating a pre-stored standard beautification template according to the offset angle, where the standard beautification template is a beautification template corresponding to the standard image. Fuse the rotated beauty template with the face image.

In one embodiment, it is also performed: acquiring the skin color, skin quality and gender corresponding to the face image in the face image. Determine the beautification parameters corresponding to the face image according to the skin color, skin quality and gender. Perform beautification processing on the face image according to the beautification parameters.

In an embodiment, it is further performed: identifying a contact person corresponding to the face image. If the contact information corresponding to the contact is stored in the electronic device, the adjusted image is sent to the electronic device corresponding to the contact.

Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Synchlink DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above examples only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims (10)

1. An image processing method, characterized in that, comprising: Perform face recognition on the image to be processed, and identify the face image in the image to be processed; Acquiring a first front face image corresponding to the face image; selecting a plurality of feature points from the first front face image according to preset rules, and obtaining horizontal distance values between the plurality of feature points and the contour of the face; determining the face shape corresponding to the face image according to the ratio between the horizontal distance values; Adjusting the facial features in the face image and the outline of the face image according to the face shape. 2. The method of claim 1. It is characterized in that said acquiring the horizontal distance values between said plurality of feature points and the human face contour respectively comprises: Determining the central axis of the first front face image, and determining the positions of the plurality of feature points according to the central axis and the facial features feature points in the first front face image; Acquiring horizontal distance values between the plurality of feature points and the contour of the human face. 3. The method according to claim 1, wherein said obtaining the first front face image corresponding to said human face image comprises: If the face image is a side face image, obtain the offset angle of the face image relative to the standard image; Acquire a first front face image corresponding to the face image according to the offset angle. 4. The method according to claim 3, characterized in that: The offset angle includes a first angle, a second angle and a third angle; The first angle represents the rotation angle of the face image relative to the standard image on the first plane; The second angle represents the rotation angle of the face image relative to the standard image on the second plane; The third angle represents the rotation angle of the face image relative to the standard image on a third plane; Every two planes among the first plane, the second plane and the third plane are perpendicular to each other. 5. method according to claim 3, is characterized in that, described method also comprises: Rotating a pre-stored standard beautification template according to the offset angle, the standard beautification template being a beautification template corresponding to the standard image; The rotated beauty template is fused with the face image. 6. The method according to any one of claims 1 to 5, further comprising: Obtaining the skin color, skin quality and gender corresponding to the face image in the face image; determining the beautification parameters corresponding to the face image according to the skin color, skin quality and gender; Perform beautification processing on the face image according to the beautification parameters. 7. The method according to any one of claims 1 to 5, wherein the method further comprises: identifying the contact person corresponding to the face image; If the contact information corresponding to the contact is stored in the electronic device, the adjusted image is sent to the electronic device corresponding to the contact. 8. An image processing device, comprising: The identification module is used to perform face recognition on the image to be processed, and identify the face image in the image to be processed; An acquisition module, configured to acquire a first front face image corresponding to the face image; A face shape determination module, configured to select a plurality of feature points from the first frontal face image according to preset rules, and respectively obtain horizontal distance values between the plurality of feature points and the contour of the face; The ratio determines the face shape corresponding to the face image; A processing module, configured to adjust the facial features in the face image and the contour of the face image according to the face shape. 9. An electronic device, comprising a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the instructions are executed by the processor, the processor executes any one of claims 1 to 7. method described in the item. 10. A computer-readable storage medium, on which a computer program is stored, wherein the computer program is executed by a processor according to the method according to any one of claims 1 to 7.