CN112184580B - A method, device, equipment and storage medium for enhancing face image - Google Patents
A method, device, equipment and storage medium for enhancing face image Download PDFInfo
- Publication number
- CN112184580B CN112184580B CN202011031567.7A CN202011031567A CN112184580B CN 112184580 B CN112184580 B CN 112184580B CN 202011031567 A CN202011031567 A CN 202011031567A CN 112184580 B CN112184580 B CN 112184580B
- Authority
- CN
- China
- Prior art keywords
- face
- image
- face image
- enhanced
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000002708 enhancing effect Effects 0.000 title description 9
- 230000001815 facial effect Effects 0.000 claims abstract description 21
- 238000006731 degradation reaction Methods 0.000 claims description 56
- 230000015556 catabolic process Effects 0.000 claims description 55
- 238000013145 classification model Methods 0.000 claims description 43
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 13
- 230000006870 function Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000003672 processing method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000366 juvenile effect Effects 0.000 description 3
- 208000002874 Acne Vulgaris Diseases 0.000 description 2
- 206010000496 acne Diseases 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 208000003351 Melanosis Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000002087 whitening effect Effects 0.000 description 1
- 230000037303 wrinkles Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
本公开涉及一种人脸图像增强方法、装置、设备及存储介质，所述方法包括：基于待增强人脸图像的人脸属性，确定对应的人脸类别，作为目标类别。从各个人脸类别分别对应的图像增强模型中，确定所述目标类别对应的图像增强模型，作为目标模型；各个图像增强模型为基于自身对应的人脸类别的人脸图像样本训练得到。将待增强人脸图像输入至目标模型中，经过增强处理后得到待增强人脸图像对应的人脸增强图像。由于本公开利用不同的图像增强模型分别对不同类别的人脸图像进行增强处理，图像增强模型能够针对对应类别的人脸图像的图像增强需求进行增强处理，对于每一张人脸图像而言，能够得到尽量满足其增强需求的人脸增强图像，提高人脸图像增强效果。
The present disclosure relates to a facial image enhancement method, device, equipment and storage medium, the method comprising: based on the facial attributes of the facial image to be enhanced, determining the corresponding facial category as the target category. From the image enhancement models corresponding to each facial category, determining the image enhancement model corresponding to the target category as the target model; each image enhancement model is trained based on the facial image samples of the facial category to which it corresponds. The facial image to be enhanced is input into the target model, and after enhancement processing, a facial enhanced image corresponding to the facial image to be enhanced is obtained. Since the present disclosure uses different image enhancement models to perform enhancement processing on facial images of different categories respectively, the image enhancement model can perform enhancement processing according to the image enhancement requirements of the facial images of the corresponding categories. For each facial image, a facial enhanced image that meets its enhancement requirements as much as possible can be obtained, thereby improving the facial image enhancement effect.
Description
Technical Field
The disclosure relates to the technical field of image processing, and in particular relates to a face image enhancement method, a device, equipment and a storage medium.
Background
At present, the image enhancement technology is widely applied to the fields of image and video processing and mainly comprises technologies such as image denoising, sharpening, deblurring, color enhancement, super resolution and the like. The image enhancement technique may perform enhancement processing on the entire image, or may perform individual enhancement processing on only a certain region in the image. Face image enhancement generally refers to detecting a region where a face is located in an image, and then performing separate enhancement processing on the region where the face is located.
In the existing face image enhancement method, the same image enhancement mode is adopted for all face images, and different types of face images have different characteristics, so that different image enhancement processing requirements possibly exist.
Therefore, how to implement enhancement processing on face images of different types is a technical problem that needs to be solved at present.
Disclosure of Invention
In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a face image enhancement method, a device, equipment and a storage medium.
In a first aspect, the present disclosure provides a face image enhancement method, the method including:
Based on the face attribute of the face image to be enhanced, determining the face category corresponding to the face image to be enhanced as a target category;
Determining an image enhancement model corresponding to the target class from the image enhancement models corresponding to the face classes respectively, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model;
And inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancing treatment of the target model.
In an optional implementation manner, the determining, based on the face attribute of the face image to be enhanced, the face category corresponding to the face image to be enhanced, as the target category includes:
Inputting a face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model, wherein the image classification model is obtained by training based on a face image sample with a class label, and the class label is determined based on the face attribute of the face image sample;
And determining the face category corresponding to the face image to be enhanced as a target category based on the classification result.
In an optional implementation manner, the determining, from the image enhancement models corresponding to the face classes, the image enhancement model corresponding to the target class, before being used as the target model, further includes:
Constructing a face image sample set corresponding to a first face category, wherein the face image sample set comprises a noiseless face image sample and a noisy face image sample;
training a preset machine model by using the face image sample corresponding to the first face class to obtain an image enhancement model corresponding to the first face class.
In an optional implementation manner, the constructing a face image sample set corresponding to the first face category includes:
Acquiring a noise-free face image sample corresponding to a first face class;
carrying out degradation treatment on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face class;
And constructing a face image sample set corresponding to the first face class based on the noisy face image sample and the noiseless face image sample.
In an optional implementation manner, the performing degradation processing on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face class includes:
Determining a degradation processing mode corresponding to the first face class from degradation processing modes corresponding to the face classes respectively, wherein the degradation processing mode comprises at least one mode for increasing image noise;
And carrying out degradation processing on the noiseless face image sample by utilizing a degradation processing mode corresponding to the first face class to obtain a noisy face image sample corresponding to the first face class.
In an optional implementation manner, before the face image to be enhanced is input into the image classification model and the classification result corresponding to the face image to be enhanced is obtained after the classification processing of the image classification model, the method further includes:
denoising the face image to be enhanced according to preset denoising intensity to obtain a denoised face image;
correspondingly, the step of inputting the face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model comprises the following steps:
and inputting the face image after denoising into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model.
In an alternative embodiment, the face attribute includes at least one of age, gender, face value, and race.
In a second aspect, the present disclosure provides a face image enhancement apparatus, the apparatus comprising:
The first determining module is used for determining a face category corresponding to the face image to be enhanced based on the face attribute of the face image to be enhanced, and taking the face category as a target category;
The second determining module is used for determining an image enhancement model corresponding to the target class from the image enhancement models corresponding to the face classes respectively as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model;
The enhancement processing module is used for inputting the face image to be enhanced into the target model, and obtaining a face enhancement image corresponding to the face image to be enhanced after enhancement processing of the target model.
In an alternative embodiment, the first determining module includes:
The image classification module is used for inputting a face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model, wherein the image classification model is obtained by training a face image sample with a class label, and the class label is determined based on the face attribute of the face image sample;
And the first determining submodule is used for determining the face category corresponding to the face image to be enhanced based on the classification result and taking the face category as a target category.
In an alternative embodiment, the apparatus further comprises:
The system comprises a building module, a first face classification module and a second face classification module, wherein the building module is used for building a face image sample set corresponding to the first face classification, and the face image sample set comprises a noiseless face image sample and a noisy face image sample;
And the training module is used for training a preset machine model by utilizing the face image sample set corresponding to the first face class to obtain an image enhancement model corresponding to the first face class.
In an alternative embodiment, the building block comprises:
the acquisition sub-module is used for acquiring a noiseless face image sample corresponding to the first face class;
The first degradation processing submodule is used for carrying out degradation processing on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face class;
And the construction submodule is used for constructing a face image sample set corresponding to the first face class based on the noisy face image sample and the noiseless face image sample.
In an alternative embodiment, the first degradation processing sub-module includes:
the second determining submodule is used for determining a degradation processing mode corresponding to the first face class from degradation processing modes corresponding to the face classes respectively, wherein the degradation processing mode comprises at least one mode for increasing image noise;
And the first degradation processing submodule is used for carrying out degradation processing on the noiseless face image sample by utilizing a degradation processing mode corresponding to the first face category to obtain a noisy face image sample corresponding to the first face category.
In an alternative embodiment, the apparatus further comprises:
The denoising module is used for denoising the face image to be enhanced according to preset denoising intensity to obtain a denoised face image;
Correspondingly, the classifying sub-module is specifically configured to:
and inputting the face image after denoising into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model.
In an alternative embodiment, the face attribute includes at least one of age, gender, face value, and race.
In a third aspect, the present disclosure provides a computer readable storage medium having instructions stored therein which, when executed on a terminal device, cause the terminal device to implement the method of any one of the above.
In a fourth aspect, the present disclosure provides an apparatus comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of the preceding claims when executing the computer program.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:
The embodiment of the disclosure provides a face image enhancement method, which comprises the steps of firstly, determining a face category corresponding to a face image to be enhanced based on a face attribute of the face image to be enhanced, and taking the face category as a target category. And then, determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model. And finally, inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancing treatment of the target model. According to the face image enhancement processing method and device, different types of face images are respectively enhanced by using different image enhancement models, and because the various types of image enhancement models are obtained by training based on the face image sample sets of the corresponding types, the image enhancement models can enhance the image enhancement requirements of the face images of the corresponding types, and for each face image, the face enhancement images meeting the enhancement requirements as much as possible can be obtained, the face image enhancement effect is improved, and the problem that the face image enhancement generalization is poor is avoided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a flowchart of a face image enhancement method provided in an embodiment of the present disclosure;
FIG. 2 is a flow chart of a degradation process provided by an embodiment of the present disclosure;
Fig. 3 is a schematic structural diagram of a face image enhancement device according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a face image enhancement device according to an embodiment of the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein, and it is apparent that the embodiments in the specification are only some, rather than all, of the embodiments of the present disclosure.
The face image is an image in which a face is displayed in the image, and the displayed face occupies a large proportion of the whole image. The enhancement of the face image means that the quality of the face image is enhanced and the display effect of the face image is improved by carrying out enhancement processing such as denoising, sharpening, deblurring, color enhancement, detail generation and the like on the face image. Face image enhancement is also known as face image restoration.
Face image enhancement is commonly used to repair faces in old photos, old videos, or poor quality photos, videos. The face image enhancement method can be realized based on a network model at present, specifically, a large number of face image samples are utilized to train the network model, a trained network model is obtained, and then the network model is utilized to enhance the face image, so that the enhanced face image is obtained.
However, in the existing face image enhancement method, because the network model is obtained by training together based on face image samples of various types, the face image enhancement processing mode learned by the trained network model is a mode for enhancing common characteristics of face images of various types. For example, for a male face image and a female face image, the current face image enhancement method is the same for both enhancement treatment modes, and in fact, the requirements of the male and female for face enhancement treatment are different, and the female may need not only whitening, filtering, acne removal, freckle removal, but also facial form, eyes, nose and other shape and angle adjustment, and the male needs relatively less face treatment. Therefore, the same enhancement processing is performed on the face images of various categories by using the same network model, and obviously, the generalization of the face image enhancement is not good, and the expected enhancement effect cannot be achieved on the face images of various categories.
Therefore, the present disclosure provides a face image enhancement method, first, based on a face attribute of a face image to be enhanced, a face category corresponding to the face image to be enhanced is determined as a target category. And then, determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model. And finally, inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancing treatment of the target model. According to the face image enhancement processing method and device, different types of face images are enhanced by using different image enhancement models, and because the various types of image enhancement models are obtained by training the face image sample sets based on the corresponding types, the face images of the corresponding types are enhanced by using the image enhancement models of the corresponding types, and for each face image, the face enhancement images meeting enhancement requirements as much as possible can be obtained, the enhancement effect of the face images is improved, and the problem that the generalization of face image enhancement is poor is avoided.
Based on this, the present disclosure provides a face image enhancement method, referring to fig. 1, fig. 1 is a flowchart of a face image enhancement method provided by an embodiment of the present disclosure, where the face image enhancement method includes:
s101, determining a face class corresponding to a face image to be enhanced based on the face attribute of the face image to be enhanced, and taking the face class as a target class.
In the embodiment of the disclosure, before determining a face class corresponding to a face image to be enhanced, the face image to be enhanced is firstly classified based on a face attribute. The face attribute may include gender, age, race, face value, etc. That is, the embodiment of the present disclosure may divide the face image to be enhanced into face categories based on at least one attribute of gender, age, race, and face value.
Taking the face attribute including gender and age as an example, the face image can be divided into two types of men and women according to gender, the face image can be divided into three types of teenagers, middle-aged and elderly people according to age, and the face image of the young and elderly people has stronger age than gender characteristics, so that the gender does not need to be distinguished, and therefore, the face image can be determined into four types, namely teenagers, middle-aged and young men, middle-aged and young women and elderly people by combining the attribute of gender and age.
In the embodiment of the present disclosure, a face class corresponding to a face image to be enhanced may be determined as a target class based on a face attribute of the face image to be enhanced based on the above-described face class classification method.
In an alternative embodiment, a deep learning manner may be used to determine a face class corresponding to the face image to be enhanced. Specifically, the image classification model may be used to classify the face image to be enhanced to obtain a classification result, and then the face class corresponding to the face image to be enhanced is determined based on the classification result. Specifically, a face image to be enhanced is firstly input into an image classification model, and a classification result corresponding to the face image to be enhanced is obtained after classification processing of the image classification model, wherein the image classification model is obtained by training based on a face image sample with a class label, and the class label is determined based on face attributes of the face image sample. And then, based on the classification result, determining the face category corresponding to the face image to be enhanced as a target category.
Prior to this, a face image training sample with class labels for training an image classification model is first constructed. And then, training the image classification model by using a large number of face image training samples with class labels to finish training the image classification model. The image classification model may be implemented based on Resnet residual networks, and the like.
In another optional implementation manner, feature analysis may be performed on the face image to be enhanced based on the face attribute of the face image to be enhanced, so as to determine the face category corresponding to the face image to be enhanced.
In order to improve accuracy of determining a face class corresponding to a face image to be enhanced, the embodiment of the disclosure may perform slight image denoising processing on the face image to be enhanced before determining the face class corresponding to the face image to be enhanced, so as to remove noise with larger intensity on the face image to be enhanced, so as to avoid influence of the noise with larger intensity on determining the face class of the face image to be enhanced. Specifically, denoising the face image to be enhanced with preset denoising intensity. The preset denoising strength may be set to a weaker denoising strength. Taking NL-Mean (Non-Local Means) denoising algorithm as an example, when the NL-Mean denoising algorithm is used to denoise the face image to be enhanced, the function of fastNlMeansDenoisingColored functions in opencv is to denoise the color image, and the filter intensity parameters h and hForColorComponents are set to about 3.
In practical application, denoising treatment is carried out on the face image to be enhanced with preset denoising intensity, after the face image after denoising is obtained, the face image after denoising is input into an image classification model, and after the classification treatment of the image classification model, a classification result corresponding to the face image to be enhanced is obtained. The embodiment of the disclosure performs slight image denoising processing on the face image to be enhanced, so that the noise with larger intensity on the face image to be enhanced can be removed, thereby avoiding the influence of the noise with larger intensity on the face type determination of the face image to be enhanced, and improving the accuracy of the face type determination.
S102, determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model.
In practical application, before the enhancement processing is performed on the face image to be enhanced by using the image enhancement model, firstly, respectively constructing a corresponding image enhancement model for each face class, respectively constructing face image samples corresponding to each class, and then training the image enhancement model of the corresponding class by using the face image samples corresponding to each class.
In an alternative embodiment, a large number of face images may be collected for use in constructing a face image sample. Specifically, for each collected face image, a face class corresponding to the face image is detected first, for example, the face image belongs to a young and middle-aged man class, and then a face image sample set of the face class is constructed by using the face image.
In order to improve the processing capacity of the image enhancement model and further improve the enhancement effect on the face image, the embodiment of the disclosure can respectively construct a high-quality face image sample set and a low-quality face image sample set aiming at each category, and the image enhancement model is trained by utilizing the high-quality face image sample set and the low-quality face image sample set at the same time, so that the trained image enhancement model can obtain a better enhancement effect on the low-quality face image.
In practical application, the high-quality face image refers to a high-definition face image with less noise and clearer image, namely a noise-free face image sample. After the high-quality face image sample set corresponding to each category is obtained, the low-quality face image sample set corresponding to the category can be constructed by utilizing the high-quality face image sample set.
In an optional implementation manner, in a manner of constructing a low-quality face image sample set of a corresponding category based on high-quality face image sample sets of each category, degradation processing is performed on face image samples in the high-quality face image sample sets of each category to obtain degraded face image samples, and the degraded face image samples are used for constructing the low-quality face image sample set of the category, namely noisy face image samples.
Because the requirements of the face images of different categories on the face enhancement effect are different, the degradation processing modes of the embodiment of the disclosure for the high-quality face image sample sets of different categories are different. According to the face image sample collection processing method and device, after the degradation processing modes corresponding to the face categories are determined, the corresponding degradation processing modes are determined from the degradation processing modes corresponding to the face categories, and then the degradation processing is carried out on the face image samples in the high-quality face image sample collection of the categories by the corresponding degradation processing modes, so that the low-quality face image sample collection corresponding to the categories is finally obtained.
Taking a first face class as an example, determining a degradation processing mode corresponding to the first face class from degradation processing modes respectively corresponding to the face classes, wherein the degradation processing mode comprises at least one mode for increasing image noise, and then carrying out degradation processing on the noiseless face image sample by utilizing the degradation processing mode corresponding to the first face class to obtain a noisy face image sample corresponding to the first face class. And further, a face image sample set corresponding to the first face class is constructed based on the noisy face image sample and the noiseless face image sample. The face image sample set comprises a noiseless face image sample and a noisy face image sample. And finally, training a preset machine model by using a face image sample corresponding to the first face class to obtain an image enhancement model corresponding to the first face class.
Taking four categories of teenagers, young and middle-aged men, young and middle-aged women and old people as examples, as shown in fig. 2, a degradation processing flow chart is provided in an embodiment of the disclosure. The face images of the juvenile class and the young and middle-aged male class have low requirements on enhancement effects, so that when a low-quality face image sample set is generated, image samples in the high-quality face image sample set of the juvenile class are subjected to resolution processing firstly, namely the image samples are subjected to downsampling to low-resolution images, and then are subjected to processing of downsampling to return to the original resolution images. And then, performing Jpeg strong noise processing on the image sample subjected to the resolution processing, namely adding Jpeg noise with higher intensity into the image sample to obtain the image sample subjected to the Jpeg noise processing, and constructing a low-quality face image sample set of the juvenile class. Aiming at the image samples in the high-quality face image sample set of the middle-young male category, firstly, carrying out the Resize processing, and secondly, carrying out the Jpeg medium-strength noise processing on the image samples subjected to the Resize processing to obtain the image samples subjected to the Jpeg noise processing, and constructing the low-quality face image sample set of the middle-young male category.
Because face images of middle-aged and young women and old people have high requirements on enhancement effects, more degradation processing steps are needed when a low-quality face image sample set is generated. Specifically, for the image samples in the high-quality face image sample set of the middle-aged and young women, the resolution processing is firstly performed, then the Gaussian noise processing is performed, so that the trained image enhancement model can remove acne on the face, then the Jpeg strong noise processing is performed, and finally the image samples subjected to the Jpeg noise processing are obtained and are used for constructing the low-quality face image sample set of the middle-aged and young women. Similarly, for the image samples in the high-quality face image sample set of the elderly class, firstly, carrying out the resolution processing, then carrying out the Gaussian blur processing, so that the trained image enhancement model can remove wrinkles on the face, further carrying out the middle-strength Jpeg noise processing, finally obtaining the image samples subjected to the Jpeg noise processing, and constructing the low-quality face image sample set of the elderly class.
In another optional implementation manner, the low-quality face image sample set of each category may also be obtained by directly collecting the low-quality face image, and for other ways of obtaining the low-quality face image sample set, the disclosure embodiment will not be repeated.
In the embodiment of the disclosure, after a high-quality face image sample set and a low-quality face image sample set of each category are obtained, training an image enhancement model corresponding to the corresponding category by using the high-quality face image sample set and the low-quality face image sample set of each category to obtain a trained image enhancement model corresponding to each category. After the face class corresponding to the face image to be enhanced is determined as the target class, determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes, and taking the image enhancement model as the target model for enhancing the face image to be enhanced.
S103, inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancing treatment of the target model.
In the embodiment of the disclosure, after determining a face class corresponding to a face image to be enhanced, an image enhancement model corresponding to the face class, that is, a target model, is further determined, and then the image enhancement processing is performed on the face image to be enhanced by using the target model. The image enhancement model may be implemented using various deep learning models, among others.
The image enhancement processing is performed on the face image to be enhanced, and the quality of the face image to be enhanced can be enhanced by performing enhancement processing such as denoising, sharpening, deblurring, color enhancement, detail generation and the like on the face image to be enhanced, so that the display effect of the face image to be enhanced is improved.
In the face image enhancement method provided by the embodiment of the present disclosure, first, a face class corresponding to a face image to be enhanced is determined as a target class based on a face attribute of the face image to be enhanced. And then, determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model. And finally, inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancing treatment of the target model. According to the face image enhancement processing method and device, different types of face images are enhanced by using different image enhancement models, and because the various types of image enhancement models are obtained by training the face image sample sets based on the corresponding types, the face images of the corresponding types are enhanced by using the image enhancement models of the corresponding types, and for each face image, the face enhancement images meeting enhancement requirements as much as possible can be obtained, the enhancement effect of the face images is improved, and the problem that the generalization of face image enhancement is poor is avoided.
Corresponding to the foregoing method implementation manner, the present disclosure further provides a facial image enhancement device, referring to fig. 3, which is a schematic structural diagram of a facial image enhancement device provided by an embodiment of the present disclosure, where the device includes:
a first determining module 301, configured to determine, based on a face attribute of a face image to be enhanced, a face class corresponding to the face image to be enhanced, as a target class;
The second determining module 302 is configured to determine, from the image enhancement models respectively corresponding to the face classes, an image enhancement model corresponding to the target class as a target model, where each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model;
The enhancement processing module 303 is configured to input the face image to be enhanced into the target model, and obtain a face enhancement image corresponding to the face image to be enhanced after enhancement processing of the target model.
In an optional implementation manner, the machine model may be used to perform classification processing on the face image to be enhanced, so as to improve accuracy of the classification result, so that the first determining module includes:
The image classification module is used for inputting a face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model, wherein the image classification model is obtained by training a face image sample with a class label, and the class label is determined based on the face attribute of the face image sample;
And the first determining submodule is used for determining the face category corresponding to the face image to be enhanced based on the classification result and taking the face category as a target category.
In an alternative embodiment, the machine model may be trained based on the noisy face image sample and the noiseless face image sample, to obtain a better image enhancement model, so that the apparatus further includes:
The system comprises a building module, a first face classification module and a second face classification module, wherein the building module is used for building a face image sample set corresponding to the first face classification, and the face image sample set comprises a noiseless face image sample and a noisy face image sample;
And the training module is used for training a preset machine model by utilizing the face image sample set corresponding to the first face class to obtain an image enhancement model corresponding to the first face class.
In an alternative embodiment, noisy image samples may be constructed based on noise-free face image samples. To this end, the building block comprises:
the acquisition sub-module is used for acquiring a noiseless face image sample corresponding to the first face class;
The first degradation processing submodule is used for carrying out degradation processing on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face class;
And the construction submodule is used for constructing a face image sample set corresponding to the first face class based on the noisy face image sample and the noiseless face image sample.
In an alternative embodiment, to improve the processing capability of the image enhancement model, the first degradation processing sub-module includes:
the second determining submodule is used for determining a degradation processing mode corresponding to the first face class from degradation processing modes corresponding to the face classes respectively, wherein the degradation processing mode comprises at least one mode for increasing image noise;
And the first degradation processing submodule is used for carrying out degradation processing on the noiseless face image sample by utilizing a degradation processing mode corresponding to the first face category to obtain a noisy face image sample corresponding to the first face category.
In an alternative embodiment, to improve accuracy of the classification result, the apparatus further includes:
The denoising module is used for denoising the face image to be enhanced according to preset denoising intensity to obtain a denoised face image;
Correspondingly, the classifying sub-module is specifically configured to:
and inputting the face image after denoising into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model.
In an alternative embodiment, the face attribute includes at least one of age, gender, face value, and race.
In the face image enhancement device provided by the embodiment of the present disclosure, first, a face class corresponding to a face image to be enhanced is determined as a target class based on a face attribute of the face image to be enhanced. And then, determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model. And finally, inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancing treatment of the target model. Because the embodiment of the disclosure uses different image enhancement models to enhance the face images of different categories, the image enhancement models can enhance the image enhancement requirements of the face images of corresponding categories, and for each face image, the embodiment of the disclosure can obtain the face enhancement image which meets the enhancement requirements as much as possible, and the face image enhancement effect is improved.
In addition, the embodiment of the present disclosure further provides a facial image enhancement device, as shown in fig. 4, which may include:
A processor 401, a memory 402, an input device 403 and an output device 404. The number of processors 401 in the face image enhancement device may be one or more, one processor being exemplified in fig. 4. In some embodiments of the invention, the processor 401, memory 402, input device 403, and output device 404 may be connected by a bus or other means, with the bus connection being exemplified in FIG. 4.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing of the face image enhancement device by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area that may store an operating system, application programs required for at least one function, and the like, and a storage data area. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. The input means 403 may be used to receive input numeric or character information and to generate signal inputs related to user settings and function control of the facial image enhancement apparatus.
In particular, in this embodiment, the processor 401 loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement the various functions of the face image enhancement device.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of face image enhancement, the method comprising:
Based on the face attribute of the face image to be enhanced, determining the face category corresponding to the face image to be enhanced as a target category;
Determining an image enhancement model corresponding to the target class from the image enhancement models corresponding to the face classes respectively, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model;
Inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after enhancement processing of the target model;
The method comprises the steps of determining an image enhancement model corresponding to the target class from the image enhancement models respectively corresponding to the face classes, and further comprises the steps of:
Constructing a face image sample set corresponding to a first face category, wherein the face image sample set comprises a noiseless face image sample and a noisy face image sample;
training a preset machine model by utilizing a face image sample corresponding to the first face class to obtain an image enhancement model corresponding to the first face class;
the constructing a face image sample set corresponding to the first face category includes:
Acquiring a noise-free face image sample corresponding to a first face class;
carrying out degradation treatment on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face class;
constructing a face image sample set corresponding to the first face class based on the noisy face image sample and the noiseless face image sample;
The step of performing degradation processing on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face category includes:
Determining a degradation processing mode corresponding to the first face class from degradation processing modes corresponding to the face classes respectively, wherein the degradation processing mode comprises at least one mode for increasing image noise;
And carrying out degradation processing on the noiseless face image sample by utilizing a degradation processing mode corresponding to the first face class to obtain a noisy face image sample corresponding to the first face class.
2. The method according to claim 1, wherein the determining, based on the face attribute of the face image to be enhanced, the face category corresponding to the face image to be enhanced as the target category includes:
Inputting a face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model, wherein the image classification model is obtained by training based on a face image sample with a class label, and the class label is determined based on the face attribute of the face image sample;
And determining the face category corresponding to the face image to be enhanced as a target category based on the classification result.
3. The method according to claim 2, wherein the step of inputting the face image to be enhanced into an image classification model, after the classification processing of the image classification model, before obtaining the classification result corresponding to the face image to be enhanced, further comprises:
denoising the face image to be enhanced according to preset denoising intensity to obtain a denoised face image;
correspondingly, the step of inputting the face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model comprises the following steps:
and inputting the face image after denoising into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model.
4. The method of claim 1, wherein the face attribute comprises at least one of age, gender, face value, and race.
5. A facial image enhancement apparatus, the apparatus comprising:
The first determining module is used for determining a face category corresponding to the face image to be enhanced based on the face attribute of the face image to be enhanced, and taking the face category as a target category;
The second determining module is used for determining an image enhancement model corresponding to the target class from the image enhancement models corresponding to the face classes respectively as the target model, wherein each image enhancement model is obtained by training a face image sample based on the face class corresponding to the image enhancement model;
The enhancement processing module is used for inputting the face image to be enhanced into the target model, and obtaining a face enhancement image corresponding to the face image to be enhanced after enhancement processing of the target model;
the apparatus further comprises:
The system comprises a building module, a first face classification module and a second face classification module, wherein the building module is used for building a face image sample set corresponding to the first face classification, and the face image sample set comprises a noiseless face image sample and a noisy face image sample;
The training module is used for training a preset machine model by utilizing the face image sample set corresponding to the first face class to obtain an image enhancement model corresponding to the first face class;
The construction module comprises:
the acquisition sub-module is used for acquiring a noiseless face image sample corresponding to the first face class;
The first degradation processing submodule is used for carrying out degradation processing on the noiseless face image sample to obtain a noisy face image sample corresponding to the first face class;
A construction sub-module, configured to construct a face image sample set corresponding to the first face class based on the noisy face image sample and the noiseless face image sample;
the first degradation processing sub-module includes:
the second determining submodule is used for determining a degradation processing mode corresponding to the first face class from degradation processing modes corresponding to the face classes respectively, wherein the degradation processing mode comprises at least one mode for increasing image noise;
And the first degradation processing submodule is used for carrying out degradation processing on the noiseless face image sample by utilizing a degradation processing mode corresponding to the first face category to obtain a noisy face image sample corresponding to the first face category.
6. The apparatus of claim 5, wherein the first determining module comprises:
The image classification module is used for inputting a face image to be enhanced into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model, wherein the image classification model is obtained by training a face image sample with a class label, and the class label is determined based on the face attribute of the face image sample;
And the first determining submodule is used for determining the face category corresponding to the face image to be enhanced based on the classification result and taking the face category as a target category.
7. The apparatus of claim 6, wherein the apparatus further comprises:
The denoising module is used for denoising the face image to be enhanced according to preset denoising intensity to obtain a denoised face image;
Correspondingly, the classifying sub-module is specifically configured to:
and inputting the face image after denoising into an image classification model, and obtaining a classification result corresponding to the face image to be enhanced after the classification processing of the image classification model.
8. The apparatus of claim 5, wherein the face attribute comprises at least one of age, gender, face value, and race.
9. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein instructions, which when run on a terminal device, cause the terminal device to implement the method according to any of claims 1-4.
10. An apparatus comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of claims 1-4 when the computer program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011031567.7A CN112184580B (en) | 2020-09-27 | 2020-09-27 | A method, device, equipment and storage medium for enhancing face image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011031567.7A CN112184580B (en) | 2020-09-27 | 2020-09-27 | A method, device, equipment and storage medium for enhancing face image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112184580A CN112184580A (en) | 2021-01-05 |
| CN112184580B true CN112184580B (en) | 2025-01-10 |
Family
ID=73945126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011031567.7A Active CN112184580B (en) | 2020-09-27 | 2020-09-27 | A method, device, equipment and storage medium for enhancing face image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112184580B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114418868B (en) * | 2021-12-21 | 2024-11-12 | 云南联合视觉科技有限公司 | Image enhancement method, device, server and storage medium |
| CN116797466B (en) * | 2022-03-14 | 2024-11-26 | 腾讯科技（深圳）有限公司 | Image processing method, device, equipment and readable storage medium |
| CN116363013A (en) * | 2023-04-06 | 2023-06-30 | 深圳市威富视界有限公司 | Image processing method and device |
| CN117764853B (en) * | 2024-01-11 | 2024-07-05 | 荣耀终端有限公司 | Face image enhancement method and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102027505A (en) * | 2008-07-30 | 2011-04-20 | 泰塞拉技术爱尔兰公司 | Automatic face and skin beautification using face detection |
| CN103413270A (en) * | 2013-08-15 | 2013-11-27 | 北京小米科技有限责任公司 | Method and device for image processing and terminal device |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104715227B (en) * | 2013-12-13 | 2020-04-03 | 北京三星通信技术研究有限公司 | Method and device for locating key points of face |
| US8917925B1 (en) * | 2014-03-28 | 2014-12-23 | Heartflow, Inc. | Systems and methods for data and model-driven image reconstruction and enhancement |
| CN104537630A (en) * | 2015-01-22 | 2015-04-22 | 厦门美图之家科技有限公司 | Method and device for image beautifying based on age estimation |
| US20160284381A1 (en) * | 2015-03-25 | 2016-09-29 | Cyberlink Corp. | Systems and Methods for Quick Decision Editing of Media Content |
| CN107274354A (en) * | 2017-05-22 | 2017-10-20 | 奇酷互联网络科技（深圳）有限公司 | image processing method, device and mobile terminal |
| CN107545536A (en) * | 2017-08-17 | 2018-01-05 | 上海展扬通信技术有限公司 | The image processing method and image processing system of a kind of intelligent terminal |
| CN107578372B (en) * | 2017-10-31 | 2020-02-18 | Oppo广东移动通信有限公司 | Image processing method, apparatus, computer-readable storage medium and electronic device |
| CN107729885B (en) * | 2017-11-23 | 2020-12-29 | 中电科新型智慧城市研究院有限公司 | A face enhancement method based on multiple residual learning |
| CN107993209B (en) * | 2017-11-30 | 2020-06-12 | Oppo广东移动通信有限公司 | Image processing method, apparatus, computer-readable storage medium and electronic device |
| CN108717530B (en) * | 2018-05-21 | 2021-06-25 | Oppo广东移动通信有限公司 | Image processing method, apparatus, computer-readable storage medium and electronic device |
| US10963753B2 (en) * | 2019-01-28 | 2021-03-30 | Applied Materials, Inc. | Automated image measurement for process development and optimization |
| CN111445424B (en) * | 2019-07-23 | 2023-07-18 | 广州市百果园信息技术有限公司 | Image processing method, device, equipment and medium for processing mobile terminal video |
-
2020
- 2020-09-27 CN CN202011031567.7A patent/CN112184580B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102027505A (en) * | 2008-07-30 | 2011-04-20 | 泰塞拉技术爱尔兰公司 | Automatic face and skin beautification using face detection |
| CN103413270A (en) * | 2013-08-15 | 2013-11-27 | 北京小米科技有限责任公司 | Method and device for image processing and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112184580A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112184580B (en) | A method, device, equipment and storage medium for enhancing face image | |
| Fan et al. | Half wavelet attention on M-Net+ for low-light image enhancement | |
| Gu et al. | A brief review of image denoising algorithms and beyond | |
| Zuo et al. | Learning iteration-wise generalized shrinkage–thresholding operators for blind deconvolution | |
| CN113592776B (en) | Image processing method and device, electronic device, and storage medium | |
| Liang et al. | Self-supervised low-light image enhancement using discrepant untrained network priors | |
| Feng et al. | URNet: A U-Net based residual network for image dehazing | |
| CN111292272B (en) | Image processing method, image processing apparatus, image processing medium, and electronic device | |
| Chrysos et al. | Motion deblurring of faces | |
| CN114897741B (en) | Image blind deblurring method based on depth residual Fourier transform | |
| CN111681198A (en) | A morphological attribute filtering multimode fusion imaging method, system and medium | |
| Shi et al. | Weighted median guided filtering method for single image rain removal | |
| Liu et al. | Learning noise-decoupled affine models for extreme low-light image enhancement | |
| CN110728692A (en) | Image edge detection method based on Scharr operator improvement | |
| Yu et al. | Semantic-driven face hallucination based on residual network | |
| CN111178118B (en) | Image acquisition and processing method, device and computer-readable storage medium | |
| CN113744141B (en) | Image enhancement method and device and automatic driving control method and device | |
| Liu et al. | Blind image deblurring via local maximum difference prior | |
| CN113807237A (en) | Training of in vivo detection model, in vivo detection method, computer device, and medium | |
| Li et al. | Innovative adaptive edge detection for noisy images using wavelet and Gaussian method | |
| Chen et al. | Structure-preserving image smoothing with semantic cues | |
| Zheng et al. | A new artistic information extraction method with multi channels and guided filters for calligraphy works | |
| Walha et al. | Handling noise in textual image resolution enhancement using online and offline learned dictionaries | |
| CN118982677A (en) | An improved method for high-frequency detail enhanced remote sensing image feature extraction based on Mask R-CNN | |
| Li et al. | Self-supervised normalizing flow for jointing low-light enhancement and deblurring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |