CN113569723A - A face detection method, device, electronic device and storage medium - Google Patents
A face detection method, device, electronic device and storage medium Download PDFInfo
- Publication number
- CN113569723A CN113569723A CN202110850122.XA CN202110850122A CN113569723A CN 113569723 A CN113569723 A CN 113569723A CN 202110850122 A CN202110850122 A CN 202110850122A CN 113569723 A CN113569723 A CN 113569723A
- Authority
- CN
- China
- Prior art keywords
- face detection
- detected
- low
- trained
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
本发明提供一种人脸检测方法、装置、电子设备及存储介质，对进行了图像增强处理的待检测低光照图像进行多尺度变换处理；针对每个预训练的人脸检测模型，将进行多尺度变换处理得到的多张目标待检测低光照图像输入预训练的人脸检测模型，使预训练的人脸检测模型对每张目标待检测低光照图像进行人脸检测，得到每张目标待检测低光照图像的人脸检测结果，对将每个目标待检测低光照图像的人脸检测结果进行合并的结果进行处理，得到预训练的人脸检测模型的目标人脸检测结果；对将各个预训练的人脸检测模型的目标人脸检测结果进行合并的结果进行非极大值抑制处理，得到待检测低光照图像的人脸检测结果。本发明，能够在低光照情况下，提高人脸检测精度。
The invention provides a face detection method, device, electronic equipment and storage medium, which perform multi-scale transformation processing on the low-light image to be detected that has undergone image enhancement processing; for each pre-trained face detection model, multiple The multiple low-light images of the target to be detected obtained by the scale transformation are input into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-light image of the target to be detected, and each target to be detected is obtained. The face detection result of the low-light image, the result of combining the face detection results of each target low-light image to be detected is processed, and the target face detection result of the pre-trained face detection model is obtained; The target face detection results of the trained face detection model are merged and subjected to non-maximum suppression processing to obtain the face detection result of the low-light image to be detected. The present invention can improve the accuracy of face detection under low light conditions.
Description
Technical Field
The present invention relates to the field of face detection technologies, and in particular, to a face detection method, an apparatus, an electronic device, and a storage medium.
Background
In recent years, face recognition is the most popular topic in the research of the visual field, and has important application value in the fields of security monitoring, testimony comparison, face unlocking, man-machine interaction, face special effect and the like, and the basis of face recognition is face detection.
With the development of deep learning, the face detection has made great progress, and very excellent performance can be achieved in most scenes. Outdoor scenes are susceptible to complex visual degradation in inclement weather, for example, the task of face detection in low light conditions can result in reduced visibility and loss of signal detail, leading to missed and false detections.
Disclosure of Invention
In view of this, the present invention provides a face detection method, an apparatus, an electronic device and a storage medium, so as to achieve the purpose of improving the face detection accuracy under the low light condition.
The invention discloses a face detection method in a first aspect, which comprises the following steps:
acquiring a low-illumination image to be detected, and performing image enhancement processing on the low-illumination image to be detected;
performing multi-scale transformation processing on the to-be-detected low-illumination image subjected to image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, wherein the sizes of the target to-be-detected low-illumination images are different;
aiming at each pre-trained face detection model, inputting a plurality of low-illumination images to be detected of the target into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the target to obtain a face detection result of each low-illumination image to be detected of the target, and processing a result obtained by combining the face detection results of each low-illumination image to be detected of the target to obtain a target face detection result of the pre-trained face detection model;
and carrying out non-maximum suppression processing on the result obtained by combining the target face detection results of the pre-trained face detection models to obtain the face detection result of the low-illumination image to be detected.
Optionally, the acquiring a low-illumination image to be detected and performing image enhancement processing on the low-illumination image to be detected includes:
acquiring a low-illumination image to be detected;
and performing image enhancement processing on the low-illumination image to be detected by using a multi-scale retina enhancement algorithm MSRCR with color recovery.
Optionally, the pre-trained face detection models have different backbone networks, and each pre-trained face detection model is obtained by training a historical low-illumination image to be detected, which is subjected to image enhancement processing.
Optionally, the process of training the face detection model to be trained by using the historical low-illumination image to be detected subjected to image enhancement processing to obtain the pre-trained face detection model includes:
acquiring a plurality of training data, wherein each training data is obtained by performing image enhancement processing on a historical low-illumination image to be detected by using a multi-scale retina enhancement algorithm MSRCR with color recovery and a small-face data enhancement algorithm;
and inputting the training data into a face detection model to be trained aiming at each training data, so that the face detection model to be trained carries out face detection on the training data to obtain a face detection result of the historical low-illumination image to be detected, and calculating a corresponding loss function according to the face detection result of the historical low-illumination image to be detected so as to adjust the parameters of the face detection model to be trained according to the loss function until the face detection model to be trained converges to obtain the face detection model.
Optionally, if the main network of the face detection model to be trained is a ResNet50 neural network, the ResNet50 neural network includes an input layer, 3 stages, convolutional layers connected to each stage, and a context module connected to each convolutional layer, and the process of obtaining the pre-trained face detection model by training the face detection model to be trained by using the historical low-illumination image to be detected, which is subjected to image enhancement, includes:
inputting the training data into a ResNet50 neural network to be trained through the input layer;
processing the training data by using the 3 stages respectively to obtain a characteristic diagram corresponding to each stage;
respectively utilizing the convolution layers corresponding to each stage to carry out reduction processing on the feature graph corresponding to each stage to obtain a target feature graph corresponding to each convolution layer, and inputting a fusion result obtained by carrying out multi-scale fusion processing on each target feature graph into the context module corresponding to each convolution layer;
and calculating a multitask loss function of each context module based on a feature map obtained by processing an input fusion result by each context module, and adjusting parameters of the ResNet50 neural network to be trained by using the multitask loss function of each context module until the ResNet50 neural network to be trained converges to obtain a face detection model.
Optionally, the face detection result at least includes at least one face frame and a confidence corresponding to each face frame, and the method further includes:
judging whether the confidence corresponding to each face frame is larger than a preset confidence threshold;
and if the confidence corresponding to each face frame is greater than the preset confidence threshold, determining the face detection result as the final face detection result of the low-illumination image to be detected.
A second aspect of the present invention discloses a face detection apparatus, comprising:
the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for acquiring a low-illumination image to be detected and carrying out image enhancement processing on the low-illumination image to be detected;
the multi-scale transformation processing unit is used for carrying out multi-scale transformation processing on the to-be-detected low-illumination image subjected to image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, wherein the sizes of the target to-be-detected low-illumination images are different;
a face detection unit, configured to input a plurality of low-illumination images to be detected of the target into each pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the target, so as to obtain a face detection result of each low-illumination image to be detected of the target, and process a result obtained by combining the face detection results of each low-illumination image to be detected of the target, so as to obtain a target face detection result of the pre-trained face detection model;
and the non-maximum suppression processing unit is used for performing non-maximum suppression processing on a result obtained by combining the target face detection results of the pre-trained face detection models to obtain a face detection result of the low-illumination image to be detected.
Optionally, the first obtaining unit includes:
the second acquisition unit is used for acquiring a low-illumination image to be detected;
and the image enhancement processing unit is used for carrying out image enhancement processing on the low-illumination image to be detected by utilizing a multi-scale retina enhancement algorithm MSRCR with color recovery.
A third aspect of the present invention discloses an electronic apparatus comprising: the processor and the memory are connected through a communication bus; the processor is used for calling and executing the program stored in the memory; a memory for storing a program for implementing the face detection method as disclosed in the first aspect of the invention.
In a fourth aspect of the present invention, a computer-readable storage medium is disclosed, in which computer-executable instructions are stored, and the computer-executable instructions are configured to execute the face detection method disclosed in the first aspect of the present invention.
The invention provides a face detection method, a face detection device, electronic equipment and a storage medium, wherein a low-illumination image to be detected is obtained, and image enhancement processing is carried out on the low-illumination image to be detected; carrying out multi-scale transformation processing on the to-be-detected low-illumination image subjected to image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, wherein the sizes of the target to-be-detected low-illumination images are different; aiming at each pre-trained face detection model, inputting a plurality of low-illumination images to be detected of the target into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the target to obtain a face detection result of each low-illumination image to be detected of the target, and processing a result obtained by combining the face detection results of each low-illumination image to be detected of the target to obtain a target face detection result of the pre-trained face detection model; and carrying out non-maximum suppression processing on the target face detection result of each pre-trained face detection model to obtain the face detection result of the low-illumination image to be detected. According to the technical scheme, the human face detection model is obtained by training the historical low-illumination image to be detected after image enhancement, the model can better learn the characteristics of the human face in a low-illumination outdoor scene, the performance of the model in the scene is greatly improved, then after the low-illumination image to be detected is subjected to image enhancement and multi-scale transformation to obtain a plurality of low-illumination images to be detected of targets with different sizes, the plurality of low-illumination images to be detected of targets with different sizes are input, the human face detection model is obtained by training the historical low-illumination image to be detected after the image enhancement is performed on each low-illumination image to be detected, and the human face detection precision can be improved under the low-illumination condition.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic flow chart of a face detection method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a ResNet50 neural network according to an embodiment of the present invention;
fig. 3 is an exemplary diagram of a face detection method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a face detection apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules, or units, and are not used for limiting the order or interdependence of the functions performed by the devices, modules, or units.
It is noted that references to "a", "an", and "the" modifications in the disclosure are exemplary rather than limiting, and that those skilled in the art will understand that "one or more" unless the context clearly dictates otherwise.
Referring to fig. 1, a schematic flow diagram of a face detection method provided by an embodiment of the present invention is shown, where the face detection method specifically includes the following steps:
s101: and acquiring a low-illumination image to be detected, and performing image enhancement processing on the low-illumination image to be detected.
In the specific process of executing step S101, a low-light image to be detected is obtained, and image enhancement processing is performed on the obtained low-light image to be detected by using a Multi-Scale retina enhancement algorithm (MSRCR) with Color recovery.
The MSRCR is an image enhancement method widely applied based on Retinex theory, and Retinex is established on the basis of scientific experiments and scientific analysis and can adaptively enhance various different types of images. In the practical process of the application, a parameter suitable for a low-illumination condition can be set so as to achieve a better image enhancement effect.
S102: and performing multi-scale transformation processing on the to-be-detected low-illumination image subjected to image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, wherein the sizes of the target to-be-detected low-illumination images are different.
In the specific execution process of step S102, after the low-illumination image to be detected is subjected to image enhancement processing, a multi-level multi-scale fusion strategy is used to perform multi-scale transformation processing on the low-illumination image to be detected subjected to image enhancement processing, so as to obtain a plurality of target low-illumination images to be detected of different sizes.
S103: and aiming at each pre-trained face detection model, inputting a plurality of low-illumination images to be detected of targets into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the targets to obtain a face detection result of each low-illumination image to be detected of the targets, and processing a result obtained by combining the face detection results of each low-illumination image to be detected of the targets to obtain a target face detection result of the pre-trained face detection model.
In the process of step S103, a plurality of face detection models are trained in advance, each face detection model is obtained by training a historical low-light image to be detected, which is subjected to image enhancement processing, wherein the backbone network of each pre-trained face detection model is different.
Specifically, the process of training the historical low-illumination image to be detected for image enhancement processing to obtain the face detection model includes: acquiring a plurality of historical low-illumination images to be detected, and performing image enhancement processing on each historical low-illumination image to be detected by using a multi-scale retina enhancement algorithm MSRCR with color recovery and a small face data enhancement algorithm to obtain training data corresponding to each historical low-illumination image to be detected.
And aiming at each training data, inputting the training data into the face detection model to be trained so that the face detection model to be trained performs face detection on the training data to obtain a face detection result of the historical low-illumination image to be detected, and calculating a corresponding loss function according to the face detection result of the historical low-illumination image to be detected so as to adjust parameters of the face detection model to be trained according to the loss function until the face detection model to be trained converges to obtain the face detection model.
In the embodiment of the present application, a specific process of processing the historical low-illumination image to be detected by using the small-face data enhancement algorithm may be as follows: randomly selecting a face in the history low-illumination image to be detected, recording the size of the selected face as s, selecting the value with the minimum absolute value of the difference value of s from {16, 32,64,128,256,512}, and recording the value as a1Index is ind1Then from [1, ind ]1]Randomly selects an index ind2Obtaining the value a corresponding to the index2Calculating scale ═ a2(s) zooming in and out the image on a scale basisThen, the scaled image is randomly cropped and filled to obtain an image with a size of 640 x 640. After the image is subjected to the first data enhancement, we need to perform a cropping operation again, specifically, randomly select a value from {0.3,0.45,0.6,0.8,1.0}, for example, select 0.6, then calculate to obtain 0.6 × 640, crop a block 384 × 384 size from the 640 × 640 image, and include the image block of the face, and then re-enlarge the image block to 640 × 640 size.
In the embodiment of the present application, the backbone network of the face detection model may be a ResNet50 neural network, and the ResNet50 neural network includes an input layer, 3 stages, convolutional layers connected to each stage, and a context module connected to each convolutional layer, as shown in fig. 2.
If the main network of the face detection model to be trained is a ResNet50 neural network, the process of obtaining the pre-trained face detection model by training the face detection model to be trained by using the historical low-illumination image to be detected, which is subjected to image enhancement, may be as follows: acquiring training data with the size of 640 x 3, inputting the training data into a ResNet50 neural network to be trained through an input layer of the ResNet50 neural network to be trained, and processing the input training data by using 3 different stages respectively to obtain the feature maps P2 of the second stage of the first stage of the feature maps P1 and 40 x 1024 with the size of 80 x 256 and the feature map P3 of the third stage of the size of 20 x 2048.
Reducing the number of channels of the feature map P1 to 256 by using the convolution layer corresponding to the first stage to obtain a target feature map P1 ' with the size of 80 × 256, reducing the number of channels of the feature map P2 to 256 by using the convolution layer corresponding to the second stage to obtain a target feature map P2 ' with the size of 40 × 256, and reducing the number of channels of the feature map P3 to 256 by using the convolution layer corresponding to the third stage to obtain a target feature map P3 ' with the size of 20 × 256; fusing the target feature map P3 'with the empty target feature map to obtain a fused result C3, namely directly taking the target feature map P3' as the fused result C3, inputting a context module corresponding to the convolutional layer corresponding to the third stage so that the upper text module and the lower text module can be processed as the fused result C3, and outputting the feature map with the original size; fusing the target feature map P2' with the fusion result C3 to obtain a fusion result C2, inputting the fusion result C2 into a context module corresponding to the convolution layer corresponding to the second stage, so that the upper text module and the lower text module are used for processing the fusion result C2, and outputting a feature map with the original size; and fusing the target feature map P1' with the fusion result C2 to obtain a fusion result C1, inputting the fusion result C1 into a context module corresponding to the convolution layer corresponding to the first stage, so that the upper text module and the lower text module are used for processing the fusion result C1, and outputting the feature map with the original size. The context module is composed of a plurality of convolution layers with convolution kernel sizes larger than 5 x 5 and 7 x 7, and can achieve the purpose of introducing more context information by enlarging the receptive field on the basis of ensuring the size of the output feature graph unchanged.
And aiming at the feature diagram which is output by each context module and keeps the original size, calculating the position loss and the classification loss of the regression frame of the context module according to the feature diagram which is output by the context module and keeps the original size, and further calculating the multitask loss function of the context module according to the position loss and the classification loss of the regression frame of the context module. Taking C3 as an example, the feature size of C3 is 20 × 256 after passing through the corresponding context module, and in order to calculate the position loss of the regression frame of the context module, the feature size needs to be changed to 20 × 4 by the detection head (a convolution operation of 1 × 1), wherein the channel dimension is 4, and represents the position information of the regression frame; similarly, in order to calculate the regression frame classification loss, it is necessary to change the feature size to 20 × 2 through a classification header (a convolution operation of 1 × 1), where the channel dimension is 2, and represents the classification information of the regression frame (whether the face is a human face or not), and finally add the calculated regression frame position loss function to the regression frame classification loss function to obtain the multitask loss function of the context module, as shown in formula (1).
After the multitask loss function of each context module is calculated, the parameters of the ResNet50 neural network to be trained of the multitask loss function of each context module are used for adjusting until the ResNet50 neural network to be trained converges, and a face detection model is obtained.
Where i represents the index of the regression box, PiProbability, P, representing whether the regression box is a facei *The label 1 that is the regression box represents a face, and 0 represents not a face. t is tiIs the positional information of the regression frame, ti *Is the position information of the face regression box. L isclsThe softmax loss function of two types (human face and background) is used, the smoothL1 loss function is used for Lreg, and Pi *Multiplication ensures that only the position loss function is calculated for the face regression box, and lambda is used for balancing the regression box position loss function and the regression box classification loss function.
Further, in this embodiment of the application, after the face detection model is trained, the trained face detection model can be further tested, specifically: acquiring a plurality of historical low-illumination images to be detected, respectively carrying out image enhancement processing on each historical low-illumination image to be detected by utilizing a multi-scale retina enhancement algorithm MSRCR with color recovery, and carrying out multi-scale transformation processing on the low-illumination images to be detected subjected to the image enhancement processing to obtain test data of each historical low-illumination image to be detected, wherein the test data of each historical low-illumination image to be detected comprises a plurality of historical low-illumination images to be detected subjected to the image enhancement processing and in different sizes.
Aiming at each test data, a face detection model pre-trained by the test data is used, so that the pre-trained face detection model processes each historical low-illumination image to be detected which is subjected to image enhancement processing and has different sizes, the face detection result of each historical low-illumination image to be detected which is subjected to image enhancement processing and has different sizes is obtained, and the combined result of the face detection results is subjected to non-maximum suppression processing to obtain the face detection result of the historical low-illumination image to be detected corresponding to the test data and the face detection result of the historical low-illumination image to be detected corresponding to the test data, actual face detection calculation of historical low-illumination images to be detected corresponding to the test data, calculating Average value (mAP) of AP of each category, and when the mAP of the face detection model reaches a preset mAP, determining that the performance of the face detection model reaches the standard.
Among them, AP is the most commonly used evaluation index in target detection, and mean Average Precision (mapp) is the Average value of each class of AP, because there is only one class, i.e. face, in the face detection task. AP is the area under the PR curve (Precision-Recall curve), the larger the area, the higher the AP, i.e. the better the model performance, with the range of AP being 0-1. The predetermined mAP may be 0.6944, and may be set according to practical applications, and the embodiment of the present application is not limited.
In the specific execution process of step S103, a plurality of target to-be-detected low-illumination images are input into each pre-trained face detection model, and for each pre-trained face detection model, the pre-trained face detection model performs face detection on each target to-be-detected low-illumination image to obtain a face detection result of each target to-be-detected low-illumination image, merges the face detection results of each target to-be-detected low-illumination image, and performs non-maximum suppression processing on the merged result to obtain a target face detection result of the pre-trained face detection model.
S104: and carrying out non-maximum suppression processing on the result obtained by combining the target face detection results of the pre-trained face detection models to obtain the face detection result of the low-illumination image to be detected.
In the specific process of executing step S104, after the target face detection result of each pre-trained face detection module is obtained, the target face detection results of each pre-trained face detection module are merged, and the merged result is subjected to non-maximum suppression processing to obtain the face detection result of the low-illumination image to be detected.
Further, in the embodiment of the application, the face detection result of the low-illumination image to be detected includes at least one face frame, the position information of each face frame, and the confidence level of the face frame, so that whether the confidence level corresponding to each face frame is greater than a preset confidence level threshold value can be judged arbitrarily; and if the confidence corresponding to each face frame is greater than a preset confidence threshold, determining the face detection result as the final face detection result of the low-illumination image to be detected. And if the confidence corresponding to a certain face frame is not greater than the preset confidence threshold, carrying out face detection on the low-illumination image to be detected again.
The invention provides a face detection method, which comprises the steps of obtaining a low-illumination image to be detected, and carrying out image enhancement processing on the low-illumination image to be detected; carrying out multi-scale transformation processing on the to-be-detected low-illumination image subjected to image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, wherein the sizes of the target to-be-detected low-illumination images are different; aiming at each pre-trained face detection model, inputting a plurality of low-illumination images to be detected of targets into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the targets to obtain a face detection result of each low-illumination image to be detected of the targets, and processing a result obtained by combining the face detection results of each low-illumination image to be detected of the targets to obtain a target face detection result of the pre-trained face detection model; and carrying out non-maximum suppression processing on the target face detection result of each pre-trained face detection model to obtain the face detection result of the low-illumination image to be detected. According to the technical scheme, the human face detection model is obtained by training the historical low-illumination image to be detected after image enhancement, the model can better learn the characteristics of the human face in a low-illumination outdoor scene, the performance of the model in the scene is greatly improved, then after the low-illumination image to be detected is subjected to image enhancement and multi-scale transformation to obtain a plurality of low-illumination images to be detected of targets with different sizes, the plurality of low-illumination images to be detected of targets with different sizes are input, the human face detection model is obtained by training the historical low-illumination image to be detected after the image enhancement is performed on each low-illumination image to be detected, and the human face detection precision can be improved under the low-illumination condition.
In order to better understand the above, the following description is given by way of example.
For example, as shown in fig. 3, a 3-person face detection model is trained in advance, a low-illumination image to be detected (such as an input image in fig. 3) is obtained, image enhancement processing is performed on the low-illumination image to be detected, and the resolution of the image subjected to image enhancement is amplified by 2 times and 3 times respectively to obtain two target low-illumination images to be detected with larger scales, namely a target low-illumination image to be detected with a size of 1 and a target low-illumination image to be detected with a size of 2.
Inputting the low-illumination image to be detected of the target with the size of 1 and the low-illumination image to be detected of the target with the size of 2 into a pre-trained face detection model I, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the target to obtain the face detection result of each low-illumination image to be detected of the target, and performing non-maximum suppression processing on the result obtained by combining the face detection results of each low-illumination image to be detected of the target to obtain the target face detection result of the pre-trained face detection model I.
Inputting the low-illumination image to be detected of the target with the size of 1 and the low-illumination image to be detected of the target with the size of 2 into a pre-trained face detection model II, so that the pre-trained face detection model carries out face detection on the low-illumination image to be detected of each target to obtain the face detection result of the low-illumination image to be detected of each target, and carrying out non-maximum suppression processing on the result obtained by combining the face detection results of the low-illumination images to be detected of each target to obtain the target face detection result of the pre-trained face detection model II.
Inputting the low-illumination image to be detected of the target with the size of 1 and the low-illumination image to be detected of the target with the size of 2 into a pre-trained face detection model III, so that the pre-trained face detection model III performs face detection on each low-illumination image to be detected of the target to obtain a face detection result of each low-illumination image to be detected of the target, and performing non-maximum suppression processing on a result obtained by combining the face detection results of each low-illumination image to be detected of the target to obtain a target face detection result of the pre-trained face detection model III.
And after the target face detection result of each pre-trained face detection module is obtained, combining the target face detection results of each pre-trained face detection module, and performing non-maximum suppression processing on the combined result to obtain the face detection result of the low-illumination image to be detected.
Based on the face detection method disclosed by the embodiment of the invention, the embodiment of the invention also correspondingly discloses a face detection device, and as shown in fig. 4, the face detection device comprises:
the first obtaining unit 41 is configured to obtain a low-illumination image to be detected, and perform image enhancement processing on the low-illumination image to be detected;
the multi-scale transformation processing unit 42 is configured to perform multi-scale transformation processing on the to-be-detected low-illumination image subjected to the image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, where the size of each target to-be-detected low-illumination image is different;
a face detection unit 43, configured to input multiple target low-illumination images to be detected into the pre-trained face detection model for each pre-trained face detection model, so that the pre-trained face detection model performs face detection on each target low-illumination image to be detected, so as to obtain a face detection result of each target low-illumination image to be detected, and process a result obtained by combining the face detection results of each target low-illumination image to be detected, so as to obtain a target face detection result of the pre-trained face detection model;
and the non-maximum suppression processing unit 44 is configured to perform non-maximum suppression processing on a result obtained by combining the target face detection results of the pre-trained face detection models, so as to obtain a face detection result of the low-illumination image to be detected.
The specific principle and the execution process of each unit in the face detection device disclosed in the embodiment of the present invention are the same as those of the face detection method disclosed in the embodiment of the present invention, and reference may be made to corresponding parts in the face detection method disclosed in the embodiment of the present invention, which are not described herein again.
The invention provides a face detection device, which is used for acquiring a low-illumination image to be detected and carrying out image enhancement processing on the low-illumination image to be detected; carrying out multi-scale transformation processing on the to-be-detected low-illumination image subjected to image enhancement processing to obtain a plurality of target to-be-detected low-illumination images, wherein the sizes of the target to-be-detected low-illumination images are different; aiming at each pre-trained face detection model, inputting a plurality of low-illumination images to be detected of targets into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each low-illumination image to be detected of the targets to obtain a face detection result of each low-illumination image to be detected of the targets, and processing a result obtained by combining the face detection results of each low-illumination image to be detected of the targets to obtain a target face detection result of the pre-trained face detection model; and carrying out non-maximum suppression processing on the target face detection result of each pre-trained face detection model to obtain the face detection result of the low-illumination image to be detected. According to the technical scheme, the human face detection model is obtained by training the historical low-illumination image to be detected after image enhancement, the model can better learn the characteristics of the human face in a low-illumination outdoor scene, the performance of the model in the scene is greatly improved, then after the low-illumination image to be detected is subjected to image enhancement and multi-scale transformation to obtain a plurality of low-illumination images to be detected of targets with different sizes, the plurality of low-illumination images to be detected of targets with different sizes are input, the human face detection model is obtained by training the historical low-illumination image to be detected after the image enhancement is performed on each low-illumination image to be detected, and the human face detection precision can be improved under the low-illumination condition.
Optionally, the first obtaining unit includes:
the second acquisition unit is used for acquiring a low-illumination image to be detected;
and the image enhancement processing unit is used for performing image enhancement processing on the low-illumination image to be detected by using a multi-scale retina enhancement algorithm MSRCR with color recovery.
Optionally, the pre-trained face detection models have different backbone networks, and each pre-trained face detection model is obtained by training with a training unit.
Optionally, the training unit includes:
the third acquisition unit is used for acquiring a plurality of training data, and each training data is obtained by performing image enhancement processing on the historical low-illumination image to be detected by using a multi-scale retina enhancement algorithm MSRCR with color recovery and a small-face data enhancement algorithm;
and the training subunit is used for inputting the training data into the to-be-trained face detection model aiming at each training data, so that the to-be-trained face detection model performs face detection on the training data to obtain a face detection result of the historical to-be-detected low-illumination image, and calculating a corresponding loss function according to the face detection result of the historical to-be-detected low-illumination image, so as to adjust the parameters of the to-be-trained face detection model according to the loss function until the to-be-trained face detection model converges, and thus obtaining the face detection model.
Optionally, if the backbone network of the face detection model to be trained is a ResNet50 neural network, the ResNet50 neural network includes an input layer, 3 stages, convolutional layers connected to each stage, and a context module connected to each convolutional layer, and the face detection unit includes:
the input unit is used for inputting training data into a ResNet50 neural network to be trained through an input layer;
the first processing unit is used for processing the training data by utilizing 3 stages respectively to obtain a characteristic diagram corresponding to each stage;
the second processing unit is used for respectively utilizing the convolution layers corresponding to each stage to carry out reduction processing on the feature graph corresponding to each stage to obtain a target feature graph corresponding to each convolution layer, and inputting a fusion result obtained by carrying out multi-scale fusion processing on each target feature graph into a context module corresponding to each convolution layer;
and the parameter adjusting unit is used for calculating the multitask loss function of each context module based on a feature map obtained by processing the input fusion result by each context module, and adjusting the parameters of the ResNet50 neural network to be trained by using the multitask loss function of each context module until the ResNet50 neural network to be trained converges to obtain the face detection model.
Further, the face detection result at least includes at least one face frame and the confidence corresponding to each face frame, and the face detection apparatus provided by the present invention further includes:
the judging unit is used for judging whether the confidence coefficient corresponding to each face frame is larger than a preset confidence coefficient threshold value or not;
and the determining unit is used for determining the face detection result as the final face detection result of the low-illumination image to be detected if the confidence coefficient corresponding to each face frame is greater than a preset confidence coefficient threshold value.
An embodiment of the present application provides an electronic device, as shown in fig. 5, the electronic device includes a processor 501 and a memory 502, the memory 502 is used for program codes and data of face detection, and the processor 501 is used for calling program instructions in the memory to execute steps shown in the method for implementing face detection in the foregoing embodiments.
The embodiment of the application provides a storage medium, the storage medium comprises a storage program, and when the program runs, a device where the storage medium is located is controlled to execute the face detection method shown in the embodiment.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are merely illustrative, wherein units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that it is obvious to those skilled in the art that various modifications and improvements can be made without departing from the principle of the present invention, and these modifications and improvements should also be considered as the protection scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110850122.XA CN113569723B (en) | 2021-07-27 | 2021-07-27 | Face detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110850122.XA CN113569723B (en) | 2021-07-27 | 2021-07-27 | Face detection method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113569723A true CN113569723A (en) | 2021-10-29 |
| CN113569723B CN113569723B (en) | 2025-12-19 |
Family
ID=78167912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110850122.XA Active CN113569723B (en) | 2021-07-27 | 2021-07-27 | Face detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113569723B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663951A (en) * | 2022-03-28 | 2022-06-24 | 深圳市赛为智能股份有限公司 | Low illumination face detection method, device, computer equipment and storage medium |
| CN115147892A (en) * | 2022-03-28 | 2022-10-04 | 深圳市赛为智能股份有限公司 | Low-light illumination face detection method, device, computer equipment and storage medium |
| CN116229435A (en) * | 2023-03-13 | 2023-06-06 | 广东工业大学 | A detection method for drowned diatoms based on multi-scale feature fusion |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622604A (en) * | 2012-02-14 | 2012-08-01 | 西安电子科技大学 | Multi-angle human face detecting method based on weighting of deformable components |
| CN108090417A (en) * | 2017-11-27 | 2018-05-29 | 上海交通大学 | A kind of method for detecting human face based on convolutional neural networks |
| CN109508654A (en) * | 2018-10-26 | 2019-03-22 | 中国地质大学（武汉） | Merge the human face analysis method and system of multitask and multiple dimensioned convolutional neural networks |
| CN110287927A (en) * | 2019-07-01 | 2019-09-27 | 西安电子科技大学 | Object detection method for remote sensing images based on deep multi-scale and context learning |
| CN112446247A (en) * | 2019-08-30 | 2021-03-05 | 北京大学 | Low-illumination face detection method based on multi-feature fusion and low-illumination face detection network |
| CN112541859A (en) * | 2019-09-23 | 2021-03-23 | 武汉科技大学 | Illumination self-adaptive face image enhancement method |
| WO2021088300A1 (en) * | 2019-11-09 | 2021-05-14 | 北京工业大学 | Rgb-d multi-mode fusion personnel detection method based on asymmetric double-stream network |
| CN113052210A (en) * | 2021-03-11 | 2021-06-29 | 北京工业大学 | Fast low-illumination target detection method based on convolutional neural network |
-
2021
- 2021-07-27 CN CN202110850122.XA patent/CN113569723B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622604A (en) * | 2012-02-14 | 2012-08-01 | 西安电子科技大学 | Multi-angle human face detecting method based on weighting of deformable components |
| CN108090417A (en) * | 2017-11-27 | 2018-05-29 | 上海交通大学 | A kind of method for detecting human face based on convolutional neural networks |
| CN109508654A (en) * | 2018-10-26 | 2019-03-22 | 中国地质大学（武汉） | Merge the human face analysis method and system of multitask and multiple dimensioned convolutional neural networks |
| CN110287927A (en) * | 2019-07-01 | 2019-09-27 | 西安电子科技大学 | Object detection method for remote sensing images based on deep multi-scale and context learning |
| CN112446247A (en) * | 2019-08-30 | 2021-03-05 | 北京大学 | Low-illumination face detection method based on multi-feature fusion and low-illumination face detection network |
| CN112541859A (en) * | 2019-09-23 | 2021-03-23 | 武汉科技大学 | Illumination self-adaptive face image enhancement method |
| WO2021088300A1 (en) * | 2019-11-09 | 2021-05-14 | 北京工业大学 | Rgb-d multi-mode fusion personnel detection method based on asymmetric double-stream network |
| CN113052210A (en) * | 2021-03-11 | 2021-06-29 | 北京工业大学 | Fast low-illumination target detection method based on convolutional neural network |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663951A (en) * | 2022-03-28 | 2022-06-24 | 深圳市赛为智能股份有限公司 | Low illumination face detection method, device, computer equipment and storage medium |
| CN115147892A (en) * | 2022-03-28 | 2022-10-04 | 深圳市赛为智能股份有限公司 | Low-light illumination face detection method, device, computer equipment and storage medium |
| CN116229435A (en) * | 2023-03-13 | 2023-06-06 | 广东工业大学 | A detection method for drowned diatoms based on multi-scale feature fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113569723B (en) | 2025-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110852383B (en) | Target detection method and device based on attention mechanism deep learning network | |
| CN108805016B (en) | Head and shoulder area detection method and device | |
| CN117611994A (en) | Remote sensing image target detection method based on weighted feature fusion of attention mechanism | |
| CN109815881A (en) | Training method, the Activity recognition method, device and equipment of Activity recognition model | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN105654067A (en) | Vehicle detection method and device | |
| CN114821647B (en) | A sleeping post identification method, device, equipment and medium | |
| CN113569723A (en) | A face detection method, device, electronic device and storage medium | |
| CN110378837A (en) | Object detection method, device and storage medium based on fish-eye camera | |
| CN111104830A (en) | Deep learning model for image recognition, training device and method of deep learning model | |
| CN111860316B (en) | Driving behavior recognition method, device and storage medium | |
| CN111539456B (en) | Target identification method and device | |
| CN113065379A (en) | Image detection method, device and electronic device for fused image quality | |
| CN114220125A (en) | Pedestrian detection method, device, equipment and medium | |
| CN115294636A (en) | Face clustering method and device based on self-attention mechanism | |
| CN112990009A (en) | End-to-end-based lane line detection method, device, equipment and storage medium | |
| CN114359892A (en) | Three-dimensional target detection method and device and computer readable storage medium | |
| CN113688810B (en) | Target capturing method and system of edge device and related device | |
| CN117893868A (en) | A reference-free image object detection method based on feature enhancement and multi-scale | |
| CN114529890B (en) | State detection method, device, electronic device and storage medium | |
| CN119339303B (en) | Video processing method, device and equipment | |
| CN112487911B (en) | Real-time pedestrian detection method and device based on improvement yolov under intelligent monitoring environment | |
| CN114863472A (en) | Multi-stage pedestrian detection method, device and storage medium | |
| CN119478351A (en) | A method for target detection based on improved YOLOv8-DES model | |
| CN115862127B (en) | Attention-YOLOv network gesture recognition method based on CBC classifier |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |