CN114092383A - ISP adaptive adjustment control method and device based on face image - Google Patents

Abstract

The invention discloses an ISP self-adaptive adjustment control method and a device based on a face image, wherein the method comprises the steps of obtaining the face image collected by image collecting equipment, and inputting the face image into a face image quality evaluation model for analysis; obtaining an analysis result of the facial image quality evaluation model as an image analysis parameter of the facial image, and judging whether an ISP (internet service provider) corresponding to the image acquisition equipment needs to be adjusted or not according to the image analysis parameter; if yes, adjusting the ISP according to the image analysis parameters until the parameters of the ISP meet preset conditions, and triggering and executing the operation of acquiring the face image acquired by the image acquisition equipment again. Therefore, the dynamic adjustment of the ISP can be realized, the ISP is always in a state of being in fit with the face application, the real-time and accurate acquisition of the face image is realized, the face application is favorably ensured to acquire relatively stable high-quality face input data, and the effectiveness and the safety of face verification (such as entrance guard face refreshing authentication and the like) are improved.

Description

ISP adaptive adjustment control method and device based on face image

technical field

The present invention relates to the technical field of image processing, and in particular, to a method and device for self-adaptive adjustment and control of ISP based on a face image.

Background technique

With the rapid development of science and technology, the development of face recognition technology has become more and more mature. Based on the advantages of fast response speed and non-contact identity authentication, face recognition technology has become more and more widely used, such as face unlocking, enterprise attendance, community access control and mobile payment.

In practical applications, the quality of the collected image directly affects the accuracy of face recognition, and the quality of the collected image depends on the adjustment of the ISP (Image Signal Processor, image signal processor) of the camera. At present, the adjustment of ISP is mainly carried out through the global information obtained by the image sensor to adapt to changes in the environment. However, practice has found that this ISP adjustment method is completely isolated from the ISP and the application that calls the camera. The application calling the camera can only passively use the raw data provided by the camera, and cannot guide the ISP to make targeted adjustments. The ISP adjustment cannot be dynamically adjusted according to the actual situation, so that the real-time and accurate image cannot be obtained.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide an ISP adaptive adjustment control method and device based on a face image, which can dynamically adjust the ISP according to the actual situation and realize real-time and accurate acquisition of the face image.

In order to solve the above technical problems, the first aspect of the present invention discloses a face image-based ISP adaptive adjustment control method, the method includes:

acquiring the face image collected by the image acquisition device, and inputting the face image into a predetermined face image quality assessment model for analysis;

Obtain the analysis result of the face image quality assessment model as an image analysis parameter of the face image, and determine whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameter;

When it is determined that the ISP needs to be adjusted, the ISP is adjusted according to the image analysis parameters until the parameters of the ISP meet the preset conditions, and the execution of the acquisition of the face image collected by the image collection device is re-triggered. operate.

As an optional implementation manner, in the first aspect of the present invention, the image analysis parameters include exposure parameters of the face image, white balance parameters of the face image, and focus parameters of the face image at least one of;

Wherein, determining whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameter includes:

When the image analysis parameter is the exposure parameter of the face image, the face image is intercepted based on the calibration results corresponding to the determined key points of the face to obtain a target face region, wherein the target face region The area of is less than the area of the face image;

Obtain the first brightness value corresponding to the target face area, and determine whether the first brightness value is greater than or equal to a preset brightness value threshold, when it is determined that the first brightness value is greater than or equal to the preset brightness value threshold , determine that the ISP corresponding to the image acquisition device needs to be adjusted; or,

When the image analysis parameter is the white balance parameter of the face image, determine the mean value of each of the three RGB channels of the target face area, and determine all the channels according to all the mean values Whether the proportional distribution of the image capture device matches the pre-determined proportional distribution, when it is determined that the result is no, it is determined that the ISP corresponding to the image capture device needs to be adjusted; or,

When the image analysis parameter is the focus parameter of the face image, obtain the target position where the target face region appears on the image acquisition device, and determine whether the target position is within a predetermined position range , when it is determined that the result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

As an optional implementation manner, in the first aspect of the present invention, the method further includes:

When it is determined that the first brightness value is not greater than or equal to the preset brightness value threshold, the effective area of the target face region is expanded according to a preset expansion method to obtain an enlarged target face region, the preset The expansion mode is a mode set based on the center position of the target face area;

determining the second brightness value of the background area corresponding to the expanded target face area, and calculating the backlight of the face image based on the first brightness value and the second brightness value;

It is determined whether the backlight intensity is less than or equal to a preset backlight intensity threshold, and when it is determined that the backlight intensity is less than or equal to the preset backlight intensity threshold, it is determined that the ISP corresponding to the image capture device needs to be adjusted.

As an optional implementation manner, in the first aspect of the present invention, before acquiring the face image collected by the image collection device, the method further includes:

Determine whether the ISP needs to be adjusted according to the current state of the ISP, and when it is determined that the ISP needs to be adjusted, trigger the execution of the operation of acquiring the face image collected by the image capture device;

Wherein, determining whether the ISP needs to be adjusted according to the current state of the ISP includes:

Read the target time currently displayed by the image processing system corresponding to the ISP, and obtain the completion time corresponding to the last adjustment of the ISP;

Calculate the interval duration between the target moment and the completion moment, and determine whether the interval duration is less than or equal to a predetermined interval duration threshold, and when it is determined that the interval duration is less than or equal to the interval duration threshold, determine The ISP needs to be adjusted.

As an optional implementation manner, in the first aspect of the present invention, after it is determined that the interval duration is less than or equal to the interval duration threshold, and before it is determined that the ISP needs to be adjusted, the method further includes:

Acquire the total adjustment times of the ISP within the first preset duration, and determine whether the total adjustment times is less than the preset adjustment times threshold, and when it is determined that the total adjustment times is less than the preset adjustment times threshold, triggering The operation of determining that the ISP needs to be adjusted is performed, wherein the cut-off time of the first preset duration is the last time the ISP was adjusted.

As an optional implementation manner, in the first aspect of the present invention, after the acquisition of the face image collected by the image acquisition device, and the input of the face image into the pre-determined face image quality assessment Before performing analysis in the model, the method further includes:

Determine the image parameters corresponding to the face image, the image parameters include at least the area of the face image, the position where the face image appears on the image acquisition device, and the facial information of the face image. One, the facial information of the face image includes the position of the left virtual pupil of the face image, the position of the right virtual pupil, the position of the left corner of the virtual mouth, the position of the right corner of the virtual mouth and the virtual nose tip s position;

Determine whether the face image satisfies the pre-determined adjustment conditions of the ISP according to the image parameters, and when it is determined that the face image meets the adjustment conditions of the ISP, trigger the execution of the The face image input is analyzed in the pre-determined face image quality assessment model.

As an optional implementation manner, in the first aspect of the present invention, the ISP is adjusted according to the image analysis parameter until the current parameter of the ISP satisfies a preset condition, and the re-triggering execution Before the operation of obtaining the face image collected by the image collection device, the method further includes:

Determine whether the real-time display time of the image processing system corresponding to the ISP has reached the cut-off time of the second preset duration, and when it is determined that the real-time display time has reached the cut-off time of the second preset duration, re-trigger the execution of the The operation of acquiring the face image collected by the image collection device, the starting moment of the second preset duration is the moment when it is determined that the ISP needs to be adjusted.

A second aspect of the present invention discloses an ISP adaptive adjustment control device based on a face image. The device includes an acquisition module, an analysis module, a first judgment module and an adjustment module, wherein:

The acquisition module is used to acquire the face image collected by the image acquisition device;

The analysis module is used to input the face image into a pre-determined face image quality assessment model for analysis;

The obtaining module is further configured to obtain the analysis result of the face image quality assessment model as an image analysis parameter of the face image;

The first judgment module is configured to judge whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameter;

The adjustment module is configured to adjust the ISP according to the image analysis parameters when the first judgment module determines that the ISP needs to be adjusted, until the parameters of the ISP meet preset conditions, and trigger the acquisition The module re-executes the operation of acquiring the face image acquired by the image acquisition device.

As an optional implementation manner, in the second aspect of the present invention, the image analysis parameters include exposure parameters of the face image, white balance parameters of the face image, and focus parameters of the face image at least one of;

Wherein, the first judging module includes a first judging sub-module or a second judging sub-module or a third judging sub-module, wherein the first judging sub-module includes an intercepting unit, an acquiring unit, a judging unit and a determining unit, wherein :

The intercepting unit is configured to intercept the face image based on the calibration result corresponding to the determined face key points when the image analysis parameter is the exposure parameter of the face image, to obtain the target face area, wherein , the area of the target face region is smaller than the area of the face image;

the obtaining unit, configured to obtain the first brightness value corresponding to the target face region;

the judging unit, configured to judge whether the first brightness value is greater than or equal to a preset brightness value threshold;

the determining unit, configured to determine that the ISP corresponding to the image capturing device needs to be adjusted when the determining unit determines that the first brightness value is greater than or equal to the preset brightness value threshold; or,

The second judgment submodule is configured to determine the mean value of each of the three RGB channels of the target face region when the image analysis parameter is the white balance parameter of the face image, and According to all the mean values, it is determined whether the proportional distribution of all the channels matches the predetermined proportional distribution, and when the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted; or,

The third judging sub-module is configured to obtain the target face region appearing at the target position of the image acquisition device when the image analysis parameter is the focus parameter of the face image, and judge the target Whether the position is within the predetermined position range, when the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

As an optional implementation manner, in the second aspect of the present invention, the acquisition unit is further configured to: when the determination unit determines that the first brightness value is not greater than or equal to the preset brightness value threshold, Enlarging the effective area of the target face area according to a preset enlargement method to obtain the enlarged target face area, and the preset enlargement method is a method set based on the center position of the target face area;

The determining unit is further configured to determine the second brightness value of the background region corresponding to the expanded target face region;

The acquiring unit is further configured to calculate the back lightness of the face image based on the first brightness value and the second brightness value;

The judging unit is configured to judge whether the backlight intensity is less than or equal to a preset backlight intensity threshold, and when judging that the backlight intensity is less than or equal to the preset backlight intensity threshold, trigger the determining unit to execute the determination process. The ISP corresponding to the image acquisition device needs to be adjusted.

As an optional implementation manner, in the second aspect of the present invention, the device further includes a second judgment module, wherein:

The second judgment module is used to judge whether the ISP needs to be adjusted according to the current state of the ISP before the acquisition module obtains the face image collected by the image collection device; when it is judged that the ISP needs to be adjusted , triggering the acquisition module to perform the operation of acquiring the face image collected by the image acquisition device;

Wherein, the second judgment module includes a reading sub-module, an acquisition sub-module, a calculation sub-module and a fourth judgment sub-module, wherein:

The reading submodule is used to read the target moment currently displayed by the image processing system corresponding to the ISP;

The obtaining submodule is used to obtain the completion time corresponding to the last adjustment of the ISP;

The calculation submodule is used to calculate the interval duration between the target moment and the completion moment;

The fourth judgment submodule is used to judge whether the interval duration is less than or equal to a predetermined interval duration threshold;

The acquisition submodule is further configured to determine that the ISP needs to be adjusted when the fourth determination submodule determines that the interval duration is less than or equal to the interval duration threshold.

As an optional implementation manner, in the second aspect of the present invention, the acquisition submodule is further configured to, after the fourth determination submodule determines that the interval duration is less than or equal to the interval duration threshold, and Before determining that the ISP needs to be adjusted, obtain the total number of times of adjustment of the ISP within a first preset duration, where the cut-off time of the first preset duration is the last time the ISP was adjusted;

The fourth judgment sub-module is further configured to judge whether the total adjustment times is less than a preset adjustment times threshold, and when it is judged that the total adjustment times is less than the preset adjustment times threshold, trigger the acquisition sub-module to execute The operation of determining that the ISP needs to be adjusted.

As an optional implementation manner, in the second aspect of the present invention, the device further includes a determination module and a third determination module, wherein:

The determining module is used for analyzing after the acquiring module acquires the facial image collected by the image acquisition device, and after the analyzing module inputs the facial image into a pre-determined facial image quality assessment model for analysis Before, determine the image parameters corresponding to the face image, the image parameters include the area of the face image, the position where the face image appears in the image acquisition device, and the facial information of the face image At least one of, the facial information of the face image includes the position of the left virtual pupil of the face image, the position of the right virtual pupil, the position of the left corner of the virtual mouth, the position of the right corner of the virtual mouth, and the position of the virtual nose tip;

The third judgment module is configured to judge whether the face image satisfies the predetermined adjustment condition of the ISP according to the image parameter, and when it is judged that the face image satisfies the adjustment condition of the ISP, Triggering the analysis module to perform the operation of inputting the face image into a predetermined face image quality assessment model for analysis.

As an optional implementation manner, in the second aspect of the present invention, the device further includes a fourth judgment module, wherein:

The fourth judging module is configured to adjust the ISP according to the image analysis parameter until the current parameter of the ISP meets a preset condition, and after triggering the acquisition module to re-trigger the execution of the ISP Before the operation of acquiring the face image collected by the image acquisition device, determine whether the real-time display time of the image processing system corresponding to the ISP has reached the cut-off time of the second preset duration, and when it is determined that the real-time display time has reached the At the end time of the second preset duration, trigger the acquisition module to re-trigger and execute the operation of acquiring the face image collected by the image acquisition device, and the start time of the second preset duration is to determine the Moments when the ISP needs to adjust.

A third aspect of the present invention discloses another ISP adaptive adjustment control device based on a face image, the device comprising:

memory in which executable program code is stored;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory to execute the face image-based ISP adaptive adjustment control method disclosed in the first aspect of the present invention.

A fourth aspect of the present invention discloses a computer storage medium, the computer storage medium stores computer instructions, and when the computer instructions are invoked, is used to perform the face image-based ISP adaptive adjustment disclosed in the first aspect of the present invention Control Method.

Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

In the embodiment of the present invention, an ISP adaptive adjustment control method and device based on a face image are disclosed. The method includes acquiring a face image collected by an image acquisition device, and inputting the face image into a predetermined person. Perform analysis in the face image quality assessment model; obtain the analysis results of the face image quality assessment model as the image analysis parameters of the face image, and determine whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters; When the ISP needs to be adjusted, adjust the ISP according to the image analysis parameters until the parameters of the ISP meet the preset conditions, and re-trigger to execute the above-mentioned operation of acquiring the face image collected by the image collection device. It can be seen that in the embodiment of the present invention, by combining the ISP of the image acquisition device with the face-related application carrying the face image quality assessment model, the collected face image is input into the face image quality assessment model for analysis, and can be analyzed according to the image analysis. Parameters, namely the image quality of the ISP, evaluate the parameter settings of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the face image will be re-acquired to perform the same ISP adjustment operation, which can realize the dynamic adjustment of the ISP and make the ISP It has been in a state that is compatible with the face application, and realizes the acquisition of real-time and accurate face images, which helps to ensure that the face application obtains relatively stable and high-quality face input data, thereby improving face verification (for example: access control swipe). face authentication, etc.) effectiveness and security; and adaptive adjustment of ISP according to the results of face imaging, without increasing the cost, make full use of the ISP self-adjustment function of the existing image acquisition equipment, and significantly improve the existing image equipment ISP imaging effect.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic flowchart of an ISP adaptive adjustment control method based on a face image disclosed in an embodiment of the present invention;

2 is a schematic flowchart of another face image-based ISP adaptive adjustment control method disclosed in an embodiment of the present invention;

3 is a schematic structural diagram of an ISP adaptive adjustment control device based on a face image disclosed in an embodiment of the present invention;

4 is a schematic structural diagram of another face image-based ISP adaptive adjustment control device disclosed in an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of another device for self-adaptive adjustment and control of ISP based on a face image disclosed in an embodiment of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The invention discloses an ISP adaptive adjustment control method and device based on a face image, which can input the collected face image by combining the ISP of an image acquisition device with a face-related application carrying a face image quality evaluation model. The face image quality evaluation model is used for analysis, which can evaluate the parameter settings of the ISP according to the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the face image will be re-acquired to perform the same ISP. The adjustment operation can realize the dynamic adjustment of the ISP, so that the ISP is always in a state in line with the face application, realize the acquisition of real-time and accurate face images, and help ensure that the face application can obtain relatively stable and high-quality faces. Input data to improve the effectiveness and security of face verification (for example, access control face recognition, etc.); and adaptively adjust ISP according to the results of face imaging, making full use of existing image acquisition without increasing costs The ISP self-adjustment function of the equipment has significantly improved the ISP imaging effect of the existing image equipment. Each of them will be described in detail below.

Example 1

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an ISP adaptive adjustment control method based on a face image disclosed in an embodiment of the present invention. Among them, the ISP adaptive adjustment control method based on the face image described in FIG. 1 is suitable for the image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built in a terminal that can collect faces equipment, wherein the terminal equipment includes user terminals and/or access control equipment, etc. As shown in Figure 1, the ISP adaptive adjustment control method based on the face image may include the following operations:

101. Acquire a face image collected by an image collection device.

In the embodiment of the present invention, the image acquisition device is provided with an ISP, and the face image may be a real-time image acquired by the image acquisition device, or an image acquired at a preset time interval (eg, 30s).

In this embodiment of the present invention, after the face image is acquired, a preprocessing operation is further performed on the face image. At this time, the face image in step 102 is a preprocessed face image, wherein the preprocessing Operations include filtering operations, image segmentation operations, and the like. In this way, after the face image is obtained, the preprocessing operation is performed on the face image first, so that a high-quality face image can be obtained, which is beneficial to improve the analysis result of the face image, and further help to improve the judgment accuracy of adjusting the ISP. .

In an optional embodiment, before step 101 is performed, the ISP adaptive adjustment control method based on the face image may further include the following operations:

Initialize the image processing system, specifically, initialize the ISP adjustment information structure of the image processing system, where the ISP adjustment information structure is used to store the variables of the ISP in the adjustment process, for example: the detected face information of the latest frame (that is, the face appears at the specific position of the sensor included in the image acquisition device), the specific adjustment time of the last ISP, the adjustment times of the ISP in the current operation of the image processing system, etc.;

After the image processing system is initialized, the execution of step 101 may be triggered.

It can be seen that, in this optional embodiment, the image processing system is initialized before acquiring the face image, which can improve the information processing capability of the image processing system.

102. Input the above-mentioned face image into a predetermined face image quality assessment model for analysis, and obtain the analysis result of the face image quality assessment model as an image analysis parameter of the face image.

In the embodiment of the present invention, the image analysis parameter (that is, the representation of the imaging quality of the sensor of the image acquisition device) includes at least one of an exposure parameter of the face image, a white balance parameter of the face image, and a focus parameter of the face image , the embodiments of the present invention are not limited. Further, the image analysis parameters may also include, but are not limited to, color correction matrix, sharpness, curve attributes, and saturation. In this way, the more content the image analysis parameters include, the better the accuracy of determining whether the ISP needs to be adjusted.

103. Determine whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters; when it is determined that the ISP needs to be adjusted, trigger the execution of step 103; when it is determined that the ISP does not need to be adjusted, this process can be ended.

104. Adjust the ISP according to the image analysis parameters until the parameters of the ISP meet the preset conditions, and trigger step 101 again.

In the embodiment of the present invention, as an optional implementation manner, determining whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters, including:

When the above-mentioned image analysis parameter is the exposure parameter of the above-mentioned face image, the face image is intercepted based on the calibration results corresponding to the determined face key points (for example: 106 face key points) to obtain the target face area, wherein the The area of the target face area is smaller than the area of the face image;

Obtain the first brightness value corresponding to the target face area, and determine whether the first brightness value is greater than or equal to a preset brightness value threshold, and when it is determined that the first brightness value is greater than or equal to the preset brightness value threshold, determine the image acquisition The ISP corresponding to the device needs to be adjusted.

In this optional implementation manner, optionally, acquiring the first brightness value corresponding to the target face region may include:

Convert the target face region from RGB representation to YcbCr representation, and calculate the mean value of its Y component as the first luminance value corresponding to the target face region.

In this optional embodiment, further, when it is determined that the first brightness value is not greater than or equal to the preset brightness value threshold, it is determined that the ISP corresponding to the image capture device does not need to be adjusted, that is, the exposure of the image capture device is normal, and this process ends .

It can be seen that this optional embodiment can realize the judgment that the ISP needs to be adjusted when the image acquisition device is overexposed by acquiring the brightness value of the face image and comparing the brightness value with the preset brightness value; Before the brightness of the face image, the face image is further intercepted by the calibration results corresponding to the key points of the face, which can improve the efficiency and accuracy of the determination of the brightness value of the face image, thereby helping to improve the judgment accuracy of whether the ISP needs to be adjusted or not. and reliability.

In the embodiment of the present invention, as another optional implementation manner, determining whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameter, including:

When the above-mentioned image analysis parameter is the white balance parameter of the above-mentioned face image, determine the mean value of each channel in the three RGB channels of the target face area, and judge whether the proportional distribution of all channels matches the predetermined value according to all the mean values. The ratio distribution of the image capture device matches the ratio distribution, and when it is determined that the result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

It can be seen that this optional embodiment can realize the judgment that the ISP needs to be adjusted by judging whether the allocation of the three RGB channels of the face image is reasonable.

In the embodiment of the present invention, as another optional implementation, determining whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameter, including:

When the above-mentioned image analysis parameter is the focus parameter of the above-mentioned face image, obtain the target face region appearing at the target position of the image acquisition device, and judge whether the target position is within the predetermined position range, and when the judgment result is no , to determine that the ISP corresponding to the image capture device needs to be adjusted.

It can be seen that, in this optional embodiment, by comparing the position where the acquired face image appears on the image acquisition device with a predetermined position range, it is possible to determine that the ISP needs to be adjusted.

In this embodiment of the present invention, when it is determined that the first brightness value is greater than or equal to the preset brightness value threshold and/or when it is determined that the proportional distribution of all channels does not match the predetermined proportional distribution and/or when it is determined that the target position When it is not within the predetermined location range, it is determined that the ISP needs to be adjusted, which can not only improve the judgment accuracy of the ISP needs to be adjusted, but also can improve the judgment method of providing a variety of ISP needs to adjust, and improve the judgment flexibility of the ISP needs to be adjusted.

In the embodiment of the present invention, adjusting the ISP according to the image analysis parameters until the parameters of the ISP meet the preset conditions may include:

When it is determined that the first brightness value is greater than or equal to the preset brightness value threshold, the exposure parameters currently set by the ISP are obtained, and the ISP is adjusted according to the exposure parameters currently set by the ISP and the exposure parameters of the face image until the exposure parameters of the ISP are less than or equal to the preset brightness value. set exposure parameters; or,

When it is judged that the ratio distribution of all channels does not match the pre-determined ratio distribution, the average value of the G channel of the target face image is fixed, and the gain and B of the R channel are adjusted based on the values of R/G and G/B respectively. The gain of the channels so that the ratios of all channels match the pre-determined ratios. or,

When it is determined that the target position is not within the predetermined position range, the ISP is adjusted according to the target position until the target position is within the predetermined position range.

It can be seen that in this optional embodiment, the ISP is adjusted according to different parameters, so that various parameters of the ISP meet the requirements of image acquisition, thereby improving the accuracy and efficiency of image acquisition.

In an optional embodiment, the face image-based ISP adaptive adjustment control method may further include the following operations:

When it is determined that the above-mentioned first brightness value is not greater than or equal to the preset brightness value threshold, the effective area of the target face region is enlarged according to a preset enlargement method to obtain an enlarged target face area, and the preset enlargement method is based on the target person The way the center position of the face area is set;

determining the second brightness value of the background area corresponding to the enlarged target face area, and calculating the back lightness of the face image based on the first brightness value and the second brightness value;

It is determined whether the backlight intensity is less than or equal to the preset backlight intensity threshold, and when it is determined that the backlight intensity is less than or equal to the preset backlight intensity threshold, it is determined that the ISP corresponding to the image capture device needs to be adjusted.

In this optional embodiment, the preset enlargement method is a method set based on the center position of the target face area, specifically: the preset enlargement method is to take the center position of the target face area as the center, and take the first several The short side of the target face area is doubled (for example: 1 times) as the first short side, and the rectangular image area formed by taking the second several times (for example: 1 times) the long side of the target face area as the first long side is used as the first short side. The enlarged target face area. For example, when the target face area is 10cm on the short side and 15cm on the long side, the enlarged target face area still takes the center point of the target face area as the center point, the short side becomes 15cm, and the long side becomes 15cm. Rectangular image area formed by 20cm.

In this optional embodiment, when the enlarged target face region (foreground image) is obtained, the target face region is removed from the face image by an image separation method to obtain a background region, wherein the image separation method may include One or more combinations of the background difference method, the frame difference method, the optical flow field method, and the RPCA method, among which, the method for removing the foreground image for the image separation method is an existing method, which will not be repeated here. . The calculation method of the second luminance value of the background area is the same as the calculation method of the target face area, and details are not repeated here.

In this optional embodiment, the specific calculation formula of the backlight degree of the face image is as follows:

In the formula, R is the backlight of the face image; Y _f is the first brightness value corresponding to the target face area; Y _b is the second brightness value of the background area.

In this optional embodiment, further, when it is determined that the backlight intensity is not less than or equal to the preset backlight intensity threshold, it is determined that the ISP corresponding to the image capture device does not need to be adjusted, that is, the exposure of the image capture device is normal, and this process ends .

It can be seen that in this optional embodiment, when it is determined that the brightness value of the face region is small, the brightness value of the background region corresponding to the enlarged face region is calculated, and the backlight of the face image is obtained based on the two brightness values. When it is determined that the backlight degree of the face image is small, it is determined that the ISP needs to be adjusted, so as to improve the judgment that the ISP needs to be adjusted in the case of insufficient exposure of the image acquisition device, which can improve the accuracy of the determination of the ISP needs to be adjusted, which is beneficial to The all-round dynamic adjustment of the ISP is beneficial to the accuracy of the subsequent acquisition of dynamic images.

In yet another optional embodiment, the face image-based ISP adaptive adjustment control method may further include the following operations:

When it is determined that the above-mentioned backlight is not less than or equal to the preset backlight threshold, it is determined whether the first brightness value is greater than or equal to a certain preset brightness value threshold, and the certain preset brightness value threshold is smaller than the above-mentioned preset brightness value threshold;

When it is determined that the first brightness value is not greater than or equal to a certain preset brightness value threshold, it is determined that the ISP corresponding to the image capturing device needs to be adjusted.

In this optional embodiment, when it is determined that the first brightness value is greater than or equal to a certain preset brightness value threshold, it is determined that the ISP corresponding to the image capture device does not need to be adjusted, that is, the exposure of the image capture device is normal.

It can be seen that in this optional embodiment, when it is judged that the backlight degree of the face image is large, it continues to judge whether the brightness value of the face region is large, and when it is judged that it is small, it is determined that the ISP needs to be adjusted, thereby improving the exposure of the image acquisition device. Insufficient determination accuracy, that is, to improve the judgment accuracy of the ISP needs to be adjusted, thereby improving the accuracy of the all-round ISP dynamic adjustment.

In yet another optional embodiment, before step 101 is performed, the ISP adaptive adjustment control method based on the face image may further include the following operations:

Determine whether the ISP needs to be adjusted according to the current state of the ISP, and when it is determined that the ISP needs to be adjusted, trigger the execution of step 101;

In this optional embodiment, optionally, determining whether the ISP needs to be adjusted according to the current state of the ISP, including:

Read the target time currently displayed by the image processing system corresponding to the ISP, and obtain the completion time corresponding to the last adjustment of the ISP;

Calculate the interval between the target time and the completion time, and determine whether the interval is less than or equal to the predetermined interval threshold. When it is determined that the interval is less than or equal to the interval threshold, it is determined that the ISP needs to be adjusted.

In this optional embodiment, when it is determined that the interval duration is not less than or equal to the interval duration threshold, it is determined that the ISP does not need to be adjusted, and the total number of ISP adjustments of the image processing system in the current operation process up to the target time is set to 0, so that when it is determined that the ISP needs to be adjusted in the current system operation, the calculation of the number of invalid adjustments is reduced, thereby reducing the erroneous judgment that the ISP needs to be adjusted.

In this optional embodiment, the completion time corresponding to the last adjustment of the ISP is obtained from the adjustment information structure of the ISP.

It should be noted that, this optional embodiment may also occur at the same time as step 101, or after step 101, preferably before step 101.

It can be seen that this optional embodiment determines whether the ISP needs to be adjusted according to the current state of the ISP before acquiring the face image. When the various indicators of the image capture device (for example: white balance, etc.) match the current environment, the ISP is adjusted. The situation of the face image occurs and the power consumption of the system is reduced, and the service life of the system is improved.

In yet another optional embodiment, after judging that the interval duration is less than or equal to the interval duration threshold, and before determining that the ISP needs to be adjusted, the face image-based ISP adaptive adjustment control method may further include the following operations:

Obtain the total adjustment times of the ISP within the first preset time period, and determine whether the total adjustment times are less than the preset adjustment times threshold, and when it is determined that the total adjustment times are less than the preset adjustment times threshold, trigger the execution of the above-mentioned determination that the ISP needs to be adjusted operation, wherein the cut-off time of the first preset duration is the time when the ISP was last adjusted.

In this optional embodiment, when it is determined that the total number of adjustments is not less than the preset threshold of the number of adjustments, it is determined that the ISP does not need to be adjusted, and this process is basically the same.

It can be seen that this optional embodiment determines that the ISP needs to be adjusted only when it is judged that the time to adjust the ISP last time is short, and further when it is judged that the total number of times of adjustment of the ISP during the current operation of the image processing system is small. Improve the judgment accuracy of ISP needs to be adjusted; and limit the total number of ISP adjustments in the system operation process, which can meet the performance requirements of ISP and adapt to complex environments, thereby improving user experience.

In yet another optional embodiment, after the ISP is adjusted according to the above-mentioned image analysis parameters until the current parameter of the ISP satisfies the preset condition, and before the execution of step 101 is re-triggered, the face image-based ISP adaptive adjustment control Methods can also include the following actions:

It is judged whether the real-time display time of the image processing system corresponding to the ISP has reached the cut-off time of the second preset duration. The starting time of the duration is set as the time when the ISP needs to be adjusted.

In this optional embodiment, when it is determined that the real-time display time has not reached the cut-off time of the second preset duration, the execution of the above-mentioned judgment as to whether the real-time display time of the image processing system corresponding to the ISP has reached the cut-off time of the second preset duration is triggered. operation at the moment. In this way, by monitoring the real-time display time of the image processing system, the triggering accuracy of re-executing the operation of acquiring the face image can be improved.

It can be seen that in this optional embodiment, after the ISP adjustment is completed, it is first determined whether the current time has reached the preset time, and if it is reached, the operation of acquiring the face image is re-executed, which can meet the requirements of the ISP parameters from writing to the register to taking effect. Time requirements, thereby reducing the occurrence of subsequent direct reading of images that may damage the stability of the image processing system, thereby ensuring the stability of the image processing system.

It can be seen that the ISP adaptive adjustment control method based on the face image described in FIG. 1 is implemented by combining the ISP of the image acquisition device with the face-related application carrying the face image quality assessment model, and the collected face image is input. The face image quality evaluation model is used for analysis, which can evaluate the parameter settings of the ISP according to the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the face image will be re-acquired to perform the same ISP. The adjustment operation can realize the dynamic adjustment of the ISP, so that the ISP is always in a state in line with the face application, realize the acquisition of real-time and accurate face images, and help ensure that the face application can obtain relatively stable and high-quality faces. Input data to improve the effectiveness and security of face verification (for example, access control face recognition, etc.); and adaptively adjust ISP according to the results of face imaging, making full use of existing image acquisition without increasing costs The ISP self-adjustment function of the equipment has significantly improved the ISP imaging effect of the existing image equipment.

Embodiment 2

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of another face image-based ISP adaptive adjustment control method disclosed in an embodiment of the present invention. Among them, the ISP adaptive adjustment control method based on the face image described in FIG. 2 is suitable for the image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built in a terminal that can collect faces equipment, wherein the terminal equipment includes user terminals and/or access control equipment, etc. As shown in FIG. 2, the ISP adaptive adjustment control method based on the face image may include the following operations:

201. Acquire a face image collected by an image collection device.

202. Determine the image parameters corresponding to the above-mentioned face image.

In the embodiment of the present invention, the image parameter includes at least one of the area of the face image, the position where the face image appears on the image acquisition device, and the face information of the face image, and the face information of the face image includes the face image The position of the left virtual pupil, the position of the right virtual pupil, the position of the left mouth corner of the virtual mouth, the position of the right mouth corner of the virtual mouth, and the position of the virtual nose tip.

203. Determine whether the above-mentioned face image satisfies the pre-determined adjustment conditions of the ISP according to the above-mentioned image parameters; when it is judged that the face image meets the adjustment conditions of the ISP, trigger the execution of step 204; when it is judged that the face image does not meet the adjustment conditions of the ISP; When the adjustment condition of the ISP is met, the execution of step 201 may be triggered, or the current process may be ended.

It can be seen that, in the embodiment of the present invention, after the face image is acquired, it is further judged whether the face image satisfies the adjustment conditions of the ISP, and only when it is judged that the adjustment conditions of the ISP are met, the subsequent operations are performed, which can reduce the problem that the face image is inconsistent. If the ISP adjustment conditions are met but the ISP adjustment conditions are used, the ISP adjustment error occurs, which is conducive to improving the accuracy of the ISP dynamic adjustment and reducing the power consumption of the image processing system; when it is judged that the ISP adjustment conditions are not met , and re-trigger the operation of acquiring the face image, which is conducive to acquiring an image that meets the adjustment conditions of the ISP, which in turn is conducive to the dynamic adjustment of the ISP.

204, input the above-mentioned face image into the pre-determined face image quality assessment model for analysis, and obtain the analysis result of the face image quality assessment model, as the image analysis parameter of the face image;

205. Determine whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters; when it is determined that the ISP needs to be adjusted, trigger the execution of step 206; when it is determined that the ISP does not need to be adjusted, this process can be ended.

206. Adjust the ISP according to the image analysis parameters until the parameters of the ISP meet the preset conditions, and trigger step 201 again.

In the embodiment of the present invention, as an optional implementation manner, determining whether the face image satisfies the pre-determined ISP adjustment conditions according to the image parameters includes:

When the above-mentioned image parameter is the area of the above-mentioned face image, it is judged whether the area of the face image is greater than or equal to the predetermined area range, and when the judgment result is yes, it is determined that the face image satisfies the predetermined adjustment of the ISP condition.

It can be seen that in this optional embodiment, by calculating the area of the acquired face image and comparing the area with the preset area range, it is possible to determine that the face image satisfies the adjustment conditions of the ISP.

In the embodiment of the present invention, as another optional implementation manner, determining whether the face image satisfies the pre-determined ISP adjustment conditions according to the image parameters includes:

When the image parameter is the first position of the face image, the second position of the face image collected last time is acquired, the position offset between the first position and the second position is calculated, and the position offset is determined Whether it is less than or equal to the preset position offset threshold, when the judgment result is yes, it is determined that the face image satisfies the pre-determined adjustment condition of the ISP.

This optional implementation, optionally, calculating the position offset between the first position and the second position may include:

The intersection ratio of the first position and the second position is calculated as the position offset of the first position and the second position.

It can be seen that, in this optional embodiment, by calculating the position offset between the position of the face image collected this time on the image capture device and the position of the face image collected last time on the image capture device, the face image can be realized. Judging that the image satisfies the adjustment conditions of the ISP; and by calculating the intersection ratio of the positions of the two face images, the calculation accuracy and efficiency of the position offset between the positions of the two face images can be improved.

In the embodiment of the present invention, as another optional implementation, determining whether the face image satisfies the pre-determined ISP adjustment conditions according to the image parameters includes:

When the above-mentioned image parameter is the facial information of the above-mentioned face image, calculate the mean value of the pupil abscissa between the position of the above-mentioned left virtual pupil and the position of the above-mentioned right virtual pupil, and calculate the position of the above-mentioned left corner of the mouth and the above-mentioned right corner of the mouth. The mean value of the abscissa of the mouth corner of the position;

Calculate the first abscissa difference of the above-mentioned pupil abscissa mean and the abscissa of the position of the above-mentioned virtual nose tip, and calculate the second abscissa difference of the above-mentioned mouth corner abscissa mean and the abscissa of the position of the above-mentioned virtual nose;

Calculate the target abscissa difference between the first abscissa difference and the second abscissa difference, determine whether the target abscissa difference is less than or equal to the preset abscissa difference range, and when the judgment result is yes, determine The above-mentioned face image satisfies the predetermined adjustment condition of the above-mentioned ISP.

In this optional implementation manner, the target calibration result that matches the face image is determined from the set of pre-determined calibration results of the face key points, and the face information of the face image is obtained, wherein the face key point's The calibration result set includes a plurality of calibration results, for example: the calibration result including 5 key points, the calibration result including 68 key points, and the calibration result including 106 key points. Among them, the present invention selects the calibration results of five key points as the target calibration results.

It can be seen that this optional embodiment can realize the symmetrical judgment of the virtual facial features of the collected face image by calculating the positions of the left and right pupils, the left and right corners of the mouth and the tip of the nose in the face image, thereby realizing the judgment that the face image satisfies the adjustment conditions of the ISP And improve the accuracy of the judgment that the face image meets the adjustment conditions of the ISP.

In the embodiment of the present invention, it should be noted that when two or three of the above three judgment methods for determining that the face image satisfies the adjustment conditions of the ISP are positive, it is determined that the face image meets the adjustment conditions of the ISP. It can improve the judgment accuracy that the face image meets the adjustment conditions of the ISP, which is beneficial to improve the adjustment accuracy of the subsequent ISP; by providing a variety of judgment methods for the face image to meet the adjustment conditions of the ISP, it can improve the face image to meet the adjustment of the ISP. Judgmental flexibility and possibility of conditions.

In another optional embodiment, the face image-based ISP adaptive adjustment control method may further include the following operations:

When it is judged that the face image does not meet the adjustment conditions of the ISP, analyze the factors that the face image does not meet the adjustment conditions of the ISP according to the image parameters of the face image, and send the factors to the image acquisition device to trigger the image acquisition device according to the This factor recaptures face images. Among them, the factors include incomplete facial features of the face image, low brightness of the environment, and the like.

It can be seen that this optional embodiment further generates an image acquisition prompt after determining that the face image does not meet the adjustment conditions of the ISP, and will help to improve the accuracy of acquiring the face image that meets the adjustment conditions of the ISP, thereby Improve the accuracy of dynamic adjustment of ISP.

In this embodiment of the present invention, for related descriptions of step 201 and step 204 to step 206, please refer to the detailed description of step 101 to step 104 in Embodiment 1, which is not repeated in this embodiment of the present invention.

It can be seen that the ISP adaptive adjustment control method based on the face image described in FIG. 2 is implemented by combining the ISP of the image acquisition device with the face-related application carrying the face image quality assessment model, and the collected face image is input. The face image quality evaluation model is used for analysis, which can evaluate the parameter settings of the ISP according to the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the face image will be re-acquired to perform the same ISP. The adjustment operation can realize the dynamic adjustment of the ISP, so that the ISP is always in a state in line with the face application, realize the acquisition of real-time and accurate face images, and help ensure that the face application can obtain relatively stable and high-quality faces. Input data to improve the effectiveness and security of face verification (for example, access control face recognition, etc.); and adaptively adjust ISP according to the results of face imaging, making full use of existing image acquisition without increasing costs The ISP self-adjustment function of the equipment has significantly improved the ISP imaging effect of the existing image equipment. In addition, it can also reduce the occurrence of errors in ISP adjustment due to the fact that the face image does not meet the adjustment conditions of the ISP but uses it to adjust the ISP, which is beneficial to improve the accuracy of the dynamic adjustment of the ISP and reduce the power consumption of the image processing system; It is also beneficial to obtain images that meet the adjustment conditions of the ISP, which is conducive to the dynamic adjustment of the ISP; it can also provide a variety of judgment methods for the face image to meet the adjustment conditions of the ISP, which improves the judgment of the face image meeting the adjustment conditions of the ISP. The accuracy and flexibility are beneficial to improve the adjustment accuracy of the subsequent ISP.

Embodiment 3

Please refer to FIG. 3 . FIG. 3 is a schematic structural diagram of an ISP adaptive adjustment control device based on a face image disclosed in an embodiment of the present invention. The ISP adaptive adjustment and control device based on the face image described in FIG. 3 is suitable for use in an image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built in a terminal that can collect faces equipment, wherein the terminal equipment includes user terminals and/or access control equipment, etc. As shown in FIG. 3 , the ISP adaptive adjustment control device based on the face image may include an acquisition module 301, an analysis module 302, a first judgment module 303 and an adjustment module 304, wherein:

The acquiring module 301 is configured to acquire a face image acquired by an image acquisition device.

The analysis module 302 is configured to input the above-mentioned face image into a pre-determined face image quality assessment model for analysis.

The obtaining module 301 is further configured to obtain the analysis result of the above-mentioned face image quality assessment model as the image analysis parameter of the above-mentioned face image;

The first judging module 303 is configured to judge whether the ISP corresponding to the above-mentioned image acquisition device needs to be adjusted according to the above-mentioned image analysis parameters.

The adjustment module 304 is used to adjust the ISP according to the above-mentioned image analysis parameters when the first judgment module 303 judges that the above-mentioned ISP needs to be adjusted, until the parameters of the ISP meet the preset conditions, and trigger the acquisition module 301 to re-execute the above-mentioned acquisition of images The operation of collecting the face image collected by the device.

It can be seen that the ISP adaptive adjustment control device based on the face image described in FIG. 3 combines the ISP of the image acquisition device with the face-related application carrying the face image quality assessment model, and the collected face image is input. The face image quality evaluation model is used for analysis, which can evaluate the parameter settings of the ISP according to the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the face image will be re-acquired to perform the same ISP. The adjustment operation can realize the dynamic adjustment of the ISP, so that the ISP is always in a state in line with the face application, realize the acquisition of real-time and accurate face images, and help ensure that the face application can obtain relatively stable and high-quality faces. Input data to improve the effectiveness and security of face verification (for example, access control face recognition, etc.); and adaptively adjust ISP according to the results of face imaging, making full use of existing image acquisition without increasing costs The ISP self-adjustment function of the equipment has significantly improved the ISP imaging effect of the existing image equipment.

In an optional embodiment, the above-mentioned image analysis parameters include at least one of exposure parameters of the face image, white balance parameters of the face image, and focus parameters of the face image;

The first judging module 303 may include a first judging sub-module or a second judging sub-module or a third judging sub-module, wherein the first judging sub-module includes an intercepting unit, an acquiring unit, a judging unit and a determining unit, which needs to be explained. Yes, these sub-modules and units are not shown in the corresponding figures, where:

An intercepting unit, configured to intercept the above-mentioned face image based on the calibration result corresponding to the determined face key points when the above-mentioned image analysis parameter is the exposure parameter of the above-mentioned face image, to obtain a target face region, wherein the target face The area of the region is smaller than the area of the face image.

an obtaining unit, configured to obtain the first brightness value corresponding to the target face region.

A determination unit, configured to determine whether the first brightness value is greater than or equal to a preset brightness value threshold.

a determining unit, configured to determine that the ISP corresponding to the image capturing device needs to be adjusted when the determining unit determines that the first brightness value is greater than or equal to the preset brightness value threshold; or,

The second judging submodule is configured to determine the mean value of each channel in the three RGB channels of the target face region when the image analysis parameter is the white balance parameter of the face image, and judge the average value of all channels according to all the mean values. Whether the proportional distribution matches the pre-determined proportional distribution, when it is determined that the result is no, it is determined that the ISP corresponding to the above-mentioned image capture device needs to be adjusted; or,

The third judging sub-module is configured to acquire the target position where the target face region appears in the image acquisition device when the image analysis parameter is the focus parameter of the face image, and judge whether the target position is within the predetermined target position. Within the location range, when it is determined that the result is no, it is determined that the ISP corresponding to the above image acquisition device needs to be adjusted.

It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image can also obtain the brightness value of the face image, and compare the brightness value with the preset brightness value, and/or, judge the RGB three of the face image. Whether the allocation of channels is reasonable, and/or, comparing the position where the acquired face image appears on the image capture device with the pre-determined position range can realize the judgment that the ISP needs to adjust when the image capture device is overexposed; And by providing a variety of ways to determine the ISP needs to be adjusted, it can not only improve the judgment accuracy and flexibility of the ISP need to adjust; and before obtaining the brightness of the face image, further intercept the face image through the calibration results corresponding to the key points of the face. , which can improve the efficiency and accuracy of determining the brightness value of the face image, thereby helping to improve the accuracy and reliability of whether the ISP needs to be adjusted.

In another optional embodiment, the acquisition unit is further configured to expand the effective area of the target face region according to a preset expansion method when the determination unit determines that the first brightness value is not greater than or equal to a preset brightness value threshold , to obtain an enlarged target face area, and the preset enlargement method is a method set based on the center position of the target face area.

The determining unit is further configured to determine the second brightness value of the background region corresponding to the enlarged target face region.

The acquiring unit is further configured to calculate the backlight intensity of the face image based on the first brightness value and the second brightness value.

The determining unit is configured to determine whether the backlight intensity is less than or equal to the preset backlight intensity threshold, and when it is determined that the backlight intensity is less than or equal to the preset backlight intensity threshold, trigger the determining unit to perform the above-mentioned operation of determining that the ISP corresponding to the image capture device needs to be adjusted.

It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image can also calculate the brightness value of the background region corresponding to the enlarged face region when it is determined that the brightness value of the face region is small, and based on the two brightness values. Obtain the backlight of the face image, and when it is determined that the backlight of the face image is small, determine that the ISP needs to be adjusted, so as to improve the judgment that the ISP needs to be adjusted in the case of insufficient exposure of the image acquisition device, which can improve the ISP needs to be adjusted. Therefore, it is beneficial to the all-round dynamic adjustment of the ISP, which is beneficial to the accuracy of the subsequent acquisition of dynamic images.

In yet another optional embodiment, the above-mentioned ISP adaptive adjustment and control device based on a face image may further include a second judgment module 305. In this case, the ISP adaptive adjustment and control device based on a face image may be as shown in FIG. 4 4 is a schematic structural diagram of another ISP adaptive adjustment control device based on a face image, wherein:

The second judging module 305 is used to judge whether the ISP needs to be adjusted according to the current state of the ISP before the obtaining module 301 obtains the face image collected by the image collecting device; when it is judged that the ISP needs to be adjusted, trigger the obtaining module 301 to execute The above operation of acquiring the face image collected by the image collection device.

In this optional embodiment, optionally, as shown in FIG. 4 , the second judgment module 305 may include a reading sub-module 3051, an acquisition sub-module 3052, a calculation sub-module 3053, and a fourth judgment sub-module 3054, wherein:

The reading sub-module 3051 is used to read the target time currently displayed by the image processing system corresponding to the above ISP.

The obtaining sub-module 3052 is configured to obtain the completion time corresponding to the last adjustment of the above-mentioned ISP.

The calculation sub-module 3053 is configured to calculate the interval duration between the above-mentioned target time and the above-mentioned completion time.

The fourth judging sub-module 3054 is configured to judge whether the above-mentioned interval duration is less than or equal to a predetermined interval duration threshold.

The acquisition submodule 3052 is further configured to determine that the ISP needs to be adjusted when the fourth determination submodule 3054 determines that the interval duration is less than or equal to the interval duration threshold.

It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image described in FIG. 4 can also determine whether the ISP needs to be adjusted according to the current state of the ISP before acquiring the face image. The face image operation can reduce the need to adjust the ISP, that is, when the various indicators (such as: white balance, etc.) of the image acquisition device match the current environment, the ISP adjustment operation can be reduced, which is conducive to reducing the occurrence of ISP. Incorrect adjustment leads to the situation that the image acquisition device collects the face image of poor quality, which reduces the power consumption of the system and improves the service life of the system.

In yet another optional embodiment, as shown in FIG. 4 , the obtaining sub-module 3052 is further configured to determine that the ISP needs to be adjusted after the fourth judging sub-module 3054 determines that the interval duration is less than or equal to the interval duration threshold. Before, the total number of times of adjustment of the ISP within a first preset time period is acquired, wherein the cut-off time of the first preset time period is the last time the ISP was adjusted.

The fourth judging sub-module 3054 is also used to judge whether the above-mentioned total adjustment times is less than the preset adjustment times threshold. When judging that the total adjustment times is less than the preset adjustment times threshold, trigger the acquisition sub-module 3052 to execute the above-mentioned determination that the ISP needs to be adjusted operation.

It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image described in FIG. 4 can further determine that the ISP of the image processing system is in the current running process when it is determined that the time to adjust the ISP last time is short. When the total number of adjustment times of the ISP is small, it is determined that the ISP needs to be adjusted, which can improve the accuracy of the judgment of the ISP needs to be adjusted; and by limiting the total number of adjustments made by the ISP in the system operation process, it can meet the performance requirements of the ISP and adapt to complex environments. , so as to improve the user experience.

In yet another optional embodiment, as shown in FIG. 4 , the above-mentioned ISP adaptive adjustment control device based on a face image may further include a determination module 306 and a third determination module 307, wherein:

The determination module 306 is used to determine the human face after the acquisition module 301 acquires the face image collected by the image acquisition device, and before the analysis module 302 inputs the face image into the pre-determined face image quality evaluation model for analysis. The image parameter corresponding to the face image, the image parameter includes at least one of the area of the face image, the position where the face image appears in the above-mentioned image acquisition device, and the facial information of the face image, and the facial information of the face image includes a person. The position of the left virtual pupil of the face image, the position of the right virtual pupil, the position of the left mouth corner of the virtual mouth, the position of the right mouth corner of the virtual mouth, and the position of the virtual nose tip.

The third judging module 307 is configured to judge whether the above-mentioned face image satisfies the predetermined adjustment condition of the above-mentioned ISP according to the above-mentioned image parameters, and when it is judged that the face image satisfies the above-mentioned adjustment condition of the ISP, triggering the analysis module 302 to execute the above-mentioned The face image input is analyzed in the pre-determined face image quality assessment model.

It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image described in FIG. 4 can also further determine whether the face image satisfies the adjustment conditions of the ISP after obtaining the face image, and after judging that the face image satisfies the adjustment conditions of the ISP. Follow-up operations are performed only when the conditions are adjusted, which can reduce the occurrence of errors in ISP adjustment due to the fact that the face image does not meet the adjustment conditions of the ISP but use it to adjust the ISP, which is beneficial to improve the accuracy of the dynamic adjustment of the ISP and reduce the number of images The power consumption of the processing system; when it is judged that the adjustment conditions of the ISP are not met, the operation of acquiring the face image is re-triggered, which is conducive to obtaining images that meet the adjustment conditions of the ISP, and is further conducive to the dynamic adjustment of the ISP.

In yet another optional embodiment, as shown in FIG. 4 , the above-mentioned ISP adaptive adjustment control device based on a face image may further include a fourth judgment module 308, wherein:

The fourth judging module 308 is configured to adjust the above-mentioned ISP according to the above-mentioned image analysis parameters in the adjustment module 304 until the current parameters of the ISP meet the preset conditions, and after triggering the acquisition module 301 to re-trigger the execution of the above-mentioned acquisition image acquisition equipment Before the operation of the face image of the ISP, determine whether the real-time display time of the image processing system corresponding to the ISP has reached the cut-off time of the second preset time length, and when it is judged that the real-time display time has reached the cut-off time of the second preset time length, trigger the The obtaining module 301 re-triggers the execution of the above-mentioned operation of obtaining the face image collected by the image collecting device, and the starting time of the second preset duration is the time when it is determined that the ISP needs to be adjusted.

It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image described in FIG. 4 can also determine whether the current moment reaches the preset moment after the ISP adjustment is completed, and if it does, re-execute the operation of acquiring the face image , which can meet the requirement that the parameters of the ISP take a certain time from being written to the register to take effect, thereby reducing the occurrence of subsequent direct reading of the image that may damage the stability of the image processing system, thereby ensuring the stability of the image processing system.

Embodiment 4

Please refer to FIG. 5. FIG. 5 is another apparatus for self-adaptive adjustment and control of ISP based on a face image disclosed in an embodiment of the present invention. The apparatus described in FIG. 5 can be applied to an image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built in a terminal device that can collect faces, wherein the terminal device includes User terminals and/or access control devices, etc. As shown in FIG. 5 , the ISP adaptive adjustment control device based on the face image may include:

a memory 501 storing executable program code;

a processor 502 coupled to the memory 501;

Further, it may also include an input interface 503 and an output interface 504 coupled with the processor 502;

The processor 502 invokes the executable program code stored in the memory 501 to execute the steps of the face image-based ISP adaptive adjustment control method described in the first embodiment or the second embodiment.

Embodiment 5

An embodiment of the present invention discloses a computer-readable storage medium, which stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the face image-based ISP adaptation described in the first or second embodiment Steps to adjust the control method.

Embodiment 6

An embodiment of the present invention discloses a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the description in the first embodiment or the second embodiment. The steps of ISP adaptive adjustment control method based on face image.

The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed over multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by means of hardware. Based on such understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or that make contributions to the prior art. The computer software products can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory. (Read-Only Memory, ROM), Random Access Memory (Random Access Memory, RAM), Programmable Read-only Memory (Programmable Read-only Memory, PROM), Erasable Programmable Read Only Memory (Erasable Programmable Read Only Memory, EPROM) , One-time Programmable Read-Only Memory (OTPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM), CompactDisc Read-Only Memory , CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

Finally, it should be noted that the method and device for self-adaptive adjustment and control of ISP based on face images disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, not It is limited; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that; it is still possible to modify the technical solutions recorded in the foregoing embodiments, or perform some technical features. Equivalent replacement; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. An ISP self-adaptive adjustment control method based on a face image is characterized by comprising the following steps:

acquiring a face image acquired by image acquisition equipment, and inputting the face image into a predetermined face image quality evaluation model for analysis;

obtaining an analysis result of the facial image quality evaluation model as an image analysis parameter of the facial image, and judging whether an ISP (internet service provider) corresponding to the image acquisition equipment needs to be adjusted or not according to the image analysis parameter;

and when the ISP is judged to be required to be adjusted, adjusting the ISP according to the image analysis parameters until the parameters of the ISP meet preset conditions, and triggering and executing the operation of acquiring the face image acquired by the image acquisition equipment again.

2. The ISP adaptive adjustment control method based on the facial image according to claim 1, wherein the image analysis parameter comprises at least one of an exposure parameter of the facial image, a white balance parameter of the facial image, and a focusing parameter of the facial image;

wherein, the determining whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters includes:

when the image analysis parameters are exposure parameters of the face image, intercepting the face image based on a calibration result corresponding to the determined face key points to obtain a target face region, wherein the area of the target face region is smaller than that of the face image;

acquiring a first brightness value corresponding to the target face area, judging whether the first brightness value is greater than or equal to a preset brightness value threshold, and determining that an ISP (internet service provider) corresponding to the image acquisition equipment needs to be adjusted when the first brightness value is judged to be greater than or equal to the preset brightness value threshold; or,

when the image analysis parameters are white balance parameters of the face image, determining the mean value of each channel in RGB three channels of the target face area, judging whether the proportion distribution of all the channels is matched with the predetermined proportion distribution according to all the mean values, and when the judgment result is negative, determining that an ISP (Internet service provider) corresponding to the image acquisition equipment needs to be adjusted; or,

and when the image analysis parameter is the focusing parameter of the face image, acquiring a target position of the target face region appearing in the image acquisition equipment, judging whether the target position is in a predetermined position range, and when the judgment result is negative, determining that an ISP (internet service provider) corresponding to the image acquisition equipment needs to be adjusted.

3. The ISP adaptive adjustment control method based on the facial image as claimed in claim 2, wherein the method further comprises:

when the first brightness value is judged to be not more than the preset brightness value threshold, expanding the effective area of the target face region according to a preset expansion mode to obtain an expanded target face region, wherein the preset expansion mode is a mode set based on the central position of the target face region;

determining a second brightness value of a background area corresponding to the enlarged target face area, and calculating the inverse luminosity of the face image based on the first brightness value and the second brightness value;

and judging whether the inverse luminosity is less than or equal to a preset inverse luminosity threshold value, and determining that the ISP corresponding to the image acquisition equipment needs to be adjusted when the inverse luminosity is less than or equal to the preset inverse luminosity threshold value.

4. The ISP adaptive adjustment and control method based on the facial image as claimed in any one of claims 1 to 3, wherein before the facial image acquired by the image acquisition device is acquired, the method further comprises:

judging whether the ISP needs to be adjusted or not according to the current state of the ISP, and triggering and executing the operation of acquiring the face image acquired by the image acquisition equipment when the ISP needs to be adjusted;

wherein the determining whether the ISP needs to be adjusted according to the current state of the ISP includes:

reading a target time currently displayed by the image processing system corresponding to the ISP, and acquiring a finishing time corresponding to the ISP adjustment at the last time;

and calculating the interval duration between the target time and the finishing time, judging whether the interval duration is less than or equal to a predetermined interval duration threshold, and determining that the ISP needs to be adjusted when the interval duration is less than or equal to the interval duration threshold.

5. The ISP adaptive adjustment control method based on the facial image as claimed in claim 4, wherein after determining that the interval duration is less than or equal to the interval duration threshold and before the ISP is determined to need to be adjusted, the method further comprises:

the method comprises the steps of obtaining the total adjustment times of the ISP within a first preset time length, judging whether the total adjustment times is smaller than a preset adjustment time threshold value, and triggering and executing the operation of determining that the ISP needs to be adjusted when the total adjustment times is judged to be smaller than the preset adjustment time threshold value, wherein the ending moment of the first preset time length is the moment of adjusting the ISP at the last time.

6. The ISP adaptive adjustment and control method based on facial images as claimed in claim 1, 2, 3 or 5, wherein after the facial image acquired by the image acquisition device is acquired and before the facial image is input into a predetermined facial image quality evaluation model for analysis, the method further comprises:

determining image parameters corresponding to the face image, wherein the image parameters comprise at least one of the area of the face image, the position of the face image appearing in the image acquisition equipment and the face information of the face image, and the face information of the face image comprises the position of a left virtual pupil, the position of a right virtual pupil, the position of a left mouth corner of a virtual mouth, the position of a right mouth corner of the virtual mouth and the position of a virtual nose tip of the face image;

and judging whether the face image meets the preset adjustment condition of the ISP or not according to the image parameters, and triggering and executing the operation of inputting the face image into a preset face image quality evaluation model for analysis when the face image is judged to meet the preset adjustment condition of the ISP.

7. The ISP adaptive adjustment and control method according to claim 1, 2, 3 or 5, wherein the ISP is adjusted according to the image analysis parameter until after a current parameter of the ISP satisfies a preset condition and before the operation of acquiring the facial image acquired by the image acquisition device is re-triggered, the method further comprises:

judging whether the real-time display time of the image processing system corresponding to the ISP reaches the ending time of a second preset time length, and when the real-time display time reaches the ending time of the second preset time length, re-triggering and executing the operation of acquiring the face image acquired by the image acquisition equipment, wherein the starting time of the second preset time length is the time when the ISP needs to be adjusted.

8. The ISP self-adaptive adjustment control device based on the face image is characterized by comprising an acquisition module, an analysis module, a first judgment module and an adjustment module, wherein:

the acquisition module is used for acquiring a face image acquired by the image acquisition equipment;

the analysis module is used for inputting the face image into a predetermined face image quality evaluation model for analysis;

the acquisition module is further used for acquiring an analysis result of the facial image quality evaluation model as an image analysis parameter of the facial image;

the first judging module is used for judging whether the ISP corresponding to the image acquisition equipment needs to be adjusted or not according to the image analysis parameters;

the adjusting module is configured to adjust the ISP according to the image analysis parameter when the first determining module determines that the ISP needs to be adjusted, until the parameter of the ISP meets a preset condition, and trigger the obtaining module to re-execute the operation of obtaining the face image collected by the image collecting device.

9. An ISP (Internet service provider) self-adaptive adjustment control device based on a face image is characterized by comprising:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the ISP adaptive adjustment control method based on the facial image according to any one of claims 1-7.

10. A computer storage medium, characterized in that the computer storage medium stores computer instructions, which when called, are used for executing the ISP adaptive adjustment control method based on the face image according to any one of claims 1 to 7.

Images