Background
The color can intuitively express the surface characteristics of the object, and plays an important role in image processing and practical application of computer vision. However, color formation is often affected by variations in illumination source conditions, resulting in an image with extremely unstable color characteristics. The human eyes have the visual characteristic of constant color, and the color perception of an object can be kept relatively unchanged even if the scene lighting light source is changed. The digital imaging system does not have the color constant visual characteristic of human eyes, and is easy to be subjected to the condition change of an illumination light source during imaging to cause the color instability of an image. The color constancy algorithm is used for enabling a digital imaging system to still obtain a color image which is consistent with the color perceived by a human visual system when the illumination condition changes, and is of great help and significance for improving the robustness and accuracy of related computer visual algorithms such as color feature extraction, target identification, person tracking, scene monitoring and the like.
The color constancy method is generally divided into two steps: the method comprises the steps of firstly estimating the light source color of an image on the basis of color information of a known image, and secondly converting the image color into the color under a standard light source according to a diagonal matrix transformation relation generated by the estimated light source color and the standard light source color. Since the diagonal matrix transformation process is relatively simple, the research on color constancy mainly focuses on light source color estimation, which is generally classified into a statistical-based method and a learning-based method according to a calculation process. The statistical-based color constancy method utilizes the color characteristics of the bottom layer of the image to estimate the illumination color of the image during imaging. Such as the maxRGB method, the greywold method, the SoG method, the GreyEdge method, and so forth. The color constancy method based on learning mainly establishes a certain prediction model through a large number of known images and the prior knowledge of the corresponding color of the illumination light source, and then realizes the prediction of the color information of the illumination light source based on the color information distribution of a newly shot image. At present, a colored domain mapping method is based on Bayesian theory, an image color correlation and a color constancy method based on a convolutional neural network, and the like. However, most color constancy methods achieve better accuracy of light source estimation when the assumed conditions are satisfied. To date, none of the algorithms have demonstrated good prediction accuracy across all data sets. When the light source color of the image is estimated, if the image characteristics do not meet the defined assumed conditions or the prior knowledge is insufficient, the accuracy of the light source color estimated by the light source is affected, so that a large error is generated in the light source estimation result.
In essence, the usual source colors in daily life are distributed substantially around the blackbody locus, although there are points where the source colors are further from the blackbody locus, these discrete sources are not common in practice. The IES318 light source data set collected by Aurelien et al and the HDULS543 light source data set collected in the laboratory provide spectral distribution information for a variety of different light sources. Both the IES318 and HDULS543 light source data sets are from light sources commonly found in daily life, and the light source color distribution can be obtained by applying them to the CIE1931 standard chromaticity system or the camera sony dxc930, as shown in fig. 2 and fig. 3, respectively. While the light source color distribution can be observed to be substantially distributed around the black body locus.
Disclosure of Invention
The invention mainly provides a color constancy method based on light source color distribution limitation, which is a post-processing method for various existing color constancy methods. And for the image of the light source color to be estimated, calculating or measuring a camera sensitivity curve when the image is shot, and constructing a color gamut range of light source color distribution by depending on a black body locus of black body radiation under the response of the camera sensitivity curve. And then, obtaining different light source color results for the image of the light source color to be estimated by using the existing color constancy method. And finally, limiting the light source color of which the estimation result is not in the color gamut to the color gamut boundary, thereby reducing the occurrence of larger error condition during light source estimation and achieving the improvement of robustness of various different color constancy methods.
The technical scheme adopted for solving the technical problems is a color constancy method based on light source color distribution limitation, the processing process is in an rg space, and the specific steps are as follows:
the method comprises the following steps that (1) accurate color gamut ranges are established for different cameras, and black body tracks of camera spaces are calculated;
constructing a color gamut range of light source color distribution by relying on a blackbody locus;
estimating the light source color of the image by using the existing color constancy method;
judging a light source estimation result, if the light source estimation result is in a color gamut range, not processing the light source estimation result, and if the light source estimation result is not in the color gamut range, mapping the light source estimation result into a color gamut boundary by a color gamut mapping method;
and (5) transforming the corrected chromaticity point to an RGB space, wherein the point obtained by the color gamut mapping is the chromaticity point corrected by the estimated light source.
In the step (1), for the image of the light source color to be estimated, firstly, a camera sensitivity curve for acquiring the image is predicted, and the spectral radiation of the black body is applied to the camera sensitivity curve to obtain the black body track.
And (3) constructing a color gamut range of light source color distribution by relying on the blackbody locus in the step (2). Three points are found on the blackbody locus. The corresponding chromaticity points m are respectively positioned at the high color temperature of the black body locus1The corresponding chromaticity point m at low color temperature2And finding a corresponding chromaticity point m intermediate the low and high color temperatures0. And at m0Find two points mHAnd mLTo extend the range of the color gamut. By m1,mH,m2The second order polynomial is calculated at three points to fit the upper boundary of the gamut and is scaled by m1, mL,m2A quadratic polynomial is calculated to fit the lower boundary of the gamut.
And (3) estimating the light source color of the image by utilizing various existing color constancy methods, and transforming the estimated light source color and the real light source color to an rg space.
And (4) judging whether the light source color result estimated by the method in the step (3) exists in the constructed color gamut range, if so, not processing, and if not, mapping the light source estimation result to the color gamut boundary by a color gamut mapping method to reduce errors, thereby improving the accuracy of light source estimation by the existing color constancy method.
Two gamut mapping methods:
let the point to be mapped be P (r, g), i.e., the illuminant color result estimated by the color constancy method, assuming that P is not within the constructed gamut.
(1) Minimum distance method (ICDL-D)
The idea of the minimum distance method is to map the point P to be mapped to the position having the shortest distance from the boundary of the color gamut as a result of the light source color distribution limiting method.
(2) Based on the center point method (ICDL-C)
The central point method is used to consider that most of the illumination light sources in scene shooting are in sunlight. The relative spectral power distribution of the standard illuminant D and the actual daylight is similar, and since D65 is the chromaticity point of the average daylight, the chromaticity point corresponding to 6500K is taken as the center point of the black body locus color temperature. Then, a point P to be mapped which is not within the constructed gamut is connected to the center point, and the intersection of the connecting line and the gamut boundary is taken as the result of the light source color distribution limiting method.
And (5) the point obtained by the color gamut mapping in the step (5) is the chromaticity point after the estimated light source is corrected, and the chromaticity point is converted into an RGB space, namely the corrected light source color. The method for RGB conversion is as follows: assuming that B is 1, B is 1-r-g, then
The RGB color values of the estimated light source can be obtained.
The technical scheme provided by the invention has the beneficial effects that:
after the color gamut corresponding to the light source color is constructed for the collected image camera, the light source color of the image collected by the camera is predicted by using a color constancy method. Firstly, judging whether the predicted light source estimation result exists in the constructed color gamut range, and if so, not performing color gamut mapping operation. If the color constancy method is not in the color gamut range, the estimation error of the light source of the image is larger. The light source color distribution limiting method reduces errors by mapping the light source estimation result into the constructed gamut boundary, thereby improving the accuracy of the light source estimation.
Detailed Description
The technical scheme of the invention can adopt a computer software technology to automatically carry out the process. For better understanding of the technical solutions of the present invention, the following detailed description of the present invention is made with reference to the accompanying drawings and examples. The embodiment of the invention reduces the error of the estimation results of different light sources of the SFU data set images, wherein the SFU data set is composed of 321 indoor images which are shot by a computer vision laboratory of Simon Frazier university under 11 common light sources. Referring to fig. 1, the process of the embodiment of the present invention includes the following steps:
the method comprises the following steps that (1) accurate color gamut ranges are established for different cameras, and black body tracks of camera spaces are calculated;
constructing a color gamut range of light source color distribution by relying on a blackbody locus;
estimating the light source color of the image by using the existing color constancy method;
judging a light source estimation result, if the light source estimation result is in a color gamut range, not processing the light source estimation result, and if the light source estimation result is not in the color gamut range, mapping the light source estimation result into a color gamut boundary by a color gamut mapping method;
and (5) transforming the corrected chromaticity point to an RGB space, wherein the point obtained by the color gamut mapping is the chromaticity point corrected by the estimated light source.
And (2) obtaining a camera sensitivity curve for collecting the SFU data set in the step (1), and applying the blackbody radiation intensity to the sensitivity curve to obtain a Planckian locus or a blackbody locus.
In step (2), generally, the chromaticity points of the common light source colors are approximately distributed on the color temperature of the blackbody locus from 2000K to 25000K. The chromaticity point of 2000K color temperature on the blackbody locus is recorded as m1(m1x,m1y) Color temperature 25000K is marked as m2(m2x,m2y). Since the correlated color temperature of the daylight illuminant D65 is 6500K, and the chromaticity point corresponding to 6500K is denoted as D65(a, b), the chromaticity point and the vicinity thereof generally indicate the color of daylight. Using D65 point, m1、m2A gamut of light source colors is constructed. Two suitable points are respectively taken above and below the point D65 and are marked as mH(mHx,mHy) And mL(mLx,mLy) The abscissa of the two points is equal to the abscissa a of the point D65, and m is used1,mH,m2Fitting a curve at three points as the upper boundary of the color gamut, and using m1,mL,m2A curve is fitted at three points as the lower boundary of the gamut. The constructed color gamut can be formed by two quadratic polynomials YHAnd YLExpressed, the calculation process of the constructed boundary equation on the color gamut is given by the following equations (1-4):
yH=A(1,1)x2+A(1,2)x+A(1,3) (1)
YH=AX (2)
Y=[m1y mHy m2y] (4)
and (3) estimating the light source color of the image by using the existing color constancy method, and transforming the estimated light source color and the real light source color to an rg space. Assuming that the source color is e ═ R (G, B), the conversion to the chromaticity space is calculated as:
and (4) judging whether the estimated light source result is in the constructed color gamut range in the rg space, if so, not processing, and if not, mapping the light source estimation result on the constructed color gamut boundary by two color gamut mapping methods.
Two gamut mapping methods:
let the point to be mapped be P (r, g), i.e., the illuminant color result estimated by the color constancy method, assuming that P is not within the constructed gamut.
(1) Minimum distance method (ICDL-D)
The idea of the minimum distance method is to map the point P to be mapped to the smallest distance from the gamut boundary. The simplest calculation method is to border the color gamutThe boundary is discretized into a series of points D, where D ═ D1,d2,...,dnAnd mapping the point P at the point D if the color gamut boundary point D is the minimum distance from the point P by comparing the Euclidean distance between the point P and the point P to be mapped in the set D. d is the light source color after the correction of the mapping points.
Where φ is the minimum Euclidean distance from point P to the midpoint of set D, D is the mapping point, and D ∈ D.
(2) Based on the center point method (ICDL-C)
The color temperature T of the sunlight track ranges from 4000K to 25000K, and most of lighting sources are considered to be under sunlight when a scene is shot based on a center point method. The relative spectral power distribution of the standard illuminant D and the actual sunlight is similar, and D65 is the chromaticity point of the average sunlight, so that the chromaticity point corresponding to 6500K of the black body locus color temperature is taken as the central point. The point P to be mapped which is not within the color gamut is connected to the center point, and the intersection of the connecting line and the color gamut boundary is taken as the result of the light source color distribution limiting method.
In specific calculation, the corresponding chromaticity point when the color temperature is 6500K is taken as w (a, b), the positions of w (a, b) under different camera sensitivity curves are different, and a linear equation expression determined by using the mapping point P and the point w is as follows:
simultaneous equations, solving for Y0And YH、YLThe intersection point of (2) is defined as a mapping point.
And (5) the point obtained by the color gamut mapping in the step (5) is the chromaticity point after the estimated light source is corrected, and the chromaticity point is converted into an RGB space, namely the corrected light source color. The method for RGB conversion is as follows: assuming that B is 1, B is 1-r-g, then
The RGB color values of the estimated light source can be obtained.
The feasibility of the technical scheme of the invention is proved as follows:
the angle error is the mainstream method for evaluating the color constancy algorithm which is widely applied at present, and the color e of the real light source is obtained through the imagea=(Ra,Ga,Ba) And the light source color e estimated by the algorithmb=(Rb,Gb,Bb) The angle between two color vectors is used to judge the difference between the estimated light source and the real light source, because the most concerned problem when performing light source color is to estimate the difference between the vector directions of the light source and the real light source without considering the difference of the vector sizes, and the angle error theta between the estimated light source and the real light sourceaIs defined as:
the smaller the angular error, the closer the algorithm gets to the measured true illuminant color. When a large number of data sets are used to evaluate the illuminant estimation method, evaluation indexes of a median (mediaangularror), a mean (meangulalarror), a maximum (maxaangularror) of the angle errors of the data sets, a mean error (Best-25% angularror, Best25) of the Best illuminant estimation results of the total number of 25%, and a mean error (Best-25% angularror, Best25) of the Worst illuminant estimation results of the total number of 25% are often used to better make a comprehensive overall evaluation of the illuminant estimation method.
The results of the different color constancy method experiments for the SFU dataset and the 568 dataset (481 images were used) are given below. As shown in table 1 and table 2, the improvement results of the ICDL-D method and the ICDL-C method for maxRGB method, gray world method (greywold method), shadesofgay method (SoG), GreyEdge method, GamutMapping method, and FFCC method (fast fourier color constancy) are significantly smaller than the original results, as can be observed from the Median value, Mean value, and Worst25 value. The error values of Median, Mean and Worst25 obtained after improvement are reduced, so that the error of the light source color estimation of the existing color constancy method can be reduced to a certain extent by the ICDL-D method and the ICDL-C method.
TABLE 4.1 ICDL-D and ICDL-C methods on SFU data set angular error improvement results for the existing method (xMethod)
TABLE 4.3 ICDL-D and ICDL-C results in angular error improvement on CC481 dataset for the existing method (xMethod)
The experimental data show that the two color gamut mapping methods provided by the invention both well reduce the original errors of the methods with different color constancy, and the method can effectively improve the robustness of the color constancy.
The foregoing is a more detailed description of the invention, taken in conjunction with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments disclosed. It will be understood by those skilled in the art that various changes in detail may be effected therein without departing from the scope of the invention as defined by the appended claims.