WO2008053408A2 - Color saturation enhancement - Google Patents
Color saturation enhancement Download PDFInfo
- Publication number
- WO2008053408A2 WO2008053408A2 PCT/IB2007/054341 IB2007054341W WO2008053408A2 WO 2008053408 A2 WO2008053408 A2 WO 2008053408A2 IB 2007054341 W IB2007054341 W IB 2007054341W WO 2008053408 A2 WO2008053408 A2 WO 2008053408A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pxy
- color
- pixel
- value
- scale
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/46—Colour picture communication systems
- H04N1/56—Processing of colour picture signals
- H04N1/60—Colour correction or control
- H04N1/6002—Corrections within particular colour systems
- H04N1/6005—Corrections within particular colour systems with luminance or chrominance signals, e.g. LC1C2, HSL or YUV
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/46—Colour picture communication systems
- H04N1/56—Processing of colour picture signals
- H04N1/60—Colour correction or control
- H04N1/6027—Correction or control of colour gradation or colour contrast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/86—Camera processing pipelines; Components thereof for processing colour signals for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits
Definitions
- the invention relates to a method of color saturation enhancement, a system for color enhancement, a display apparatus comprising the system, and a camera with the system.
- Color televisions have a control which allows the user to manually adjust the color saturation.
- video processing hardware and/or software applied in consumer electronics display apparatuses such as for example: a TV, a computer monitor, or a mobile device with a matrix display, often comprises algorithms which automatically enhance the image color to obtain an optimal image for the user.
- color enhancement algorithms increase the color saturation of the image, for example by multiplying the chrominance signal with a gain factor or by changing the chrominance signal in a non- linear manner.
- a first aspect of the invention provides as claimed in claim 1.
- a second aspect of the invention provides a system for color saturation enhancement as claimed in claim 11.
- a third aspect of the invention provides a display apparatus as defined in claim 12.
- a fourth aspect of the invention provides a camera as defined in claim 13.
- Advantageous embodiments are defined in the dependent claims.
- the method of color saturation enhancement in accordance with the first aspect of the invention receives an input image composed of pixels.
- the color of the pixels is defined by the color pixel values and the luminance of the pixels is defined by luminance values.
- the method operates on an input signal defined in a YUV space.
- the color pixel values are the UV values.
- the method is also applicable to other color spaces such as for example the RGB, XYZ, HSV or Lab color space. If the method is used in the RGB space, extra calculations are required to determine the color values from the RGB values.
- the color pixel values are also referred to as chrominance values.
- the luminance Y is a single value
- the chrominance UV has two values, one for U and one for V which are collectively referred to as the color pixel value.
- the color pixel values are referred to as chrominance values
- the three values YUV are referred to as the color of the pixel.
- At least three scales are generated: the lowest ranked scale is the input image, the lowest but one ranked scale is a low-pass filtered version of the input image, the lowest but two ranked scale is a low-pass filtered version of the lowest but one scale.
- the lowest but two ranked scale is a low passed version of the input image being stronger low-pass filtered than the input image for the lowest but one scale.
- the color pixel values of a same one of the pixels of the input image in different scales are compared to obtain comparison values.
- Weights are allocated to the scales for favoring a first color pixel value of the same one of the pixels in one of the scales over a second color pixel value of the same one of the pixels in another one of the scales having a higher rank if the first color pixel value and the second color pixel value differ more than a predetermined amount.
- the second color pixel value is favored over the first color pixel value if the first pixel color value and the second pixel color value differ less than the predetermined amount.
- the color saturation enhanced image is obtained by combining color pixel values of the same one of the pixels of the different scales weighted in accordance with the allocated weights.
- the weighted combination of pixel values of the scales is based on differences of pixel values in the different scales. If a pixel value of a stronger low-pass filtered version, thus of a higher ranked scale, significantly differs from the pixel value of a lower ranked scale, the lower ranked scale contains important detail information which was lost in the low-pass filtering applied to obtain the higher ranked scale. Consequently, in determining the combination, the pixel value in the lower ranked scale should be favored over the pixel value in the higher ranked scale to minimize artifacts. Or said differently, a weight allocated to the pixel value of the lower ranked scale should be higher than that allocated to the pixel value of the higher ranked scale to prevent loosing too much of the detail information.
- the lower ranked scale does not contain important detail information which gets lost due to the low-pass filtering applied to obtain the higher ranked scale.
- the pixel value in the higher ranked scale should be favored over the pixel value of the lower ranked scale to minimize artifacts.
- the detail information is identical or almost identical in these two scales. Thus, no or hardly any detail information is lost by favoring the pixel value in the highest ranked scale.
- EP 1 094 419 Al discloses a method of enhancing color images.
- a multi-resolution representation is generated which comprises band-pass detail images at multiple resolution levels and a residual image.
- the multi-resolution representation is generated of at least one color component of a color image in a selected color space by applying a decomposition procedure to the color component.
- the multi-resolution representation is modified by application of at least one non-linear modifying function.
- the modifying is obtained by multiplying the value of a detail image at a particular pixel and at a particular resolution level with a factor which is obtained by evaluating a non-linear modifying function in an argument value which depends on pixel values of the detail images of the color components.
- the multi-resolution representation comprises directional band-pass detail images.
- the multi-resolution representation is a multi-resolution gradient representation.
- the argument of the modifying function is the norm of the color gradient in a specific pixel and a specific resolution level.
- the modified color component images are generated by applying the inverse of the decomposition step to the modified multi-resolution representation.
- This prior art does not disclose the use of differences of pixel values of the same pixel in different scales to determined the weight factors allocated to the scales during the combination step.
- the color enhancement function is applied to the each one of the color pixel values separately.
- the resulting adapted color pixel values are multiplied with their associated allocated weights and the resulting terms are added. This has the advantage that different color enhancement functions are possible for pixels values of different scales.
- the weights are calculated for each scale, starting with the lowest ranked scale, by taking the minimum of the multiplication of a predefined factor and a function of the at least one of the comparison values on the one hand and the difference of the total value of the weigh factors which can be allocated minus the sum of the weight factors allocated so far.
- the total value of the weight factors which can be allocated is equal to 1.
- the predefined factor may be application dependent and may be controlled by the algorithm or by a user. This approach has a high flexibility and requires a relative low computational effort.
- the function of at least one of the comparison values is implemented by calculating the difference of the pixel values of a particular pixel in two successive ones of the scales. The calculation of these differences requires a low computational effort.
- the function of at least one of the comparison values may be a more complex function of the differences.
- the comparing of further pixel values in higher ranked scales is stopped and the combining is started once the pixel color values which are compared differ less than the predetermined amount.
- the further low passing may have no sense anymore because when the pixel values are identical or almost identical, further low-pass filtering does not remove artifacts anymore.
- the combining is obtained by calculating a sum value which is the sum of the color pixel values multiplied by their associated allocated weights.
- the combining comprises a color enhancement function applied to the sum value.
- the weighted combination of color pixel values is used as the argument in the color enhancement function.
- the color enhancement function may be any of well known color enhancement functions such as for example, a blue stretch, or a skin tone corrector.
- the color saturation enhancement is applied several times in different frequency bands of the input image.
- Fig. 1 schematically shows an example of successive scales
- Fig. 2 shows a block diagram of an embodiment of the system for color saturation enhancement in a display application
- Fig. 3 shows an example of the frequency response of the color and the luminance of the system for saturation enhancement
- Fig. 4 shows a camera application.
- Fig. 1 schematically shows an example of successive scales.
- the lowest scale SCO in the hierarchy is the input image II.
- the plane in which the input image II resides is the xy plane as indicated by the arrows x and y, respectively.
- the different scales are displaced in the z direction which is indicated by the arrow S.
- a particular pixel in the input image II is indicated by Pxy, wherein the xy indicate that the pixel at position x,y in the input image II is meant.
- the notation P(x,y) may be used.
- the notation Pxy(i) is used wherein the i indicates the scale.
- Pxy(O) is the color pixel value of the original input image pixel Pxy
- Pxy(l) is the color pixel value of the pixel Pxy after the low-pass filtering applied on the input image II to obtain the low-pass filtered image on the scale SCl.
- the color pixel value Pxy(l) correspond to the color pixel value Pxy(0) of the pixel Pxy.
- the color pixel value Pxy(2) in the scale SC2 may be obtained by low-pass filtering the original input image II stronger than the low-pass filtering used to obtain the scale SCl.
- the color pixel value Pxy(2) may be obtained by low-pass filtering the image of scale SCl.
- Fig. 2 shows a block diagram of an embodiment of the system for color saturation enhancement in a display application.
- the color saturation enhancement system SYS comprises a scale generator 2, a comparator 3, a weight allocator 4, and a combiner 5.
- the block 1 is the origin of the input image II.
- the input image II may be provided by a broadcast channel, internet, a DVD player or any other apparatus that reproduces input images such as for example a camera.
- the input image II may be retrieved from a storage medium such as for example a memory.
- the storage medium may be removable such as for example a DVD or a memory stick.
- the scale generator 2 receives the input image II and generates the different scales SCi (SCO, SCl, SC2 in the embodiment shown in Fig. 1).
- the scale generator 2 may comprise a low-pass filter and a memory.
- the color pixel values Pxy(0) of the original image are low-pass filtered to obtain the color pixel values Pxy(l) of the scale SCl which are stored in the memory.
- the low-pass filter is used to low-pass filter the stored color pixel values Pxy(l) of the scale SCl to calculate the color pixel values Pxy(2) of the scale SC2.
- the low-pass filter may have different filter coefficients for the different low-pass filter operations.
- the same low-pass filter with different filter coefficients may be used to sequentially low-pass filter the color pixels Pxy(O) of the input image to obtain the scales SCl and SC2 wherein the color pixel values Pxy(2) of the scale SC2 are a stronger low-pass filtered input image than the color pixel values Pxy(l) of the scale SCl.
- several low-pass filters may be used to create the scales SCl and SC2 from the input image II. More than two scales SCl and SC2 may be generated. It has to be noted that directly referring to an operation applied on the scale SCi means that this operation is applied on the pixels Pxy(i) of this scale SCi.
- the comparator 3 compares the color pixel values Pxy(i) of the pixel Pxy in different scales SCi to obtain comparison values CVi which indicate the difference between the color pixel values Pxy(i) of the two scales which are compared.
- the comparison values CVi may be this difference or may be a function of this difference.
- the weight allocator 4 allocates weights wi to the scales SCi corresponding to the comparison values CVi.
- the weights wi are allocated to the pixel values Pxy(i) of the pixel Pxy in the different scales SCi.
- the comparison value CVi resulting from the comparison of the color pixel value Pxy(i) of the pixel Pxy in scale SCi and the color pixel value PxyO) of pixel Pxy in scale SCj indicates that the difference (or difference function) value is smaller or larger than a predetermined value.
- the scale SCj has a higher rank than the scale SCi.
- the scale SCj is a low- pass filtered version of the scale SCi.
- the difference is smaller than the predetermined value it is concluded that no information in the image at the particular pixel Pxy is lost due to the low-pass filtering. Consequently, it is possible to take the stronger low-pass filtered value without losing detail in the image.
- the advantage is that due to the stronger low-pass filtering artifacts are better removed.
- the pixel value PxyO) should be favored with respect to the pixel value Pxy(i) by allocating a higher weight wj to the pixel value PxyO) than the weight wi to the pixel value Pxy(i).
- the actual weight wj allocated may be inverse proportional to the difference.
- the pixel value PxyO is favored over the pixel value PxyO) by allocating a higher weight wi to the pixel value Pxy(i) than the weight wj to the pixel PxyO).
- the actual weight wi allocated may be proportional to the difference. In an embodiment, the weights are calculated as
- x,y indicate a position in the input image of the pixel Pxy for which the weights wi(x,y) are calculated in the scales SCi.
- the number of scales SCi is indicated by S.
- F(D 1 ) is a function of at least one of the comparison values (CVi).
- T is a predefined factor which determines a scaling of the values of the function F(Di).
- the weights wi are indicated by W 1 (X 5 V) or in the sum as w,(x,y) to make clear that the weights wi are determined for each one of the pixels Pxy in the input image II. It has to be noted that the determination of the weights wi need not use the same function F(Di) for the whole input image II.
- the input image II may be segmented, for example in blocks which are parallel processed, or more sophisticated in areas of similar pixel values, or in objects and different functions F(Di) may be used for different areas.
- the sum only comprises the weight w ⁇ (x,y) allocated to the pixel value Pxy(O) in scale SCO of the pixel Pxy.
- the combiner 5 combines the color pixel values Pxy(i) of the scales SCi to obtain a combined value OV.
- the combination is made by using the weights wi for the color pixels values Pxy(i).
- the combining may start after all the weights wi for all the scales SCi have been determined. Alternatively, the combining is started once the pixel color value
- the combining multiplies the color pixel values Pxy(i) with their associated allocated weights wi to obtain terms and calculates the combined value OV by summing the terms:
- the value of the combined value OV which is a sum of the terms, will be close to the color pixel value Pxy(O) of the scale SCO in the presence of significant local detail, while it will be closer to the color pixel value Pxy(S-l) if there is little detail.
- a color enhancement function Fee is applied to the combined value OV to obtain a more robust value OVR for the combined value OV.
- the color enhancement function Fee is applied to the each one of the color pixel values Pxy(i) to obtain adapted color pixel values P'xy(i) for each one of the scales (SCi).
- the adapted color pixel values P'xy(i) are multiplied with the associated allocated weights wi to the corresponding scales SCi to obtain weighted adapted color pixel values WP'xy(i) which are summed to obtain the value OVR.
- Which for wmax 1 leads to the equation:
- the input signal II is separated in at least two frequency bands which may be disjunct or may partly overlap.
- the above described approach is applied to at least one of these frequency bands, but may also be applied to each frequency band separately.
- the signal processor 6 processes the combined value OV, or the robust value OVR if a color function is applied, to obtain drive signals DR suitable to drive the display device 7.
- Fig. 3 shows an example of the frequency response of the color and the luminance of the system for saturation enhancement.
- the frequency f is depicted along the horizontal axis and the normalized response RES is depicted along the vertical axis.
- the frequency response of the luminance signal Y is indicated by the curve YR and the frequency response of the color signal UV is indicated by the curve UVR.
- the DC-response of the color signal UV is higher than the DC-response of the luminance signal Y to obtain a color boost.
- the color pixel value gain for DC or low frequencies is higher than the luminance pixel value gain of the pixel Pxy.
- the color signal response UVR decreases with frequency due to the low-pass filtering in the scales.
- the luminance signal response YR increases with higher frequency to finally steeply drop for high frequencies.
- the initial increase provides a contrast boost.
- a color boost is obtained by having the DC-response higher than the DC-response for luminance, where both depend on the amount of detail present in the image. If a low amount of detail is present, the color response UVR starts decreasing soon because the color pixel values of high ranked scales are favored to minimize the visibility of disturbances. If a high amount of detail is present, the color response UVR is at a high level up to relatively high frequencies because the color pixel values of lower ranked scales are favored and thus less low-pass filtering is applied to retain the details.
- Fig. 4 shows a camera application.
- the camera 10 comprises an image sensor which directly or after suitable signal processing supplies the input image II to the color enhancement system SYS.
- any reference signs placed between parentheses shall not be construed as limiting the claim.
- Use of the verb "comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim.
- the article "a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
- the invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Of Color Television Signals (AREA)
- Image Processing (AREA)
Abstract
A method of color saturation enhancement comprises receiving (1) an input image (II) composed of pixels. At least three scales (SC0, SC1, SC2) are generated (2), the lowest ranked scale (SC0) being the input image (II), the lowest but one ranked scale (SC1) being a low-pass filtered version of the input image (II), the lowest but two ranked scale (SC2) being a low-pass filtered version of the lowest but one scale (SC1) or a low passed version of the input image (II) being stronger low-pass filtered than the input image (II) for the lowest but one scale (SC1). The color pixel values (Pxy(0), Pxy(1), Pxy(2)) of a particular pixel (Pxy) of the input image (II) in different scales (SC0, SC1, SC2) are compared (3) to obtain comparison values (CVi). Weights (w0, w1, w2) are allocated (4) to the scales (SC0, SC1, SC2) for favoring a first color pixel value (Pxy(l)) of the particular pixel (Pxy) in one of the scales (SC1) over a second color pixel value (Pxy(2)) of the particular pixel (Pxy) in another one of the scales (SC2) having a higher rank if the first color pixel value (Pxy(l)) and the second color pixel value (Pxy(2)) differ more than a predetermined amount, and for favoring the second color pixel value (Pxy(2)) over the first color pixel value (Pxy(1)) if the first pixel color value (Pxy(l)) and the second pixel color value (Pxy(2)) differ less than the predetermined amount. The color pixel values (Pxy(0), Pxy(1), Pxy(2)) of the particular pixel (Pxy) of the different scales (SC0, SC1, SC2) are combined (5) weighted in accordance with the allocated weights (w0, w1, w2).
Description
Color saturation enhancement
FIELD OF THE INVENTION
The invention relates to a method of color saturation enhancement, a system for color enhancement, a display apparatus comprising the system, and a camera with the system.
BACKGROUND OF THE INVENTION
Color televisions have a control which allows the user to manually adjust the color saturation. Further, video processing hardware and/or software applied in consumer electronics display apparatuses, such as for example: a TV, a computer monitor, or a mobile device with a matrix display, often comprises algorithms which automatically enhance the image color to obtain an optimal image for the user. Often, color enhancement algorithms increase the color saturation of the image, for example by multiplying the chrominance signal with a gain factor or by changing the chrominance signal in a non- linear manner.
In many cases, video content is compressed in the path between registration and display on the users display apparatus. It has to be noted that most video compression techniques make use of the fact that viewers are less sensitive for errors in the chrominance (for example, the color difference signal) than for errors in the luminance. Accordingly, usually, a lower bandwidth is allocated to the chrominance than to the luminance. Consequently, usually, the quantization of the chrominance is courser than that of the luminance. Enhancing the colors in an image highlights these errors in the color which were introduced by the compression scheme. Effectively, this means that the compression scheme, over which a display device has typically no control, limits the color enhancement in the display apparatus possible without introducing annoying artifacts.
SUMMARY OF THE INVENTION
It is an object of the invention to provide a color enhancement which minimizes artifacts.
A first aspect of the invention provides as claimed in claim 1. A second aspect of the invention provides a system for color saturation enhancement as claimed in claim 11.
A third aspect of the invention provides a display apparatus as defined in claim 12. A fourth aspect of the invention provides a camera as defined in claim 13. Advantageous embodiments are defined in the dependent claims.
The method of color saturation enhancement in accordance with the first aspect of the invention receives an input image composed of pixels. The color of the pixels is defined by the color pixel values and the luminance of the pixels is defined by luminance values. In the now following, it is assumed that the method operates on an input signal defined in a YUV space. In the YUV space the color pixel values are the UV values. However, the method is also applicable to other color spaces such as for example the RGB, XYZ, HSV or Lab color space. If the method is used in the RGB space, extra calculations are required to determine the color values from the RGB values.
It has to be noted that in literature the color pixel values are also referred to as chrominance values. Thus, the luminance Y is a single value, and the chrominance UV has two values, one for U and one for V which are collectively referred to as the color pixel value. In literature in which the color pixel values are referred to as chrominance values, the three values YUV are referred to as the color of the pixel.
At least three scales are generated: the lowest ranked scale is the input image, the lowest but one ranked scale is a low-pass filtered version of the input image, the lowest but two ranked scale is a low-pass filtered version of the lowest but one scale. Alternatively, the lowest but two ranked scale is a low passed version of the input image being stronger low-pass filtered than the input image for the lowest but one scale.
The color pixel values of a same one of the pixels of the input image in different scales are compared to obtain comparison values. Weights are allocated to the scales for favoring a first color pixel value of the same one of the pixels in one of the scales over a second color pixel value of the same one of the pixels in another one of the scales having a higher rank if the first color pixel value and the second color pixel value differ more than a predetermined amount. The second color pixel value is favored over the first color pixel value if the first pixel color value and the second pixel color value differ less than the predetermined amount. The color saturation enhanced image is obtained by combining color pixel values of the same one of the pixels of the different scales weighted in accordance with the allocated weights. The weighted combination of pixel values of the scales, which are the original image and successively stronger low-passed versions of the original image, is based on differences of pixel values in the different scales.
If a pixel value of a stronger low-pass filtered version, thus of a higher ranked scale, significantly differs from the pixel value of a lower ranked scale, the lower ranked scale contains important detail information which was lost in the low-pass filtering applied to obtain the higher ranked scale. Consequently, in determining the combination, the pixel value in the lower ranked scale should be favored over the pixel value in the higher ranked scale to minimize artifacts. Or said differently, a weight allocated to the pixel value of the lower ranked scale should be higher than that allocated to the pixel value of the higher ranked scale to prevent loosing too much of the detail information.
If a pixel value of a higher ranked scale does not significantly differ from the pixel value of a lower ranked scale, the lower ranked scale does not contain important detail information which gets lost due to the low-pass filtering applied to obtain the higher ranked scale. Thus, in determining the combination, the pixel value in the higher ranked scale should be favored over the pixel value of the lower ranked scale to minimize artifacts. The detail information is identical or almost identical in these two scales. Thus, no or hardly any detail information is lost by favoring the pixel value in the highest ranked scale.
It has to be noted that EP 1 094 419 Al discloses a method of enhancing color images. A multi-resolution representation is generated which comprises band-pass detail images at multiple resolution levels and a residual image. The multi-resolution representation is generated of at least one color component of a color image in a selected color space by applying a decomposition procedure to the color component.
The multi-resolution representation is modified by application of at least one non-linear modifying function. In one embodiment, the modifying is obtained by multiplying the value of a detail image at a particular pixel and at a particular resolution level with a factor which is obtained by evaluating a non-linear modifying function in an argument value which depends on pixel values of the detail images of the color components. In one embodiment, the multi-resolution representation comprises directional band-pass detail images. The multi-resolution representation is a multi-resolution gradient representation. In a specific embodiment, the argument of the modifying function is the norm of the color gradient in a specific pixel and a specific resolution level. The modified color component images are generated by applying the inverse of the decomposition step to the modified multi-resolution representation.
This prior art does not disclose the use of differences of pixel values of the same pixel in different scales to determined the weight factors allocated to the scales during the combination step.
In an embodiment, the color enhancement function is applied to the each one of the color pixel values separately. The resulting adapted color pixel values are multiplied with their associated allocated weights and the resulting terms are added. This has the advantage that different color enhancement functions are possible for pixels values of different scales.
In an embodiment, the weights are calculated for each scale, starting with the lowest ranked scale, by taking the minimum of the multiplication of a predefined factor and a function of the at least one of the comparison values on the one hand and the difference of the total value of the weigh factors which can be allocated minus the sum of the weight factors allocated so far. Usually, the total value of the weight factors which can be allocated is equal to 1. The predefined factor may be application dependent and may be controlled by the algorithm or by a user. This approach has a high flexibility and requires a relative low computational effort.
In an embodiment, the function of at least one of the comparison values is implemented by calculating the difference of the pixel values of a particular pixel in two successive ones of the scales. The calculation of these differences requires a low computational effort. Alternatively, the function of at least one of the comparison values may be a more complex function of the differences.
In an embodiment, the comparing of further pixel values in higher ranked scales is stopped and the combining is started once the pixel color values which are compared differ less than the predetermined amount. Below a particular difference the further low passing may have no sense anymore because when the pixel values are identical or almost identical, further low-pass filtering does not remove artifacts anymore. This approach has the advantage that no valuable processing time is spent on irrelevant calculations. In an embodiment, the combining is obtained by calculating a sum value which is the sum of the color pixel values multiplied by their associated allocated weights.
In an embodiment, the combining comprises a color enhancement function applied to the sum value. In this approach the weighted combination of color pixel values is used as the argument in the color enhancement function. The color enhancement function may be any of well known color enhancement functions such as for example, a blue stretch, or a skin tone corrector.
In an embodiment, the color saturation enhancement is applied several times in different frequency bands of the input image.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.
BRIEF DESCRIPTION OF THE DRAWINGS In the drawings:
Fig. 1 schematically shows an example of successive scales, Fig. 2 shows a block diagram of an embodiment of the system for color saturation enhancement in a display application,
Fig. 3 shows an example of the frequency response of the color and the luminance of the system for saturation enhancement, and Fig. 4 shows a camera application.
It should be noted that items which have the same reference numbers in different Figures, have the same structural features and the same functions, or are the same signals. Where the function and/or structure of such an item has been explained, there is no necessity for repeated explanation thereof in the detailed description.
DETAILED DESCRIPTION
Fig. 1 schematically shows an example of successive scales. The lowest scale SCO in the hierarchy is the input image II. The plane in which the input image II resides is the xy plane as indicated by the arrows x and y, respectively. The different scales are displaced in the z direction which is indicated by the arrow S.
A particular pixel in the input image II is indicated by Pxy, wherein the xy indicate that the pixel at position x,y in the input image II is meant. To indicate a pixel at position x,y also the notation P(x,y) may be used. To indicate the color values of the pixel Pxy, or corresponding to the pixel Pxy if the higher level scales are sub-sampled, the notation Pxy(i) is used wherein the i indicates the scale. Thus, Pxy(O) is the color pixel value of the original input image pixel Pxy, and Pxy(l) is the color pixel value of the pixel Pxy after the low-pass filtering applied on the input image II to obtain the low-pass filtered image on the scale SCl. In the scale SCl the color pixel value Pxy(l) correspond to the color pixel value Pxy(0) of the pixel Pxy. The color pixel value Pxy(2) in the scale SC2 may be obtained by low-pass filtering the original input image II stronger than the low-pass filtering used to obtain the scale SCl. Alternatively, the color pixel value Pxy(2) may be obtained by low-pass filtering the image of scale SCl.
Usually the small letters i or j which follow a capital letter indicate an index. These indexes are used in equations to indicate that the equations are valid for any of the items indicated by the capital letter, or to indicate the scale SCi. For example, with W1(X5V) is meant the weight factor for the pixel Pxy at the position x,y for the scale i. Fig. 2 shows a block diagram of an embodiment of the system for color saturation enhancement in a display application. The color saturation enhancement system SYS comprises a scale generator 2, a comparator 3, a weight allocator 4, and a combiner 5.
The block 1 is the origin of the input image II. The input image II may be provided by a broadcast channel, internet, a DVD player or any other apparatus that reproduces input images such as for example a camera. The input image II may be retrieved from a storage medium such as for example a memory. The storage medium may be removable such as for example a DVD or a memory stick.
The scale generator 2 receives the input image II and generates the different scales SCi (SCO, SCl, SC2 in the embodiment shown in Fig. 1). The scale generator 2 may comprise a low-pass filter and a memory. In one embodiment, first, the color pixel values Pxy(0) of the original image are low-pass filtered to obtain the color pixel values Pxy(l) of the scale SCl which are stored in the memory. Next the low-pass filter is used to low-pass filter the stored color pixel values Pxy(l) of the scale SCl to calculate the color pixel values Pxy(2) of the scale SC2. The low-pass filter may have different filter coefficients for the different low-pass filter operations. Alternatively the same low-pass filter, with different filter coefficients may be used to sequentially low-pass filter the color pixels Pxy(O) of the input image to obtain the scales SCl and SC2 wherein the color pixel values Pxy(2) of the scale SC2 are a stronger low-pass filtered input image than the color pixel values Pxy(l) of the scale SCl. In an alternative embodiment, several low-pass filters may be used to create the scales SCl and SC2 from the input image II. More than two scales SCl and SC2 may be generated. It has to be noted that directly referring to an operation applied on the scale SCi means that this operation is applied on the pixels Pxy(i) of this scale SCi.
The comparator 3 compares the color pixel values Pxy(i) of the pixel Pxy in different scales SCi to obtain comparison values CVi which indicate the difference between the color pixel values Pxy(i) of the two scales which are compared. The comparison values CVi may be this difference or may be a function of this difference.
The weight allocator 4 allocates weights wi to the scales SCi corresponding to the comparison values CVi. Thus, in fact the weights wi are allocated to the pixel values Pxy(i) of the pixel Pxy in the different scales SCi. Two possibilities exist: the comparison
value CVi resulting from the comparison of the color pixel value Pxy(i) of the pixel Pxy in scale SCi and the color pixel value PxyO) of pixel Pxy in scale SCj indicates that the difference (or difference function) value is smaller or larger than a predetermined value. It is assumed that the scale SCj has a higher rank than the scale SCi. Thus, the scale SCj is a low- pass filtered version of the scale SCi. Usually, the scale SCj is the next higher ranked scale of the scale SCi, or said differently: j = i+1. However, this is not essential to the present invention.
If the difference is smaller than the predetermined value it is concluded that no information in the image at the particular pixel Pxy is lost due to the low-pass filtering. Consequently, it is possible to take the stronger low-pass filtered value without losing detail in the image. The advantage is that due to the stronger low-pass filtering artifacts are better removed. Thus, in this situation, the pixel value PxyO) should be favored with respect to the pixel value Pxy(i) by allocating a higher weight wj to the pixel value PxyO) than the weight wi to the pixel value Pxy(i). The actual weight wj allocated may be inverse proportional to the difference.
If the difference is larger than the predetermined value it is concluded that information in the image at the particular pixel Pxy is lost due to the low-pass filtering. To prevent loss of detail, now the pixel value PxyO) is favored over the pixel value PxyO) by allocating a higher weight wi to the pixel value Pxy(i) than the weight wj to the pixel PxyO). The actual weight wi allocated may be proportional to the difference. In an embodiment, the weights are calculated as
As explained earlier, x,y indicate a position in the input image of the pixel Pxy for which the weights wi(x,y) are calculated in the scales SCi. The number of scales SCi is indicated by S. F(D1) is a function of at least one of the comparison values (CVi). T is a predefined factor which determines a scaling of the values of the function F(Di). The weights wi are indicated by W1(X5V) or in the sum as w,(x,y) to make clear that the weights wi are determined for each one of the pixels Pxy in the input image II. It has to be noted that the determination of the weights wi need not use the same function F(Di) for the whole input image II. The input image II may be segmented, for example in blocks which are parallel
processed, or more sophisticated in areas of similar pixel values, or in objects and different functions F(Di) may be used for different areas.
S-I
The total value of all summed weights wi is W^x = V W1 , usually, in a system j=0 with a normalized response wmax is equal to 1 , which means that the DC-responses for color and luminance are identical.
It has to be noted that the first equation for i = 0 calculates a sum from j = 0 to -1. This calculation comprises no summation at all and the outcome is thus zero. For i = 1, the sum only comprises the weight wθ(x,y) allocated to the pixel value Pxy(O) in scale SCO of the pixel Pxy. The function F(D1) of at least one of the comparison values CVi may for example be defined by: F(D1) = (Pi(x,y) - Pi+l(x,y))p. Wherein Pi(x,y) and Pi+l(x,y) are the pixel values of the same pixel Pxy in two successive ones of the scales SCi and SCi+ 1. In an embodiment which is easy to implement is p = 1.
The combiner 5 combines the color pixel values Pxy(i) of the scales SCi to obtain a combined value OV. The combination is made by using the weights wi for the color pixels values Pxy(i). The combining may start after all the weights wi for all the scales SCi have been determined. Alternatively, the combining is started once the pixel color value
Pxy(i+1) of the scale SCi+1 and the pixel color value Pxy(i) of the scale SCi differ less than the predetermined amount. In an embodiment, the combining multiplies the color pixel values Pxy(i) with their associated allocated weights wi to obtain terms and calculates the combined value OV by summing the terms:
The value of the combined value OV, which is a sum of the terms, will be close to the color pixel value Pxy(O) of the scale SCO in the presence of significant local detail, while it will be closer to the color pixel value Pxy(S-l) if there is little detail.
In an embodiment, a color enhancement function Fee is applied to the combined value OV to obtain a more robust value OVR for the combined value OV. Which for wmax = 1 leads to the equation:
S-I OVR = FceipV) = Fce(∑ W1 (x, y)Pxy(i)) i=l
In another embodiment the color enhancement function Fee is applied to the each one of the color pixel values Pxy(i) to obtain adapted color pixel values P'xy(i) for each one of the scales (SCi). The adapted color pixel values P'xy(i) are multiplied with the associated allocated weights wi to the corresponding scales SCi to obtain weighted adapted color pixel values WP'xy(i) which are summed to obtain the value OVR. Which for wmax = 1 leads to the equation:
S-I
OVR = ∑Wι (x,y)Fce(Pxy(i)) ι=0
Color enhancement functions as such are well known and thus not further elucidated. In an embodiment the input signal II is separated in at least two frequency bands which may be disjunct or may partly overlap. The above described approach is applied to at least one of these frequency bands, but may also be applied to each frequency band separately.
The signal processor 6 processes the combined value OV, or the robust value OVR if a color function is applied, to obtain drive signals DR suitable to drive the display device 7.
Fig. 3 shows an example of the frequency response of the color and the luminance of the system for saturation enhancement. The frequency f is depicted along the horizontal axis and the normalized response RES is depicted along the vertical axis. The frequency response of the luminance signal Y is indicated by the curve YR and the frequency response of the color signal UV is indicated by the curve UVR. In the example shown, the DC-response of the color signal UV is higher than the DC-response of the luminance signal Y to obtain a color boost. Or said differently the color pixel value gain for DC or low frequencies is higher than the luminance pixel value gain of the pixel Pxy. The color signal response UVR decreases with frequency due to the low-pass filtering in the scales. In the example shown, the luminance signal response YR increases with higher frequency to finally steeply drop for high frequencies. The initial increase provides a contrast boost.
The example shown in Fig. 3 elucidates that a color boost is obtained by having the DC-response higher than the DC-response for luminance, where both depend on the amount of detail present in the image. If a low amount of detail is present, the color response UVR starts decreasing soon because the color pixel values of high ranked scales are favored to minimize the visibility of disturbances. If a high amount of detail is present, the color response UVR is at a high level up to relatively high frequencies because the color
pixel values of lower ranked scales are favored and thus less low-pass filtering is applied to retain the details.
Fig. 4 shows a camera application. The camera 10 comprises an image sensor which directly or after suitable signal processing supplies the input image II to the color enhancement system SYS.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Claims
1. A method of color enhancement comprising: receiving (1) an input image (II) composed of pixels, generating (2) at least three scales (SCO, SCl, SC2), the lowest ranked scale (SCO) being the input image (II), the lowest but one ranked scale (SCl) being a low-pass filtered version of the input image (II), the lowest but two ranked scale (SC2) being a low- pass filtered version of the lowest but one scale (SCl) or a low passed version of the input image (II) being stronger low-pass filtered than the input image (II) for the lowest but one scale (SCl), comparing (3) color pixel values (Pxy(O), Pxy(l), Pxy(2)) defining a chrominance of a particular pixel (Pxy) of the input image (II) of different scales (SCO, SCl, SC2) to obtain comparison values (CVi), allocating (4) weights (wθ, wl, w2) to the scales (SCO, SCl, SC2) for favoring a first color pixel value (Pxy(l)) of the particular pixel (Pxy) in one of the scales (SCl) over a second color pixel value (Pxy(2)) of the particular pixel (Pxy) in another one of the scales (SC2) having a higher rank if the first color pixel value (Pxy(l)) and the second color pixel value (Pxy(2)) differ more than a predetermined amount, and for favoring the second color pixel value (Pxy(2)) over the first color pixel value (Pxy(l)) if the first pixel color value (Pxy(l)) and the second pixel color value (Pxy(2)) differ less than the predetermined amount, and combining (5) color pixel values (Pxy(O), Pxy(l), Pxy(2)) of the particular pixel (Pxy) of the different scales (SCO, SCl, SC2) weighted in accordance with the allocated weights (wθ, wl, w2).
2. A method of color enhancement as claimed in claim 1 , wherein the combining comprises calculating a sum of multiplications of a color enhancement function applied to the each one of the color pixel values (Pxy(O), Pxy(l), Pxy(2)) on the one hand and their associated allocated weights (wθ, wl, w2) on the other hand.
3. A method of color enhancement as claimed in claim 1, wherein the allocating (4) weights allocates the weights (wθ, wl, w2) to obtain a sum of allocated weights (wθ, wl, w2) causing a DC-response for the color of the pixel (Pxy) larger than a DC-response for a luminance of the pixel (Pxy).
4. A method of color enhancement as claimed in claim 1, wherein the allocating (4) the weights (wθ, wl, w2) comprises calculating:
W < 1 (x,y) = min(TxF(Di),wm!a - ∑wJ (x,y)) if O ≤ i < S - 1 j=0 ι-\ w1 (x,y) = wmsκ - ∑wJ (x,y) ifi = S j=0 wherein the weights (wθ, wl, w2) are indicated by W1(X5V) or w,(x,y), T is a predetermined factor, x,y indicate a position of the same one of the pixels (Pxy), F(D1) is a function of at
S-I least one of the comparison values (CVi), and the sum weight W1113x = V wi . j=0
5. A method of color enhancement as claimed in claim 4, wherein the sum weight which defines the DC-response for the color of the pixel (Pxy) is larger than a DC- response for a luminance of the pixel (Pxy).
6. A method of color enhancement as claimed in claim 4, wherein the function (F(D1)) of at least one of the comparison values (CVi) is: F(D1) = Pi(x,y) - Pi+l(x,y), wherein Pi(x,y) and Pi+l(x,y) are the pixel values of the particular pixel Pxy in two successive ones of the scales (SCO, SCl, SC2).
7. A method of color enhancement as claimed in claim 1 , wherein the combining is started once the first pixel color value (Pxy (I)) and the second pixel color value (Pxy(2)) differ less than the predetermined amount.
8. A method of color enhancement as claimed in claim 1 , wherein the combining comprises calculating a sum of the multiplication of the color pixel values (Pxy(O), Pxy(l), Pxy(2)) and their associated allocated weights (wθ, wl, w2) to obtain a sum value.
9. A method of color enhancement as claimed in claim 7, wherein the combining comprises a color enhancement function (6) applied to the sum value.
10. A method of color enhancement as claimed in claim 1, wherein the method as claimed in claim 1 is applied at least two times in corresponding at least two frequency bands of the input image (II).
11. A system (SYS) for color enhancement comprising: scale generator (2) for receiving (1) an input image (II) composed of pixels to generate at least three scales (SCO, SCl, SC2), the lowest ranked scale (SCO) being the input image (II), the lowest but one ranked scale (SCl) being a low-pass filtered version of the input image (II), the lowest but two ranked scale (SC2) being a low-pass filtered version of the lowest but one scale (SCl) or a low passed version of the input image (II) being stronger low-pass filtered than the input image (II) for the lowest but one scale (SCl), a comparator (3) for comparing color pixel values (Pxy(O), Pxy(l), Pxy(2)) defining a chrominance of a particular pixel (Pxy) of the input image (II) in different scales (SCO, SCl, SC2) to obtain comparison values (CVi), a weight allocator (4) for allocating weights (wθ, wl, w2) to the scales (SCO, SCl, SC2) for favoring a first color pixel value (Pxy(l)) of the particular pixel (Pxy) in one of the scales (SCl) over a second color pixel value (Pxy(2)) of the same one of the pixels (Pxy) in another one of the scales (SC2) having a higher rank if the first color pixel value (Pxy(l)) and the second color pixel value (Pxy (2)) differ more than a predetermined amount, and for favoring the second color pixel value (Pxy(2)) over the first color pixel value (Pxy(l)) if the first pixel color value (Pxy (I)) and the second pixel color value (Pxy (2)) differ less than the predetermined amount, and a combiner (5) for combining for the particular pixel (Pxy) the color pixel values (Pxy(O), Pxy(l), Pxy(2)) of the different scales (SCO, SCl, SC2) weighted in accordance with the allocated weights (wθ, wl, w2) to obtain a combined value (OV).
12. A display apparatus comprising the system (SYS) as claimed in claim 11, a display device (7), and a signal processor (6) for receiving the combined value (OV) to supply drive signals (DS) to the display device (7).
13. A camera comprising an image sensor for supplying the input signal (II) to the scale generator (2) of the system (SYS) as claimed in claim 11.
14. Computer program product comprising code enabling a processor to execute the method of claim 1.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP06123167 | 2006-10-30 | ||
EP06123167.6 | 2006-10-30 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2008053408A2 true WO2008053408A2 (en) | 2008-05-08 |
WO2008053408A3 WO2008053408A3 (en) | 2009-02-05 |
Family
ID=39344678
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2007/054341 WO2008053408A2 (en) | 2006-10-30 | 2007-10-25 | Color saturation enhancement |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2008053408A2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010084431A1 (en) * | 2009-01-20 | 2010-07-29 | Koninklijke Philips Electronics N.V. | Method and apparatus for generating enhanced images |
US9691138B2 (en) | 2013-08-30 | 2017-06-27 | Google Inc. | System and method for adjusting pixel saturation |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4523230A (en) * | 1983-11-01 | 1985-06-11 | Rca Corporation | System for coring an image-representing signal |
EP1094419A1 (en) * | 1999-10-19 | 2001-04-25 | Agfa-Gevaert N.V. | Method of enhancing colour images |
-
2007
- 2007-10-25 WO PCT/IB2007/054341 patent/WO2008053408A2/en active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4523230A (en) * | 1983-11-01 | 1985-06-11 | Rca Corporation | System for coring an image-representing signal |
EP1094419A1 (en) * | 1999-10-19 | 2001-04-25 | Agfa-Gevaert N.V. | Method of enhancing colour images |
Non-Patent Citations (5)
Title |
---|
DIPPEL S ET AL: "MULTISCALE CONTRAST ENHANCEMENT FOR RADIOGRAPHIES: LAPLACIAN PYRAMID VERSUS FAST WAVELET TRANSFORM" IEEE TRANSACTIONS ON MEDICAL IMAGING, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 21, no. 4, 1 April 2002 (2002-04-01), pages 343-353, XP001115731 ISSN: 0278-0062 * |
KALB A: "Noise Reduction in Video Images Using Coring on QMF Pyramids" THESIS, [Online] 1991, pages 1-61, XP007906223 Retrieved from the Internet: URL:http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.43.6615> * |
POWELL P G ET AL: "A METHOD FOR THE DIGITAL ENHANCEMENT OF UNSHARP, GRAINY PHOTOGRAPHIC IMAGES" INTERNATIONAL CONFERENCE ON ELECTRONIC IMAGE PROCESSING, XX, XX, 26 July 1982 (1982-07-26), pages 179-183, XP001051936 * |
STAHL M ET AL: "Noise-resistant weak-structure enhancement for digital radiography" PROCEEDINGS OF THE SPIE - MEDICAL IMAGING 1999: IMAGE PROCESSING 22-25 FEB. 1999 SAN DIEGO, CA, USA, vol. 3661, 1 January 1999 (1999-01-01), pages 1406-1417, XP000925265 The International Society for Optical Engineering SPIE-Int. Soc. Opt. Eng USA ISSN: 0277-786X * |
TOET A: "Multiscale color image enhancement" IMAGE PROCESSING AND ITS APPLICATIONS, 1992., INTERNATIONAL CONFERENCE ON MAASTRICHT, NETHERLANDS, LONDON, UK,IEE, UK, 1 January 1992 (1992-01-01), pages 583-585, XP006500244 ISBN: 978-0-85296-543-6 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010084431A1 (en) * | 2009-01-20 | 2010-07-29 | Koninklijke Philips Electronics N.V. | Method and apparatus for generating enhanced images |
US9691138B2 (en) | 2013-08-30 | 2017-06-27 | Google Inc. | System and method for adjusting pixel saturation |
Also Published As
Publication number | Publication date |
---|---|
WO2008053408A3 (en) | 2009-02-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7599578B2 (en) | Apparatus, program, and method for image tone transformation, and electronic camera | |
EP1667066B1 (en) | Visual processing device, visual processing method, visual processing program, integrated circuit, display device, imaging device, and mobile information terminal | |
US9600902B2 (en) | Image processing apparatus, image processing method, program and imaging apparatus | |
CN101512597B (en) | Method and apparatus for image enhancement | |
US8831346B2 (en) | Image processing apparatus and method, and program | |
CN105850114A (en) | Method for inverse tone mapping of an image | |
US7570829B2 (en) | Selection of alternative image processing operations to maintain high image quality | |
EP2175416A1 (en) | Method and system for image deblurring | |
CN101360250A (en) | Immersion generation method and system and factor control, content analysis and parameter prediction method | |
JP2022130642A (en) | Adaptive Bilateral (BL) Filtering for Computer Vision | |
CN107871303A (en) | An image processing method and device | |
WO2012015020A1 (en) | Method and device for image enhancement | |
US7630573B2 (en) | Noise reduction apparatus and method | |
EP1333660A2 (en) | Adjusting the color, brightness, and tone scale of rendered digital images | |
US9100570B2 (en) | Tonal stabilization of video | |
WO2008053408A2 (en) | Color saturation enhancement | |
WO2002062055A1 (en) | Image processing circuit | |
JP2002281312A (en) | Device, method and program for processing image | |
US9111362B2 (en) | Method, system and apparatus for applying histogram equalization to an image | |
EP1991008A1 (en) | Image processing device for correcting image massive feeling, image processing program, image processing method, and electronic camera | |
JP5593515B2 (en) | Image enhancement method, image enhancer, image rendering system, and computer program | |
US7894686B2 (en) | Adaptive video enhancement gain control | |
EP4068196A1 (en) | High dynamic range tone mapping | |
JPH11285019A (en) | Image processing unit and medium wherein image processing program is stored | |
JP2006519447A (en) | Variable contrast mapping of digital images |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07826864 Country of ref document: EP Kind code of ref document: A2 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 07826864 Country of ref document: EP Kind code of ref document: A2 |