JP2013187723A - Color conversion device, color sub-sampling apparatus, and program thereof - Google Patents

Abstract

Provided is a technique for suppressing image quality deterioration during sub-sampling by performing color conversion in accordance with a statistical bias in the color of an input image.
A color sub-sampling device (2) includes loss component extraction means (10, 20 and 30) for extracting loss components (difference images (P, Q, R)) lost during sub-sampling of input images A, B and C, and a difference Principal component analysis means 40 that performs principal component analysis on the vector sequence S (x, y) at each pixel position of the images P, Q, and R, and a color based on the coupling coefficient vector w _k obtained by the principal component analysis means 40 A color conversion means (color) comprising: a color space conversion means 50 that generates a conversion matrix M, multiplies the color conversion matrix M by the color components A, B, and C and converts them into new color components X, Y, and Z, respectively. Conversion device) 1 and among the color components X, Y, Z, the pixels of the images Y, Z comprising the first and second color components Y, Z from the one having the smallest eigenvalue obtained by the principal component analysis means 40 are located. Pixel thinning means 60 and 70 for thinning out a fixed amount.
[Selection] Figure 2

Description

  The present invention relates to preprocessing for encoding a color image, and more particularly to a technique for performing color conversion and color subsampling.

  In image coding, when a color image is handled, conversion into a luminance 1 component and a color difference 2 component is performed. For example, when converting three components of red, blue, and green into a luminance 1 component and a color difference 2 component, ITU-R BT. 601 standard color conversion matrix, ITU-R BT. A color conversion matrix defined in the 709 standard is used. Since the luminance is more visually noticeable than the color difference, the color conversion matrix as described above is often set so that the image quality degradation of the luminance is less than the color difference due to encoding. For example, without changing the number of pixels of the luminance image, the number of pixels is often thinned out to 1/2 or 1/4 for only the color difference image. This thinning operation is called color subsampling.

In the method of color subsampling, a signal obtained by thinning the number of pixels of the color difference image by 1/2 in only the horizontal direction is called 4: 2: 2 format, and the number of pixels of the color difference image is set in the horizontal direction and the vertical direction, respectively. The signal thinned by 1/2 is called 4: 2: 0 format. On the other hand, a signal composed of the same number of horizontal and vertical pixels having the same luminance and color difference is called a 4: 4: 4 format.
Also, in digital cameras and videos, white balance is adjusted so that the desired color tone is obtained. Conventionally, various techniques for automating white balance adjustment have been proposed (see Patent Documents 1 and 2).

Japanese Patent No. 4764163 Japanese Patent No. 4815267

  However, in the conventional standard described above, the same color conversion matrix is always used regardless of the input image. For this reason, for example, a fine texture (such as a fine stripe) composed of colors having different hues appears as a fine texture on the color difference image. When this color sub-sampling is performed, this texture is crushed or moire is generated, resulting in significant image quality degradation. Also, when the color of the input image is deviated from the standard color (for example, when the color temperature is different from the encoding method assumption such as shooting in the color of the sunset, or when color cast occurs) In addition, the power of the color difference image becomes larger than expected, and the adverse effect of color sub-sampling becomes conspicuous.

  Further, according to the technology for automating white balance adjustment as shown in Patent Documents 1 and 2, since the color of the subject can be corrected to a predetermined color, particularly the influence of color temperature difference and color cast is excluded. However, since the fineness of the texture of the subject is not taken into consideration, image quality deterioration after color subsampling cannot be prevented. Also, when the color balance is intentionally lost due to the effect of the production, the color difference component may deviate from the value assumed by the encoding device, which may lead to a decrease in encoding efficiency.

  Accordingly, an object of the present invention is to provide a technique for suppressing image quality deterioration during sub-sampling by performing color conversion according to a statistical bias in the color of an input image.

  The present invention has been made to solve the above-described problems. First, the color conversion apparatus according to claim 1 is a new color conversion device that reduces deterioration in image quality during sub-sampling for each color component of a color image. The color conversion device converts each color component into a color component, and includes a loss component extraction unit, a principal component analysis unit, and a color space conversion unit.

According to such a configuration, the color conversion apparatus extracts the loss component lost at the time of sub-sampling from each input image including the color components of the color image by the loss component extraction unit.
The color conversion apparatus performs principal component analysis on the pixel values of the respective loss components extracted by the loss component extraction unit by the principal component analysis unit.
Further, the color conversion device may convert each color component from a color space different from the color space of each color component by a color conversion matrix based on a coupling coefficient obtained by the principal component analysis by the principal component analysis unit. Each of the new color components in the space is converted into an image composed of the new color components. Thereby, the color conversion apparatus can represent each color component of the input image by a color component belonging to a new color space different from the color space to which the color component belongs. In this new color space, each axis has an orthogonal basis sequentially from the axis with the largest loss when the input image is subsampled, so that the color component that is easily affected by thinning is changed to the color component that is not easily affected by thinning. It is possible to perform color separation that is ranked as follows.

  According to a second aspect of the present invention, in the color conversion device according to the first aspect, the loss component extraction unit includes a pixel thinning unit, an interpolation unit, and a subtraction unit.

  According to such a configuration, the color conversion device thins out the pixels of the input image by a predetermined amount by the pixel thinning unit according to a predetermined rule. In addition, the color conversion apparatus generates an interpolation image in which an input image obtained by thinning a predetermined amount of pixels by the pixel thinning unit is returned to the original resolution by the interpolation unit. Further, the color conversion device generates a difference image between each input image and each interpolation image by the subtracting unit. In this way, by performing processing that simulates sub-sampling for each color component of the input image, it is possible to extract color components that are lost when sub-sampling is actually performed.

  According to a third aspect of the present invention, the color conversion device according to the first or second aspect further includes gain multiplication means.

  According to this configuration, the color conversion apparatus is configured to extract the loss component from any one or more of the loss components extracted by the loss component extraction unit by the gain multiplication unit. Is multiplied by a preset gain for the color component of the input image.

  For example, if human visual characteristics are taken into account, if color components with high visual sensitivity are lost due to sub-sampling, it is more likely that image quality deterioration will be felt than if color components with low visual sensitivity are lost. Conceivable. For this reason, for example, it is preferable to set the importance to a high visual sensitivity, set a large gain to a color component with high visual sensitivity, and set a small gain to a color component with low visual sensitivity. The multiplication result by the gain multiplication means is output to the principal component analysis means, so that the importance of each color component can be reflected in the principal component analysis, and consequently the importance of each color component is reflected in the result of color conversion. can do.

  According to a fourth aspect of the present invention, there is provided a color sub-sampling device including the color conversion device according to any one of the first to third aspects and a pixel thinning unit.

  According to such a configuration, the color sub-sampling device has a predetermined number of images from the smaller eigenvalues obtained by the principal component analysis means among the images composed of the new color components generated by the color conversion device. The pixels of the corresponding image are thinned out by a predetermined amount according to a predetermined rule by the pixel thinning means provided corresponding to the image.

  This eigenvalue is a value obtained corresponding to each principal component by performing principal component analysis on a color component estimated to be lost when the input image is subsampled. This eigenvalue corresponds to the variance of the principal component and represents how much the principal component holds the original data information. Therefore, it can be said that the larger the eigenvalue, the easier the color component is lost, and the smaller the eigenvalue, the less likely the color component is lost. In this way, the sub-sampling device generates an image with little deterioration by thinning out the low-importance color components and not thinning out the high-importance color components among the new color components by the pixel thinning-out means. be able to.

  According to a fifth aspect of the present invention, there is provided a color conversion program comprising: a computer for converting each color component of a color image into a new color component that reduces image quality deterioration during sub-sampling; The configuration is made to function as a principal component analysis unit and a color space conversion unit.

According to this configuration, the color conversion program extracts the loss component lost at the time of sub-sampling from each input image composed of the color components of the color image by the loss component extraction unit.
The color conversion program performs principal component analysis on the pixel value of each loss component extracted by the loss component extraction means by the principal component analysis means.
Further, the color conversion program may cause each color component to be different from the color space of each color component by a color conversion matrix based on a coupling coefficient obtained by the principal component analysis by the principal component analysis unit. Each of the new color components in the space is converted into an image composed of the new color components.

  The color sub-sampling program according to claim 6 comprises the new color components after each color component of the color image is converted into a new color component that reduces image quality deterioration during sub-sampling. In order to thin out a predetermined amount of pixels from an image according to a predetermined rule, the computer is configured to function as a loss component extracting unit, a principal component analyzing unit, a color space converting unit, and a pixel thinning unit.

According to this configuration, the color sub-sampling program extracts the loss component lost at the time of sub-sampling from each input image composed of each color component of the color image by the loss component extraction unit.
The color sub-sampling program performs principal component analysis on the pixel value of each loss component extracted by the loss component extraction unit by the principal component analysis unit.
Further, the color sub-sampling program is configured such that each color component is different from the color space of each color component by a color conversion matrix based on a coupling coefficient obtained by principal component analysis by the principal component analysis unit by the color space conversion unit. Each of the new color components in the color space is converted into an image composed of the new color components.
Still further, the color sub-sampling program may include a predetermined number of the smallest number of eigenvalues obtained by the principal component analysis means among the images composed of the new color components generated by the color space conversion means. Pixels corresponding to the image are thinned by a predetermined amount according to a predetermined rule by the pixel thinning means provided corresponding to the image.

  According to the first and fifth aspects of the present invention, after considering the statistical color bias of each color component of the input image, the color component is expressed by a new color component, thereby thinning out by sub-sampling. It is possible to perform color separation in which priority is given to color components that are easily affected and color components that are not easily affected by thinning. For this reason, each color component of the input image can be converted into a new color component that suppresses deterioration in image quality when subsampling is performed.

  According to the second aspect of the present invention, the loss component lost when the sub-sampling is performed can be extracted in advance by performing the process of simulating the sub-sampling on the input image. By using the extraction result of the loss component, it is possible to convert each color component of the input image into a color component that suppresses image quality degradation when sub-sampling is performed.

According to the third aspect of the invention, from the new color component obtained by converting the color space of each color component of the input image, the less important color component is thinned out, and the more important color component is thinned out. Therefore, an image with good image quality with little deterioration can be generated.
According to the fourth and sixth aspects of the present invention, since each color component of the input image can be weighted according to the importance, the loss of the color component that easily affects the image quality after sub-sampling is suppressed. be able to.

It is a conceptual diagram for demonstrating the outline | summary of the color conversion by the color conversion apparatus which concerns on 1st embodiment of this invention. It is a block diagram which shows the structural example of the color conversion apparatus or color subsampling apparatus which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of the color conversion apparatus or color subsampling apparatus which concerns on 1st embodiment of this invention. It is a block diagram which shows the other structural example of the color conversion apparatus or color subsampling apparatus which concerns on 2nd embodiment of this invention. It is a flowchart which shows operation | movement of the color conversion apparatus or color subsampling apparatus which concerns on 2nd embodiment of this invention.

<First embodiment>
[Outline of color subsampling]
Hereinafter, a color conversion device and a color subsampling device according to a first embodiment of the present invention will be described. First, an outline of a color subsampling method performed by the color subsampling apparatus according to the first embodiment will be described with reference to FIG.

  As shown in FIG. 1, the color sub-sampling apparatus inputs images A, B, and C divided for each color component (here, color components A, B, and C) that constitute a color image. The color sub-sampling device extracts a loss component lost when sub-sampling is performed on the input images A, B, and C in an image format. Here, the color component of a color image composed of three color components is converted, but the color component of a color image composed of four or more color components may be converted.

  The color sub-sampling device simulates sub-sampling by thinning out a predetermined amount of pixels from the input images A, B, and C, and generates thinned images A ′, B ′, and C ′. Next, the color sub-sampling device generates interpolated images A ″, B ″ and C ″ in which the thinned images A ′, B ′ and C ′ are returned to the number of pixels of the input images A, B and C. The color sub-sampling device generates difference images P, Q, and R obtained by taking differences between the interpolated images A ″, B ″, and C ″ and the input images A, B, and C. That is, the difference images P, Q, and R are a set for each color component of the loss component that is lost when the sub-sampling is performed on the input images A, B, and C.

  Further, the color sub-sampling device performs principal component analysis of the difference images P, Q, and R corresponding to the input images A, B, and C to obtain a coupling coefficient for each principal component. Then, the color conversion device ranks the color conversion matrix M ranked from the color coefficient estimated to be lost at the time of sub-sampling from the color component having a large influence on the image quality to the component having a small influence on the image quality. Is generated. For example, a color component with high sensitivity in consideration of human visual characteristics, that is, a color component that tends to feel that the image quality has deteriorated due to loss is a color component that has a large effect on image quality, while a color component with low sensitivity In other words, even if lost, it is possible to make a color component that hardly perceives that the image quality has deteriorated as a color component having a small influence on the image quality.

  The color sub-sampling device multiplies the color components A, B, and C of the input images A, B, and C by the color conversion matrix M to thereby obtain the color components A, B, and C of the input images A, B, and C. Then, the image is converted into images X, Y and Z composed of new color components X, Y and Z in a new color space different from the color spaces of the color components A, B and C. In this way, the color sub-sampling device generates the color conversion matrix M based on the thinned images A ′, B ′, and C ′ generated by simulating sub-sampling on the input images A, B, and C. By multiplying the color components A, B, and C of the input images A, B, and C by this color conversion matrix M, the color components A, B, and C of the input images A, B, and C are converted into image quality at the time of sub-sampling. It is possible to convert to new color components X, Y, and Z that are less affected by deterioration.

  Then, the color sub-sampling device selects the images Y, Y, and Z, which are less important color images (here, the color components Y, Z) out of the new images X, Y, and Z. The pixels of Z are thinned by a predetermined amount to generate thinned images Y ′ and Z ′. The image X and the thinned images Y ′ and Z ′ can be regarded as a 4: 2: 2 format or a 4: 2: 0 format. The image X and the thinned images Y ′ and Z ′ are input to the image encoding device.

  As described above, the feature of the color subsampling apparatus of the present invention is that a color conversion matrix is not directly generated from each color component of the input image, but subsampling is simulated for each color component of the input image before subsampling. By performing the above processing, a thinned image is generated, and a color conversion matrix is generated from a difference image between the thinned image and the input image. By multiplying each color component of the input image by this color conversion matrix, it is possible to generate a new color component considering image quality degradation when sub-sampling is performed on the input image.

Next, the configuration of the color conversion device and the color subsampling device according to the first embodiment of the present invention will be described with reference to FIG. Here, a color sub-sampling device provided with a color conversion device will be described.
As shown in FIG. 2, the color sub-sampling device 2 according to the first embodiment of the present invention includes a color conversion means (color conversion device) 1 and pixel thinning means 60 and 70.

[Configuration of color conversion means]
As shown in FIG. 2, the color conversion unit 1 includes loss component extraction units 10, 20 and 30, a principal component analysis unit 40, and a color space conversion unit 50.

The loss component extraction means 10, 20 and 30 extract, in an image format, loss components lost during sub-sampling of the input images A, B and C for the input images A, B and C composed of the color components A, B and C. Is. This loss component extraction means is provided in a number corresponding to the number of components of the color image. Here, since the color image is composed of three color components A, B, and C, three loss component extraction means are provided correspondingly. Here, the color components A, B and of C image A, but has decided to enter the B and C, instead of this, for example, the luminance component Y and color difference component C _B, such image consisting of C _R It is good also as inputting.

  Hereinafter, detailed configuration examples of the loss component extraction means 10, 20, and 30 will be described. However, since the loss component extraction means 10, 20, and 30 have the same components, in FIG. A detailed configuration will be described with reference to FIG.

Here, the loss component extraction unit 10 includes a pixel thinning unit 11, an interpolation unit 12, and a subtraction unit 13.
The pixel thinning means 11 is for thinning out a predetermined amount of pixels of the input image A according to a predetermined rule to generate a thinned image A ′ having a smaller number of pixels. For example, the pixel thinning unit 11 thins the number of pixels of the input image A by half in only the horizontal direction. Note that a thinning filter (typically a low-pass filter) may be applied during thinning. For example, the pixel decimation means 11 decimates the number of pixels in the horizontal direction of the input image A by the following formulas (1) and (2).

  Here, in Expressions (1) and (2), x and y are coordinates in the input image A, and D (i) is a filter coefficient. For example, in A (2x + i, y) on the right side of Expression (1), when a pixel outside the area of the input image A is referred to, the pixel position nearest to the pixel position in the area of the input image A By extrapolating the pixel value of the effective pixel, the pixel value at the pixel position can be defined.

  Further, for example, the pixel thinning unit 11 thins the total number of pixels to 1/4 by thinning the number of pixels of the input image A to 1/2 in the horizontal direction and 1/2 in the vertical direction, thereby thinning the image A ′. May be generated. Specifically, the pixel decimation unit 11 decimates the total number of pixels of the input image A to ¼ by the following equations (3) and (4), for example.

  Here, in Expressions (3) and (4), x and y are coordinates in the input image A, and D (i) is a filter coefficient. The thinned image A ′ thus generated is output to the interpolation unit 12.

  The interpolation unit 12 enlarges the thinned image A ′ by interpolation, and generates an interpolated image A ″ that is combined with the number of pixels of the input image A. For example, the interpolating unit 12 includes the equations (1) and (2). ), The number of pixels of the input image A is decimated to ½ in the horizontal direction, and an interpolated image A ″ is generated by the following equations (5) and (6). Note that when either the horizontal coordinate ξ or the vertical coordinate η of the thinned image A ′ is a non-integer, it is defined as A ′ (ξ, η) = 0. Further, when both the horizontal coordinate ξ or the vertical coordinate η of the thinned image A ′ are integers and either one is outside the area of the thinned image A ′, for example, the coordinates (ξ , Η) can be defined as the pixel value of the thinned image A ′ (ξ, η) by extrapolating the pixel value of the nearest effective pixel.

  Further, in the pixel thinning means 11 described above, when the number of pixels of the input image A is thinned to 1/2 in the horizontal direction and 1/2 in the vertical direction by the equations (3) and (4), the interpolation means 12 performs enlargement of 2 times in the horizontal direction and 2 times in the vertical direction by interpolation processing. In this case, the interpolation unit 12 interpolates the thinned image A ′ by the following equations (7) and (8), for example. The interpolated image A ″ generated in this way is output to the subtracting means 13.

The subtracting means 13 calculates a difference between the input image A and the interpolated image A ″ and outputs the result as a difference image P.
For example, the subtracting unit 13 generates the difference image P by the following formula (9).

  Similar to the loss component extraction unit 10 described above, the loss component extraction unit 20 generates a difference image Q from the input image B, and the loss component extraction unit 30 generates a difference image R from the input image C. The difference images P, Q, and R correspond to the loss components represented in the image format. The difference images P, Q, and R generated in this way are output to the principal component analysis means 40, respectively.

The principal component analysis means 40 performs principal component analysis on the vector S (x, y) (pixel value) at each pixel position of the difference images P, Q, and R represented by the following expression (10), A coupling coefficient vector for the principal component is obtained. In the following, it is assumed that the coupling coefficient vector for the k-th principal component (k is a natural number) is w _k, and this coupling coefficient vector w _k is a column vector.

  The principal component analysis by the principal component analysis means 40 may be performed with one image frame as one unit, a partial area obtained by dividing one image frame may be performed as one unit, or a plurality of image frames may be combined into one unit. As well as

The principal component analysis means 40 generates a color conversion matrix M represented by the following equation (11) based on the coupling coefficient vector w _k obtained by performing principal component analysis for each unit as described above, for example. To do.

In equation (11), T represents transposition of a vector or matrix. The coupling coefficient vector w _k is a unit eigenvector that takes the k-th largest eigenvalue of the variance-covariance matrix of the vector S in one image frame.
That is, in Equation (11), w ₁ is the color component having the largest eigenvalue, in other words, the color component that is most prominent in loss when sub-sampling is performed, and w ₂ has the second largest eigenvalue. It represents the color component, w ₃ is the most eigenvalue small color component, in other words, represents the most loss less noticeable color component when the subsampling is performed.
The color conversion matrix M calculated in this way is output to the color space conversion means 50.

  The color space conversion means 50 multiplies the input image A, B, and C composed of the color components A, B, and C by a color conversion matrix M by a color vector for each pixel, thereby creating a new color component X, An output image X, Y, and Z consisting of Y and Z is generated. That is, the color space conversion means 50 performs an operation as shown in the following equation (12).

[Configuration of pixel thinning means]
The pixel thinning means 60 and 70 respectively input output images Y and Z composed of new color components Y and Z generated by color conversion of the input images B and C by the color space conversion means 50 of the color conversion means 1, respectively. The pixels of the output images Y and Z are thinned out by a predetermined amount and output to the outside as thinned images Y ′ and Z ′.
Here, the eigenvalue is a set of color components estimated to be lost when the input images A, B, and C are subsampled by the principal component analysis unit 40 of the color conversion unit 1 as described above. It is a value obtained corresponding to each principal component by performing principal component analysis on the difference images P, Q and R. This eigenvalue corresponds to the variance of the principal component and represents how much the principal component holds the original data information. Here, as described above in the description of the color conversion means 1, the color component X has the largest eigenvalue, and the eigenvalues become smaller in the order of the color components Y and Z. That is, it can be said that the color component X (color component A) is the color component in which loss is most noticeable when subsampling is performed.

  For this reason, thinning out a pixel of a new color component X having the largest eigenvalue leads to degradation of image quality. Therefore, no pixel thinning means corresponding to the new color component X is provided here. Therefore, the image X composed of the new color component X generated by the color space conversion unit 50 of the color conversion unit 1 is output to the outside as it is.

Here, the pixel thinning means 60 and 70 are completely the same as the pixel thinning means 11 included in the loss component extraction means 10 of the color conversion means 1 described above.
Here, two pixel thinning means are provided, but the number of pixel thinning means is appropriately changed according to a desired image format.

[Operation of color sub-sampling device]
Next, the operation of the color subsampling apparatus 2 according to the first embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 3, the color sub-sampling device 2 includes an input image A composed of the color component A, a color by the pixel thinning unit 11 of the loss component extraction unit 10, 20 and 30 of the color conversion unit 1 from the outside. The input image B composed of the component B and the input image C composed of the color component C are respectively input (step S101). The pixel decimation means 11 decimates the pixels of the input images A, B, and C by a predetermined amount by, for example, the expressions (1) and (2) or the expressions (3) and (4), and the decimation images A ′, B ′ and C ′ is generated (step S102). The pixel thinning unit 11 outputs the generated thinned images A ′, B ′, and C ′ to the interpolation unit 12.

  When the interpolation unit 12 inputs the thinned images A ′, B ′, and C ′ from the pixel thinning unit 11, the thinned images A ′, B ′, and C ′ are converted into the equations (5), (6), or (7). , (8) are interpolated and enlarged to generate interpolated images A ″, B ″ and C ″ combined with the number of pixels of the input image A (step S103). The interpolation means 12 generates the generated interpolated images A ″, B ″ and C ″ are output to the subtracting means 13.

  The subtracting means 13 inputs the interpolated images A ″, B ″, and C ″ from the interpolating means 12 and inputs the input images A, B, and C from the outside, and the interpolated image A ″ and the input image according to the equation (9). Differences are taken between A, the interpolated image B ″ and the input image B, and between the interpolated image C ″ and the input image C to generate difference images P, Q, and R (step S104). The subtracting unit 13 outputs the generated difference images P, Q, and R to the principal component analyzing unit 40.

The principal component analysis means 40 receives the difference images P, Q, and R from the loss component extraction means 10, 20, and 30, respectively, and the vector S (x, y) at each pixel position of the difference images P, Q, and R (described above). Principal component analysis is performed on the equation (10) to obtain a coupling coefficient vector w _k for each principal component (step S105). Further, the principal component analyzing means 40 generates a color conversion matrix M shown in the above equation (11) based on the coupling coefficient vector w _k obtained in step S104 (step S106). The principal component analysis means 40 outputs the generated color conversion matrix M to the outside of the color space conversion means 50 and the color sub-sampling device 2.

The color space conversion means 50 receives the color conversion matrix M from the principal component analysis means 40, inputs images A, B, and C from the outside, and multiplies the input images A, B, and C by the color conversion matrix M, respectively. Then, output images X, Y, and Z composed of new color components X, Y, and Z are generated (step S107). The color space conversion means 50 outputs the generated output image X to the outside of the color subsampling device 2 (step S109), and outputs the output images Y and Z to the pixel thinning means 60 and 70, respectively.

Subsequently, the pixel thinning means 60 and 70 respectively receive the output images Y and Z output by the color space conversion means 50 of the color conversion means 1. Then, the pixel thinning means 60 and 70 perform the same pixel thinning processing on the output images Y and Z as the pixel thinning means 11 to thin out pixels by a predetermined amount to generate thinned images Y ′ and Z ′, respectively (step S108). . The pixel thinning means 60 and 70 output the generated thinned images Y ′ and Z ′ to the outside of the color sub-sampling device 2 (step S109).
The color subsampling device 2 operates as described above.

  Thus, the image X output to the outside from the color space conversion unit 50 of the color conversion unit 1 and the thinned images Y ′ and Z ′ output to the outside from the pixel thinning units 60 and 70 are, for example, 4: It can be regarded as an image after thinning in the 2: 2 format or 4: 2: 0 format. Such an image X and thinned images Y ′ and Z ′ are input to an image encoding device (not shown) and encoded.

In addition, the color space conversion unit 50 of the color conversion unit 1 separately notifies the color conversion matrix M to, for example, a not-shown receiving device (for example, a decoder) (for example, transmits it as header information or side information), so that it is not illustrated. For example, the result decoded by the receiving device is converted back into three primary colors composed of color components A, B and C (for example, R, G and B), or converted into a color representation that matches the color system of the display device. It becomes possible to do. For example, in order to return the image X and the thinned images Y ′ and Z ′ to the three primary colors composed of the color components A, B, and C, first, the loss component extracting unit 10 of the color converting unit 1 for the thinned images Y ′ and Z ′. The interpolation enlargement is performed by the same method as that of the interpolation means 12 to generate the interpolated images Y ″ and Z ″. Then, the color vector [X, Y ″, Z ″] composed of the respective color components of the image X and the interpolated images Y ″ and Z ″ is multiplied by the inverse matrix M ⁻¹ of the color conversion matrix from the left side. The color components A, B, and C can be restored.

According to the color sub-sampling device 2 of the present invention described above, the following operational effects can be obtained.
According to the color sub-sampling device 2 according to the first embodiment of the present invention, the color conversion means (color conversion device) 1 ranks color components that are easily affected by thinning by sub-sampling from color components that are not easily affected by thinning. Since the attached color separation is possible, the color components A, B, and C of the input images A, B, and C are converted into new color components X that suppress deterioration of image quality when sub-sampling is performed. , Y and Z, respectively.
Further, the color conversion unit 1 performs a process of simulating subsampling on the input images A, B, and C, so that color components that are lost when subsampling is performed can be extracted in advance. By generating a color conversion matrix M from this result and multiplying the input images A, B and C, it is possible to perform color conversion that minimizes information loss when subsampling is performed.

  Further, since the pixel thinning means 60 and 70 can thin out the color components of low importance from the new color components X, Y and Z and not thin out the color components of high importance, good image quality with little deterioration. Images can be generated.

  In this way, the color sub-sampling device 2 converts the color components A, B, and C, which are easily lost due to the color sub-sampling, into the color components X, Y, and Z that are not easily lost, and further, the original color component A When the image X and the thinned images Y ′ and Z ′ are generated with appropriate pixel thinning according to the conspicuous loss of B, C and C, and these are coded by an image coding device (not shown) In addition, an encoded image in which information loss is suppressed can be obtained.

  Although the pixel thinning unit 11 of the loss component extraction unit 10 and the pixel thinning units 60 and 70 in the color conversion unit 1 are completely the same, the present invention is not limited to this. For example, the filter coefficient D (i) of the pixel decimation means 11 and the filter coefficient D (i) of the pixel decimation means 60 and 70 do not necessarily match. For example, the filter coefficient D (i) of the pixel decimation means 11 may be approximately replaced with a filter coefficient D (i) of the pixel decimation means 60, 70 that allows simpler calculation. For example, when the filter coefficient D (i) of the pixel decimation means 60 and 70 is a multiple of 1/256, the filter coefficient D (i) of the pixel decimation means 11 is set to a multiple of 1/16 that approximates it. Can be considered. Conversely, for example, the filter coefficient D (i) of the pixel decimation means 60 and 70 is approximately replaced with a filter coefficient D (i) of the pixel decimation means 11 that allows simpler calculation. Also good.

  Further, for example, the tap length of the filter coefficient D (i) of the pixel thinning means 11 and the tap length of the filter coefficient D (i) of the pixel thinning means 60 and 70 do not necessarily have to match. For example, the tap length of the filter coefficient D (i) of the pixel decimation unit 11 is set to 5, the tap length of the filter coefficient D (i) of the pixel decimation units 60 and 70 is set to 7, and the filter coefficient D (i of the pixel decimation unit 11 is set. ) May be made shorter or larger than the tap length of the filter coefficient D (i) of the pixel thinning means 60, 70. However, it is preferable that the pixel thinning unit 11 and the pixel thinning units 60 and 70 are completely the same because the pixel thinning unit 11 can more accurately extract a loss component during sub-sampling of the input image A. .

  Further, for example, the color conversion means (color conversion apparatus) 1 or the color sub-sampling apparatus 2 can also realize each means as a function program in a computer. It is also possible to operate as a sampling program.

  In the first embodiment described above, the color sub-sampling device 2 including the color conversion means (color conversion device) 1 has been described. However, the color conversion means (color conversion device) 1 or the color conversion program is independent. Of course it is good.

<Second embodiment>
[Configuration of color sub-sampling device]
Next, color conversion means (color conversion device) 1A and color subsampling device 2A according to a second embodiment of the present invention will be described with reference to FIG. A color sub-sampling device 2A according to the second embodiment includes color conversion means (color conversion device) 1A and pixel thinning means 60 and 70.

  The color sub-sampling device 2A according to the second embodiment is different from the color sub-sampling device 2 according to the first embodiment in that the configuration of the color conversion unit is different. Specifically, the color conversion unit 1A is different in that it further includes gain multiplication units 81, 82, and 83 in addition to the components of the color conversion unit 1. For this reason, other identical components are denoted by the same reference numerals, and redundant description is omitted. Hereinafter, the configuration of the color conversion unit 1A will be mainly described.

[Configuration of color conversion means]
As shown in FIG. 4, the color conversion unit 1A includes loss component extraction units 10, 20, and 30, gain multiplication units 81, 82, and 83, principal component analysis unit 40, and color space conversion unit 50. ing.

The gain multiplication means 81, 82 and 83 are provided corresponding to the difference images P, Q and R generated by the loss component extraction means 10, 20 and 30, respectively. The gain multiplication means 81, 82, and 83 are previously provided in accordance with the importance of each of the color components A, B, and C constituting the input images A, B, and C from which the difference images P, Q, and R are generated. Difference images P ′, Q ′, and R ′ are generated by multiplying the corresponding difference images P, Q, and R by the set predetermined gains K _A , K _B, and K _C , respectively.

The gain multiplication means 81, 82 and 83 multiply the difference images P, Q and R generated by the loss component extraction means 10, 20 and 30 by predetermined gains K _A , K _B and K _C to obtain a difference image P ′. , Q ′ and R ′. For the gains K _A , K _B and K _C , for example, in consideration of human visual characteristics, a large value is set for a color component with high sensitivity, and a small value is set for a color component with low sensitivity. In other words, color components that tend to feel image quality degradation due to loss due to sub-sampling are highly important, so a large gain is set so that the color components are not lost, and image quality degradation occurs even if they are lost due to sub-sampling. Since color components that are difficult to feel are less important, a small gain can be set. In this way, the color components A, B, and C are weighted according to the importance so that the color components with high sensitivity can be prevented from being lost.

Here, gain multiplying means is provided corresponding to each of the loss component extracting means 10, 20 and 30, and each of the difference images P, Q and R generated by the loss component extracting means 10, 20 and 30 is predetermined. The gains K _A , K _B , and K _C are multiplied to generate the difference images P ′, Q ′, and R ′, but the present invention is not limited to this. For example, it is only necessary to provide gain multiplication means corresponding to any one or more of the color components A, B, and C.
The difference images P ′, Q ′ and R ′ thus generated are output to the principal component analysis means 40.

As shown in the following equation (13), the principal component analyzing means 40 receives the vector S (x at each pixel position of the difference images P ′, Q ′ and R ′ input from the gain multiplying means 81, 82 and 83. , Y), a principal component analysis is performed to obtain a coupling coefficient vector w _k .

[Operation of color sub-sampling device]
Next, the operation of the color subsampling apparatus 2A will be described with reference to FIG.
In the color subsampling apparatus 2A, the operations in steps S111 to S114 are the same as the operations in steps S101 to S104 of the color subsampling apparatus 2 described with reference to FIG.
The color sub-sampling device 2A multiplies the difference images P, Q, and R generated in step S114 by predetermined gains K _A , K _B, and K _C by the gain multiplication means 81, 82, and 83 to obtain the difference image P ′. , Q ′ and R ′ are generated (step S115). The gain multiplication means 81, 82 and 83 output the difference images P ′, Q ′ and R ′ to the principal component analysis means 40. In step S115, the color sub-sampling device 2A inputs the difference images P ′, Q ′, and R ′ from the gain multiplication means 81, 82, and 83 by the principal component analysis means 40, and the difference images P ′, Q ′, and R are input. Principal component analysis is performed on the vector S (x, y) at each pixel position of '(see the above equation (13)) to obtain a coupling coefficient vector w _k for each principal component. The operations in steps S116 to S120 are the same as the operations in steps S105 to S109 of the color sub-sampling apparatus 2 described with reference to FIG.

  According to the color sub-sampling apparatus 2A according to the second embodiment, the gain multiplying means 81, 82, and 83 of the color converting means 1A visually corresponds to any one or more of the color components A, B, and C. Multiply one or more of the difference images P, Q, and R corresponding to the color components A, B, and C, respectively, by a predetermined gain set according to the importance of each color component of the input image, such as the sensitivity Thus, the importance can be reflected in the result of color conversion. Therefore, the pixel thinning means 60 and 70 can perform thinning processing that reflects this importance, that is, important color components are not thinned out, and unimportant color components can be thinned out, so that there is little deterioration. An image with high image quality can be generated.

1, 1A color conversion device 2, 2A color sub-sampling device 10, 20, 30 loss component extraction means 40 principal component analysis means 50 color space conversion means 60, 70 pixel thinning means 81, 82, 83 gain multiplication means

Claims (6)

A color conversion device that converts each color component of a color image into a new color component that reduces image quality degradation during subsampling,
Loss component extraction means for extracting loss components lost during sub-sampling from each input image consisting of the color components of the color image;
Principal component analysis means for performing principal component analysis on the pixel value of each loss component extracted by the loss component extraction means;
Each color component is converted into the new color component in a color space different from the color space of each color component by a color conversion matrix based on a coupling coefficient obtained by principal component analysis by the principal component analysis means, A color conversion device comprising: color space conversion means for generating an image composed of each of the new color components. The loss component extracting means is
Pixel thinning means for thinning out a predetermined amount of pixels of each input image according to a predetermined rule;
Interpolating means for generating interpolated images obtained by returning the respective input images in which a predetermined amount of pixels have been thinned out by the pixel thinning means to their original resolution;
The color conversion apparatus according to claim 1, further comprising: a subtracting unit that generates a difference image between each input image and each interpolation image as the loss component.   For any one or more of the loss components extracted by the loss component extraction means, a gain set in advance for the color component of the input image from which the loss component is extracted is obtained. 3. The color conversion apparatus according to claim 1, further comprising gain multiplication means for multiplying. The color conversion device according to any one of claims 1 to 3,
Among the images composed of the new color components generated by the color conversion device, provided corresponding to a predetermined number of the images from the smaller eigenvalues obtained by the principal component analysis means, the corresponding A color subsampling device comprising: a pixel thinning means for thinning out a predetermined amount of pixels of an image to be performed according to a predetermined rule. In order to convert each color component of a color image into a new color component that reduces image quality degradation during subsampling,
Loss component extraction means for extracting loss components lost during sub-sampling from each input image consisting of the color components of the color image;
Principal component analysis means for performing principal component analysis on the pixel value of each loss component extracted by the loss component extraction means;
Each color component is converted into the new color component in a color space different from the color space of each color component by a color conversion matrix based on a coupling coefficient obtained by principal component analysis by the principal component analysis means, A color conversion program for functioning as color space conversion means for generating an image composed of each of the new color components. Each color component of a color image is converted into a new color component that reduces image quality degradation during sub-sampling, and then a predetermined amount of pixels is thinned out from the image composed of the new color component according to a predetermined rule. To the computer,
Loss component extraction means for extracting loss components lost during sub-sampling from each input image consisting of the color components of the color image;
Principal component analysis means for performing principal component analysis on the pixel value of each loss component extracted by the loss component extraction means;
Each color component is converted into the new color component in a color space different from the color space of each color component by a color conversion matrix based on a coupling coefficient obtained by principal component analysis by the principal component analysis means, Color space conversion means for generating an image composed of each of the new color components,
Among the images composed of the new color components generated by the color space conversion means, provided corresponding to a predetermined number of the images from the smaller eigenvalue obtained by the principal component analysis means, A color sub-sampling program for functioning as a pixel thinning means for thinning a predetermined amount of pixels of a corresponding image according to a predetermined rule.

Images